NetFlow data retrieval method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, in particular to a NetFlow data retrieval method and device, equipment and a medium. According to the method, a NetFlow data retrieval request is obtained, the retrieval request comprises an address pair condition, and an address pair comprises a first source IP address and a first destination IP address; a Z-Order coding mode is adopted, binary sequences corresponding to the first source IP address and the first destination IP address are arranged in a bit-by-bit staggered mode, and a target Z-value is generated; matching the target Z-value with the Z-value interval of the data block corresponding to each sub-data set in the current data set, and determining a target data block; and based on the NetFlow data retrieval request, performing retrieval from the NetFlow data corresponding to the target data block to generate a retrieval result. And a Z-Order coding technology is utilized, so that the system can support accurate query based on an IP address pair, and the retrieval efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a NetFlow data retrieval method, apparatus, device, and medium. Background Technology

[0002] In the field of network traffic analysis, NetFlow data serves as the core recorder of network behavior, carrying key information about data flows in the network, including source IP, destination IP, port, protocol, timestamp, etc. Processing and analyzing retrieved NetFlow data is crucial for scenarios such as network security threat detection, refined traffic management for operators, and data center performance optimization.

[0003] With the explosive growth of network scale, the amount of NetFlow data has also grown exponentially. However, traditional NetFlow data retrieval technology usually relies on full scan queries and typically only supports single-dimensional query methods, such as retrieving only source IP or destination IP. It fails to effectively utilize the correlation between IP pairs in the network space, and the retrieval process requires traversing a large amount of irrelevant data, resulting in long query times. This cannot meet the needs of applications that are sensitive to response speed, such as real-time security analysis and real-time traffic scheduling.

[0004] Therefore, existing NetFlow data retrieval technologies suffer from the problem of only supporting single-dimensional queries and having low retrieval efficiency. Summary of the Invention

[0005] This application provides a NetFlow data retrieval method, apparatus, device, and medium to address the problem that existing NetFlow data retrieval technologies only support single-dimensional queries and have low retrieval efficiency.

[0006] Firstly, this application provides a NetFlow data retrieval method, the method comprising:

[0007] Obtain a NetFlow data retrieval request, the retrieval request including address pair conditions, the address pair including a first source IP address and a first destination IP address;

[0008] Using Z-Order encoding, the binary sequences corresponding to the first source IP address and the first destination IP address are interleaved bitwise to generate the target Z-value;

[0009] The target Z-value is matched with the Z-value range of the data blocks corresponding to each subset of the current dataset to determine the target data block; wherein, the current dataset is an incremental data set divided according to different time windows, each subset contains NetFlow data collected within the corresponding time window, and the data block corresponding to each subset is obtained by sorting the Z-values ​​corresponding to each NetFlow data in the subset and dividing it according to a preset storage size;

[0010] Based on the NetFlow data retrieval request, a retrieval is performed from the NetFlow data corresponding to the target data block to generate retrieval results.

[0011] In some embodiments, the step of using Z-Order encoding to generate a target Z-value by interleaving the binary sequences corresponding to the first source IP address and the first destination IP address bit by bit includes:

[0012] Based on the IP address type, the first source IP address and the first destination IP address are converted into binary sequences of the same number of bits to obtain the first sequence and the second sequence;

[0013] Using the Z-Order encoding method, the binary bits at the same position in the first sequence and the second sequence are interleaved and concatenated in order from the high bit to the low bit to generate the target sequence;

[0014] The target sequence is numerically transformed to generate the target Z-value.

[0015] In some embodiments, data blocks are generated in the following manner:

[0016] In response to the arrival of any preset time window, determine the NetFlow data collected within the time window; wherein each piece of NetFlow data contains a second source IP address and a second destination IP address; wherein the time window includes a long-cycle time window and a short-cycle time window;

[0017] For each piece of NetFlow data, Z-Order encoding is used to interleave the binary sequences corresponding to the second source IP address and the second destination IP address of the NetFlow data bit by bit to generate a Z-value.

[0018] The Z-value corresponding to each NetFlow data within the time window is sorted according to a preset sorting method. The NetFlow data corresponding to each sorted Z-value is divided according to a preset storage size to generate at least one data block. Each data block contains at least one NetFlow data and the Z-value corresponding to the at least one NetFlow data.

[0019] In some embodiments, after generating the data block, the method further includes:

[0020] Based on the Z-value range corresponding to each data block, index information for each data block corresponding to the time window is constructed; wherein, the index key in the index information is the minimum Z-value corresponding to the NetFlow data within each data block, and the index value corresponding to the index key is the maximum Z-value corresponding to the NetFlow data within the data block.

[0021] Based on the index information of each data block corresponding to each time window, generate the index information corresponding to the current dataset;

[0022] The step of matching the target Z-value with the Z-value range of data blocks corresponding to each subset of the current dataset to determine the target data block includes:

[0023] For each index information in the current dataset, the target Z-value is compared with the index key in each index information to determine the target index key whose index key is not greater than the target Z-value;

[0024] The target Z-value is compared with the index value corresponding to the target index key to determine that the index value is not less than the target index value of the target Z-value;

[0025] The target data block is determined based on the data block corresponding to the target index value.

[0026] In some embodiments, the step of retrieving data from the NetFlow data corresponding to the target data block based on the NetFlow data retrieval request and generating retrieval results includes:

[0027] The first source IP address in the retrieval request is matched with the second source IP address corresponding to each piece of NetFlow data in the target data block, and the first destination IP address in the retrieval request is matched with the second destination IP address corresponding to each piece of NetFlow data in the target data block.

[0028] From each NetFlow data in the target data block, determine the successfully matched target NetFlow data, wherein the target NetFlow data is the NetFlow data whose second source IP address is the same as the first source IP address and whose second destination IP address is the same as the first destination IP address;

[0029] The target NetFlow data is encapsulated according to a preset format to generate search results, which include the second source IP address and the second destination IP address corresponding to the target NetFlow data.

[0030] In some embodiments, the retrieval request further includes time conditions, which include the start and end times of the NetFlow data, and the index information further includes a time interval for each data block; wherein, the start time of the time interval is the earliest time corresponding to each NetFlow data in the data block, and the end time of the time interval is the latest time corresponding to each NetFlow data in the data block.

[0031] The step of determining the target data block based on the data block corresponding to the target index value includes:

[0032] For any data block corresponding to a target index value, if the first time interval corresponding to the start time and the end time of the data block is within the second time interval corresponding to the start time and the end time of the retrieval request, then the data block is determined as the target data block.

[0033] In some embodiments, the step of retrieving data from the NetFlow data corresponding to the target data block based on the NetFlow data retrieval request and generating retrieval results includes:

[0034] The first source IP address in the retrieval request is matched with the second source IP address corresponding to each piece of NetFlow data in the target data block, and the first destination IP address in the retrieval request is matched with the second destination IP address corresponding to each piece of NetFlow data in the target data block.

[0035] From each NetFlow data in the target data block, determine the successfully matched target NetFlow data. The target NetFlow data is the NetFlow data whose second source IP address is the same as the first source IP address and whose second destination IP address is the same as the first destination IP address. The third time interval corresponding to the start time and the end time is within the second time interval corresponding to the start time and the end time of the retrieval request.

[0036] The target NetFlow data is encapsulated according to a preset format to generate search results. The search results include the second source IP address, the second destination IP address, the start time, and the end time corresponding to the target NetFlow data.

[0037] Secondly, this application provides a NetFlow data retrieval device, the device comprising:

[0038] The acquisition module is used to acquire NetFlow data retrieval requests, the retrieval requests including address pair conditions, the address pair including a first source IP address and a first destination IP address;

[0039] The generation module is used to generate the target Z-value by interleaving the binary sequences corresponding to the first source IP address and the first destination IP address bit by bit using the Z-Order encoding method.

[0040] The determination module is used to match the target Z-value with the Z-value range of the data blocks corresponding to each subset of the current dataset to determine the target data block; wherein, the current dataset is an incremental data set divided according to different time windows, each subset contains NetFlow data collected within the corresponding time window, and the data block corresponding to each subset is obtained by sorting the Z-values ​​corresponding to each NetFlow data in the subset and dividing it according to a preset storage size;

[0041] The retrieval module is used to retrieve data from the NetFlow data corresponding to the target data block based on the NetFlow data retrieval request and generate retrieval results.

[0042] Thirdly, this application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to said at least one processor, wherein:

[0043] The memory stores a computer program that can be executed by at least one processor to enable the at least one processor to perform any of the NetFlow data retrieval methods described above.

[0044] Fourthly, embodiments of this application provide a storage medium in which, when a computer program in the storage medium is executed by a processor of an electronic device, the electronic device is able to perform any of the NetFlow data retrieval methods described above.

[0045] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements any of the NetFlow data retrieval methods described above.

[0046] In this embodiment, a NetFlow data retrieval request is obtained. The retrieval request includes address pair conditions, where each address pair includes a first source IP address and a first destination IP address. Using Z-Order encoding, the binary sequences corresponding to the first source IP address and the first destination IP address are interleaved bit by bit to generate a target Z-value. The target Z-value is matched with the Z-value range of the data blocks corresponding to each subset of the current dataset to determine the target data block. The current dataset is an incremental data set divided according to different time windows. Each subset contains NetFlow data collected within the corresponding time window, and the data block corresponding to each subset is obtained by sorting the Z-values ​​of each NetFlow data in the subset and dividing it according to a preset storage size. Based on the NetFlow data retrieval request, a retrieval is performed from the NetFlow data corresponding to the target data block to generate retrieval results.

[0047] In this embodiment, Z-Order encoding technology is used to merge the key source IP address and destination IP address into a single identifier (Z-value) representing the "connection relationship". This enables the system to natively support precise queries based on the composite key of "IP address pairs", solving the problem that existing technologies only support single-dimensional queries. At the same time, by sorting NetFlow data in different time windows according to Z-value and dividing it into data blocks according to a preset storage size, the target Z-value only needs to be matched with the Z-value range of the data block. This can directly skip a large number of data blocks whose Z-value ranges do not overlap with the target Z-value, greatly reducing disk I / O and the amount of data to be processed, and effectively improving retrieval efficiency.

[0048] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0049] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0050] Figure 1 This is a schematic diagram illustrating an application scenario of a NetFlow data retrieval method provided in an embodiment of this application.

[0051] Figure 2A flowchart illustrating a NetFlow data retrieval method provided in this application embodiment;

[0052] Figure 3 This application provides a schematic diagram of data block partitioning as an embodiment.

[0053] Figure 4 This application provides a schematic diagram of the overall process of NetFlow data retrieval in an embodiment of the present application.

[0054] Figure 5 A schematic diagram illustrating a real-time block index construction process provided in an embodiment of this application;

[0055] Figure 6 This is a schematic diagram of a dynamic merging and optimization process provided in an embodiment of this application;

[0056] Figure 7 This application provides a schematic diagram of a three-level index retrieval process.

[0057] Figure 8 A structural diagram of a NetFlow data retrieval device provided in an embodiment of this application;

[0058] Figure 9 This is a schematic diagram of the hardware structure of an electronic device for implementing the NetFlow data retrieval method, provided as an embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0060] The terms "first" and "second" in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. The term "multiple" in this application can mean at least two, for example, two, three, or more, and the embodiments of this application do not impose limitations.

[0061] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0062] The acquisition, transmission, storage, and use of data in this application all comply with relevant national laws and regulations.

[0063] Before introducing the NetFlow data retrieval method provided in the embodiments of this application, some terms used in the embodiments of this application will be explained below for ease of understanding.

[0064] Z-order (Z-order curve) storage engine: This is a space-filling curve used to convert multidimensional data (such as two-dimensional coordinates) into one-dimensional ordered values ​​while preserving spatial locality, meaning that neighboring points in space are also numerically adjacent after mapping. Its core functionality is achieved by alternately concatenating binary bits of each dimension; for example, interleaving the x and y coordinate bits of a two-dimensional point to generate a one-dimensional Z-value. In this application, this technology is used to map source IP addresses and destination IP addresses (two-dimensional IP pairs) to one-dimensional Z-values, allowing neighboring IP pairs in network space to naturally cluster during storage, thus improving data locality.

[0065] NetFlow is a technology system for network traffic monitoring. It collects and records detailed information about data flows in the network (such as source / destination IP, port, protocol, transmission time, etc.), providing data support for network traffic analysis, security threat identification, and bandwidth resource management. Through standardized data formats and processing flows, this technology enables refined monitoring of network communication behavior and is one of the core technical means to ensure efficient network operation and security protection.

[0066] The design concept of the embodiments of this application is briefly introduced below:

[0067] In the field of network traffic analysis, NetFlow data serves as the core recorder of network behavior, carrying key information about data flows in the network, including source IP, destination IP, port, protocol, timestamp, etc. Processing and analyzing retrieved NetFlow data is crucial for scenarios such as network security threat detection, refined traffic management for operators, and data center performance optimization.

[0068] With the explosive growth of network scale, the amount of NetFlow data has also grown exponentially. However, traditional NetFlow data retrieval technology usually relies on full scan queries and typically only supports single-dimensional query methods, such as retrieving only source IP or destination IP. It fails to effectively utilize the correlation between IP pairs in the network space, and the retrieval process requires traversing a large amount of irrelevant data, resulting in long query times. This cannot meet the needs of applications that are sensitive to response speed, such as real-time security analysis and real-time traffic scheduling.

[0069] Given that existing NetFlow data retrieval technologies only support single-dimensional queries and have low retrieval efficiency, this application provides a NetFlow data retrieval method, apparatus, device, and medium. The method obtains a NetFlow data retrieval request, which includes address pair conditions, where each address pair includes a first source IP address and a first destination IP address. Using Z-Order encoding, the binary sequences corresponding to the first source IP address and the first destination IP address are interleaved bitwise to generate a target Z-value. The target Z-value is matched with the Z-value range of data blocks corresponding to each subset of the current dataset to determine the target data block. The current dataset is an incremental data set divided according to different time windows. Each subset contains NetFlow data collected within the corresponding time window, and each data block is obtained by sorting the Z-values ​​corresponding to each NetFlow data in the subset and dividing it according to a preset storage size. Based on the NetFlow data retrieval request, a retrieval is performed from the NetFlow data corresponding to the target data block to generate retrieval results.

[0070] In this embodiment, Z-Order encoding technology is used to merge the key source IP address and destination IP address into a single identifier (Z-value) representing the "connection relationship". This enables the system to natively support precise queries based on the composite key of "IP address pairs", solving the problem that existing technologies only support single-dimensional queries. At the same time, by sorting NetFlow data in different time windows according to Z-value and dividing it into data blocks according to a preset storage size, the target Z-value only needs to be matched with the Z-value range of the data block. This can directly skip a large number of data blocks whose Z-value ranges do not overlap with the target Z-value, greatly reducing disk I / O and the amount of data to be processed, and effectively improving retrieval efficiency.

[0071] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0072] The NetFlow data retrieval method provided in this application is applicable to computer devices. These computer devices can be servers or network devices; this application does not specifically limit them. See also Figure 1 , Figure 1 This diagram illustrates an application scenario of a NetFlow data retrieval method provided in this embodiment. It includes a user terminal 102 and a server 104, wherein the user terminal 102 communicates with the server 104 via a network. The user terminal 102 sends a NetFlow data retrieval request to the server 104, and the server 104 retrieves the corresponding data from its database based on the retrieval request and returns it to the user terminal 102. The user terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices, and the server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0073] See Figure 2 , Figure 2 A flowchart of a NetFlow data retrieval method provided in an embodiment of this application is shown, the method including the following steps.

[0074] In step 201, a NetFlow data retrieval request is obtained. The retrieval request includes address pair conditions, and the address pair includes a first source IP address and a first destination IP address.

[0075] The address pair condition is used to limit the start and end points of the data flow communication; the source IP address is the IP address of the host that initiates the network communication, and the destination IP address is the IP address of the target host that receives the communication; through this address pair condition, the system can filter out data records that match the specified communication path from massive NetFlow data to meet the needs of subsequent analysis, auditing or security detection. For example, the user can specify the address pair condition "source IP is 192.168.1.100, destination IP is 10.0.0.200" as the search target.

[0076] In step 202, the Z-Order encoding method is used to interleave the binary sequences corresponding to the first source IP address and the first destination IP address bit by bit to generate the target Z-value.

[0077] Considering the widespread adoption of IPv6 and the gradual transition of network environments to a phase where IPv4 and IPv6 coexist for an extended period, traditional technologies exhibit significant shortcomings in handling cross-generational IP address compatibility. Since IPv4 addresses are 32 bits and IPv6 addresses are 128 bits, existing methods typically require designing independent indexing systems for each address type. This significantly increases index construction complexity and fails to leverage the spatial relationships between IP pairs. For example, in a hybrid network environment, hash indexes for IPv4 addresses and indexes for IPv6 addresses cannot be directly correlated. This necessitates additional conversion and adaptation steps for cross-version IP pair queries, increasing system complexity, causing severe loss of data locality, and further exacerbating the decline in retrieval efficiency.

[0078] In view of this, this application introduces the concept of Z-order space filling curve into IP pair processing. By using binary bit interleaving technology, the two-dimensional IP space is mapped into a one-dimensional ordered Z-value, so that spatially adjacent IP pairs are numerically closely adjacent, which greatly improves data locality and solves the problem of locality loss in traditional technology.

[0079] In practice, based on the IP address type, the first source IP address and the first destination IP address can be converted into binary sequences of the same number of bits to obtain the first sequence and the second sequence. Using the Z-Order encoding method, the binary bits at the same position in the first sequence and the second sequence are interleaved and concatenated in the order of the high bits to the low bits to generate the target sequence. The target sequence is then numerically converted to generate the target Z-value.

[0080] Among them, IP address types include IPv4 addresses and IPv6 addresses. Due to the differences between IPv4 and IPv6 addresses, IPv4 addresses can be extended to 128 bits by padding with zeros at the high bits. The binary conversion process of these two address types is described below.

[0081] IPv4 address binary processing:

[0082] Taking the IPv4 address 192.168.1.1 as an example, first, each decimal number is converted to 8 bits of binary form. If it is less than 8 bits, leading zeros are added. For example, 192 corresponds to 11000000, 168 corresponds to 10101000, and 1 corresponds to 0000000. Therefore, the 32-bit binary representation of 192.168.1.1 is (110000000 101010000000001 00000001). Then, 96 zeros are added before this 32-bit binary string to form a fixed-length 128-bit binary string. The high-order zeros correspond to the high 96 bits of the IPv6 address space, ensuring bit alignment with the IPv6 address during Z-order mapping. That is, 192.168.1.1 is converted to (00000000 00000000...00000000). 11000000 101010000000000100000001), the first 96 bits are all zeros, and the last 32 bits are the original IPv4 binary.

[0083] IPv6 address binary processing:

[0084] Since IPv6 addresses are 128 bits, they are actually written using hexadecimal with colons as separators. Therefore, converting an IPv6 address to binary is as simple as converting each segment to 16 bits and then concatenating them; no additional processing is required. For example, the IPv6 address 2001:0db8:85a3::8a2e:0370:7334 consists of 8 groups, each 16 bits (2 bytes), for a total of 128 bits. The double colons :: indicate a continuous segment of abbreviated zero values. 2001 corresponds to 0010000000000001 in binary, 0db8 corresponds to 0000110110111000, and 85a3 corresponds to 10... The binary representation of 00010110100011 and 8a2e is 1000101000101110, the binary representation of 0370 is 0000001101110000, and the binary representation of 7334 is 0111001100110100. Therefore, the binary sequence corresponding to 2001:0db8:85a3::8a2e:0370:7334 is (00100000000000001). 0000110110111000 1000010110100011000000000000000000000000000000000 1000101000101110 00000011011100000111001100110100).

[0085] In practice, based on the IP address type of the address pair in the retrieval request, the first source IP address and the first destination IP address can be converted into binary sequences of the same number of bits, i.e., converted into 128-bit binary sequences, to obtain the first sequence corresponding to the first source IP address and the second sequence corresponding to the first destination IP address. Then, using the Z-Order encoding method, the binary bits at the same position in the first and second sequences are interleaved and concatenated in the order of the high bits to the low bits of the binary sequences to generate the target sequence.

[0086] For example: the first sequence is S = S1S2S3…S… 128 The second sequence is D = D1D2D3…D 128 Interleaving operation, according to Z 2i-1 =S i Z 2i =D i The interleaving rule alternately splices each bit of the two values ​​to generate a 256-bit Z-value binary sequence Z = Z1Z2Z3…Z 256 .

[0087] For example, if the first 4 bits of the first sequence are 1010 and the first 4 bits of the second sequence are 0101, then the first 8 bits of the interleaved Z-value binary sequence are 10011001.

[0088] Finally, the 256-bit Z-value binary sequence can be converted into a decimal or hexadecimal value as the target Z-value.

[0089] In step 203, the target Z-value is matched with the Z-value range of the data blocks corresponding to each subset in the current dataset to determine the target data block; wherein, the current dataset is an incremental data set divided according to different time windows, each subset contains NetFlow data collected within the corresponding time window, and the data block corresponding to each subset is obtained by sorting the Z-values ​​of each NetFlow data in the subset and dividing it according to a preset storage size.

[0090] In practice, data blocks are generated in the following ways:

[0091] In response to the arrival of any preset time window, determine the NetFlow data collected within that time window; wherein each piece of NetFlow data contains a second source IP address and a second destination IP address; wherein the time window includes a long-cycle time window and a short-cycle time window; for example, the long-cycle time window can be 12 hours or 24 hours, and the short-cycle time window can be 5 minutes or 10 minutes.

[0092] For each piece of NetFlow data, Z-Order encoding is used to interleave the binary sequences corresponding to the second source IP address and the second destination IP address of the NetFlow data bit by bit to generate a Z-value.

[0093] The Z-value corresponding to each NetFlow data within the time window is sorted according to a preset sorting method. The NetFlow data corresponding to each sorted Z-value is divided according to a preset storage size to generate at least one data block. Each data block contains at least one NetFlow data and at least one Z-value corresponding to the NetFlow data.

[0094] In specific implementation, taking a short period time window of 10 minutes as an example, every 10 minutes, the Z-value can be calculated for each incoming NetFlow data. The calculation process of Z-value is referred to step 202, which will not be repeated here. The preset sorting method is, for example, ascending sorting. The Z-value corresponding to each NetFlow data within the 10 minutes is sorted in ascending order, and then the data is divided into blocks in units of preset storage size (such as 128MB) to generate data blocks.

[0095] For example, assuming 1280 NetFlow data entries flow in within 10 minutes, 1280 Z-values ​​can be obtained through calculation. If these 1280 Z-values ​​are sorted in ascending order, and each NetFlow data entry is 1MB in size, and the data is divided into blocks of 128MB each, then 10 data blocks can be obtained, with each data block containing 128 NetFlow data entries.

[0096] refer to Figure 3 , Figure 3 This is a schematic diagram of data block partitioning provided in an embodiment of this application. In the diagram, a is the minimum Z-value among 1280 NetFlow data entries, t is the maximum Z-value among 1280 NetFlow data entries, b is the maximum Z-value of data block 1, c is the minimum Z-value of data block 2, d is the maximum Z-value of data block 2, and so on. It can be seen that each data block contains a minimum Z-value and a maximum Z-value. In specific implementation, each data block also includes an HDFS storage path, a timestamp range, etc.

[0097] In practice, taking a long-term time window of 1 hour as an example, every hour, the Z-value of each NetFlow data that flows in within that hour is calculated, and the Z-values ​​of each NetFlow data in that hour are sorted in ascending order. Then, the data is divided into blocks in units of a preset storage size (e.g., 128MB) to generate data blocks. In other words, each NetFlow data that flows in within the 6 short-term time windows of that hour is re-sorted according to the calculated Z-values, and the data in the relevant data blocks are re-merged.

[0098] In practice, to further accelerate retrieval efficiency, after generating data blocks, index information for each data block within a given time window can be constructed based on the Z-value range (i.e., the minimum and maximum Z-values ​​of the data block). The index key in the index information is the minimum Z-value of the NetFlow data within each data block, and the index value corresponding to the index key is the maximum Z-value of the NetFlow data within the data block. Then, based on the index information of each data block within each time window, the index information for the current dataset is generated.

[0099] Using the minimum Z-value as the index key and the maximum Z-value as the corresponding index value, the system can quickly determine whether the target Z-value falls within the range of the data block. The system then summarizes the data block indices within each time window to generate global index information for the current dataset.

[0100] In practical implementation, taking the index information of each data block corresponding to a short-period time window of the current dataset as an example, the index key is the minimum Z-value corresponding to each data block, and the index value corresponding to each index key is the maximum Z-value corresponding to that data block. Therefore, when determining the target data block, the target Z-value can be compared with the index key in each index information, that is, compared with the minimum Z-value corresponding to each data block, to determine the target index key whose index key is not greater than the target Z-value. Then, the target Z-value is compared with the index value corresponding to the target index key to determine the target index value whose index value is not less than the target Z-value. Based on the data block corresponding to the target index value, the target data block is determined.

[0101] For example, the global index information of the current dataset consists of the index information of 5 data blocks, as shown in Table 1.

[0102] Table 1

[0103]

[0104]

[0105] If the target Z-value is 25, then the target index keys are Z-value1, Z-value11, and Z-value21. Among them, the index values ​​Z-value10 and Z-value20 corresponding to index key Z-value1 are both less than the target Z-value, while the index value Z-value30 corresponding to index key Z-value21 is greater than the target Z-value. Therefore, the target index value is Z-value30. Thus, the data block 3 corresponding to Z-value30 is determined as the target data block.

[0106] In practice, if the current dataset is the index information of each data block corresponding to two short-period time windows, and each time window still contains 5 data blocks, the index information is shown in Table 2.

[0107] Table 2

[0108]

[0109] If the target Z-value remains 25, then the target index keys are Z-value1, Z-value11, and Z-value21 for time window 1, and Z-value2, Z-value12, and Z-value22 for time window 2. In time window 1, the index values ​​Z-value10 and Z-value20 corresponding to index key Z-value1 and index key Z-value11 are both less than the target Z-value, while the index value Z-value30 corresponding to index key Z-value21 is greater than the target Z-value. In time window 2, the index value Z-value2 is greater than the target Z-value. The index values ​​Z-value21 corresponding to index key Z-value11 and Z-value12 are both less than the target Z-value, while the index value Z-value31 corresponding to index key Z-value22 is greater than the target Z-value. Therefore, the target index value in time window 1 is Z-value30 and the target index value in time window 2 is Z-value31. Thus, data block 3 corresponding to Z-value30 in time window 1 is determined as the target data block, and data block 8 corresponding to Z-value31 in time window 2 is determined as the target data block. That is, if the current dataset contains index information corresponding to multiple time windows, the determined target data block may not be unique.

[0110] In step 204, based on the NetFlow data retrieval request, a retrieval is performed from the NetFlow data corresponding to the target data block to generate retrieval results.

[0111] In practice, after determining the target data block, precise matching can be performed. This involves matching the first source IP address in the search request with the second source IP address corresponding to each NetFlow data entry in the target data block, and matching the first destination IP address in the search request with the second destination IP address corresponding to each NetFlow data entry in the target data block. Successfully matched target NetFlow data is then identified from each NetFlow data entry in the target data block. Target NetFlow data is defined as NetFlow data whose second source IP address and first source IP address are the same, and whose second destination IP address and first destination IP address are also the same. The target NetFlow data is then encapsulated according to a preset format to generate search results, which include the second source IP address and second destination IP address corresponding to the target NetFlow data.

[0112] In this way, by using Z-Order encoding technology, the key source IP address and destination IP address are merged into a single identifier (Z-value) representing the "connection relationship". This enables the system to natively support precise queries based on the composite key of "IP address pairs", solving the problem that existing technologies only support single-dimensional queries. At the same time, by sorting NetFlow data in different time windows according to Z-value and dividing it into data blocks according to a preset storage size, the target Z-value only needs to be matched with the Z-value range of the data block. This can directly skip a large number of data blocks whose Z-value ranges do not overlap with the target Z-value, greatly reducing disk I / O and the amount of data to be processed, and effectively improving retrieval efficiency.

[0113] In practice, the retrieval request also includes time conditions, which include the start and end times of the NetFlow data. For example, the retrieval request may include NetFlow data with timestamps of [2025-05-12 00:00:00, 2025-05-12 00:10:00]. The index information also includes the time interval for each data block. The start time of the time interval is the earliest time corresponding to each NetFlow data in the data block, and the end time of the time interval is the latest time corresponding to each NetFlow data in the data block.

[0114] Therefore, time intervals can also be combined to filter the time intervals of the data blocks corresponding to the target index value. In specific implementation, if the first time interval corresponding to the start and end times of the data block is within the second time interval corresponding to the start and end times of the retrieval request, then the data block is determined as the target data block, which can further improve the accuracy of retrieval.

[0115] At this point, each NetFlow data in the determined target data block is precisely matched. The determined target NetFlow data that is successfully matched is the NetFlow data whose second source IP address is the same as the first source IP address, whose second destination IP address is the same as the first destination IP address, and whose third time interval corresponding to the start time and end time is within the second time interval corresponding to the start time and end time of the retrieval request. The target NetFlow data is encapsulated according to a preset format, and the generated retrieval results include the second source IP address, second destination IP address, start time, and end time corresponding to the target NetFlow data.

[0116] The above process will be explained below with specific examples.

[0117] See Figure 4 , Figure 4 This application provides a schematic diagram of the overall process of NetFlow data retrieval, which includes the following steps.

[0118] Step 1: First, network devices (switches, etc.) enable the NetFlow function to capture traffic and perform five-tuple aggregation. Then, the collected NetFlow is sent to the designated collector via UDP protocol. The collector listens on a specific port to receive and parse the traffic data, converts it into JSON structured data, and then extends the IPv4 address to 128 bits (padding the first 96 bits with zeros) to unify it with the IPv6 address into a 128-bit binary format.

[0119] Step 2: Convert source and destination IP pairs into one-dimensional Z-values ​​using binary bit interleaving technology: First, pad the high-order bits of IPv4 to extend it to 128 bits, then process it uniformly with IPv6. Alternately concatenate the binary bits of both to generate a 256-bit Z-value, achieving numerical aggregation of spatially adjacent IP pairs. Based on Flink's short-cycle time window: A 10-minute sliding window is used for micro-batch processing of NetFlow data. After sorting by Z-value in ascending order, the data is divided into 128MB blocks. Metadata such as the Z-value range (minimum Z-value: Z_min / maximum Z-value: Z_max) for each block is calculated to build a block-level index. Block data is stored in HDFS, with paths generated according to "Z-value range + timestamp," and metadata is written to the index table in real time.

[0120] Among them, see Figure 5 , Figure 5 This is a schematic diagram of a real-time block index construction process provided in an embodiment of this application, including three parts: IP address standardization, Z-value calculation logic, and block storage.

[0121] This method lays the foundation for subsequent block-level fast filtering and efficient retrieval through real-time encoding, ordered block partitioning, and index construction, achieving localized optimized storage and dynamic index management of large-scale NetFlow data.

[0122] Step 3: Upon reaching the long-term time window, the dynamic merging and optimization process is automatically initiated. (See [link to relevant documentation]). Figure 6 , Figure 6 This is a schematic diagram of a dynamic merging and optimization process provided in an embodiment of this application. Data blocks within a long-term time window are merged, and the NetFlow data corresponding to each data block is re-sorted in ascending order by Z-value and then divided into 128MB blocks. Metadata such as the Z-value range (minimum Z-value: Z_min / maximum Z-value: Z_max) of each block is calculated to construct a block-level index, which helps improve retrieval efficiency.

[0123] Step 4: When performing a three-level index retrieval, see [link / reference]. Figure 7 , Figure 7 This document provides a schematic diagram of a three-level index retrieval process. The source IP / destination IP in the search criteria is converted into a Z-value, and combined with time to generate a multi-dimensional interval. The index is used to quickly filter target data blocks, filtering out 99% of irrelevant blocks to reduce IO load. Next, target data blocks in HDFS are loaded in parallel according to the storage path. Finally, MapReduce is used to parallelize the target data blocks, comparing each record against IP pairs and timestamps for precise intra-block matching. The search results are then merged and returned, achieving a second-level retrieval response for billions of NetFlow data points.

[0124] Based on the same inventive concept, this application provides a NetFlow data retrieval device. The principle of the NetFlow data retrieval device in solving the problem is similar to that of the NetFlow data retrieval method described above. Therefore, the implementation of the NetFlow data retrieval device can refer to the implementation of the NetFlow data retrieval method, and the repeated parts will not be described again.

[0125] Please refer to Figure 8 , Figure 8 A structural diagram of a NetFlow data retrieval device provided in this application embodiment is shown. The device includes:

[0126] The acquisition module 801 is used to acquire a NetFlow data retrieval request, the retrieval request including address pair conditions, the address pair including a first source IP address and a first destination IP address;

[0127] The generation module 802 is used to generate a target Z-value by interleaving the binary sequences corresponding to the first source IP address and the first destination IP address bit by bit using the Z-Order encoding method.

[0128] The determination module 803 is used to match the target Z-value with the Z-value range of the data blocks corresponding to each subset of the current dataset to determine the target data block; wherein, the current dataset is an incremental data set divided according to different time windows, each subset contains NetFlow data collected within the corresponding time window, and the data block corresponding to each subset is obtained by sorting the Z-values ​​corresponding to each NetFlow data in the subset and dividing it according to a preset storage size;

[0129] The retrieval module 804 is used to retrieve data from the NetFlow data corresponding to the target data block based on the NetFlow data retrieval request and generate retrieval results.

[0130] In some embodiments, the generation module 802 is specifically used for:

[0131] Based on the IP address type, the first source IP address and the first destination IP address are converted into binary sequences of the same number of bits to obtain the first sequence and the second sequence;

[0132] Using the Z-Order encoding method, the binary bits at the same position in the first sequence and the second sequence are interleaved and concatenated in order from the high bit to the low bit to generate the target sequence;

[0133] The target sequence is numerically transformed to generate the target Z-value.

[0134] In some embodiments, a partitioning module 805 is further included for generating data blocks in the following manner:

[0135] In response to the arrival of any preset time window, determine the NetFlow data collected within the time window; wherein each piece of NetFlow data contains a second source IP address and a second destination IP address; wherein the time window includes a long-cycle time window and a short-cycle time window;

[0136] For each piece of NetFlow data, Z-Order encoding is used to interleave the binary sequences corresponding to the second source IP address and the second destination IP address of the NetFlow data bit by bit to generate a Z-value.

[0137] The Z-value corresponding to each NetFlow data within the time window is sorted according to a preset sorting method. The NetFlow data corresponding to each sorted Z-value is divided according to a preset storage size to generate at least one data block. Each data block contains at least one NetFlow data and the Z-value corresponding to the at least one NetFlow data.

[0138] In some embodiments, it also includes:

[0139] The indexing module 806 is used to construct index information for each data block corresponding to the time window based on the Z-value range corresponding to each data block after the partitioning module 805 generates the data blocks; wherein, the index key in the index information is the minimum Z-value corresponding to the NetFlow data in each data block, and the index value corresponding to the index key is the maximum Z-value corresponding to the NetFlow data in the data block.

[0140] Based on the index information of each data block corresponding to each time window, generate the index information corresponding to the current dataset;

[0141] The determining module 803 is specifically used for:

[0142] For each index information in the current dataset, the target Z-value is compared with the index key in each index information to determine the target index key whose index key is not greater than the target Z-value;

[0143] The target Z-value is compared with the index value corresponding to the target index key to determine that the index value is not less than the target index value of the target Z-value;

[0144] The target data block is determined based on the data block corresponding to the target index value.

[0145] In some embodiments, the retrieval module 804 is specifically used for:

[0146] The first source IP address in the retrieval request is matched with the second source IP address corresponding to each piece of NetFlow data in the target data block, and the first destination IP address in the retrieval request is matched with the second destination IP address corresponding to each piece of NetFlow data in the target data block.

[0147] From each NetFlow data in the target data block, determine the successfully matched target NetFlow data, wherein the target NetFlow data is the NetFlow data whose second source IP address is the same as the first source IP address and whose second destination IP address is the same as the first destination IP address;

[0148] The target NetFlow data is encapsulated according to a preset format to generate search results, which include the second source IP address and the second destination IP address corresponding to the target NetFlow data.

[0149] In some embodiments, the retrieval request further includes time conditions, which include the start and end times of the NetFlow data, and the index information further includes a time interval for each data block; wherein, the start time of the time interval is the earliest time corresponding to each NetFlow data in the data block, and the end time of the time interval is the latest time corresponding to each NetFlow data in the data block.

[0150] The determining module 803 is specifically used for:

[0151] For any data block corresponding to a target index value, if the first time interval corresponding to the start time and the end time of the data block is within the second time interval corresponding to the start time and the end time of the retrieval request, then the data block is determined as the target data block.

[0152] In some embodiments, the retrieval module 804 is specifically used for:

[0153] The first source IP address in the retrieval request is matched with the second source IP address corresponding to each piece of NetFlow data in the target data block, and the first destination IP address in the retrieval request is matched with the second destination IP address corresponding to each piece of NetFlow data in the target data block.

[0154] From each NetFlow data in the target data block, determine the successfully matched target NetFlow data. The target NetFlow data is the NetFlow data whose second source IP address is the same as the first source IP address and whose second destination IP address is the same as the first destination IP address. The third time interval corresponding to the start time and the end time is within the second time interval corresponding to the start time and the end time of the retrieval request.

[0155] The target NetFlow data is encapsulated according to a preset format to generate search results. The search results include the second source IP address, the second destination IP address, the start time, and the end time corresponding to the target NetFlow data.

[0156] The module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other division methods are possible. Furthermore, the functional modules in each embodiment of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. Coupling between modules can be achieved through interfaces, typically electrical communication interfaces, but mechanical interfaces or other types of interfaces are also possible. Therefore, modules described as separate components may or may not be physically separate; they can be located in one place or distributed across different locations on the same or different devices. The integrated modules described above can be implemented in hardware or as software functional modules.

[0157] Having introduced the NetFlow data retrieval method and apparatus according to exemplary embodiments of this application, we will now introduce an electronic device according to another exemplary embodiment of this application.

[0158] The following reference Figure 9 To describe an electronic device 130 implemented according to this embodiment of the present application. Figure 9 The electronic device 130 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0159] like Figure 9 As shown, the electronic device 130 is presented in the form of a general-purpose electronic device. The components of the electronic device 130 may include, but are not limited to: at least one processor 131, at least one memory 132, and a bus 133 connecting different system components (including memory 132 and processor 131).

[0160] Bus 133 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0161] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0162] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0163] Electronic device 130 can also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 130, and / or with any device that enables electronic device 130 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 135. Furthermore, electronic device 130 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 136. As shown, network adapter 136 communicates with other modules used in electronic device 130 via bus 133. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0164] In an exemplary embodiment, the electronic device of this application may include at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, it enables the at least one processor to perform the steps of any NetFlow data retrieval method provided in the embodiments of this application.

[0165] In an exemplary embodiment, a storage medium is also provided, which enables the electronic device to execute the NetFlow data retrieval method described above when a computer program in the storage medium is executed by a processor of the electronic device. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0166] In an exemplary embodiment, a computer program product is also provided, which, when executed by an electronic device, enables the electronic device to implement any of the NetFlow data retrieval methods provided in this application.

[0167] It should be noted that although several modules or sub-modules of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0168] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0171] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, then this application also includes such modifications and variations.

Claims

1. A NetFlow data retrieval method, characterized in that, include: Obtain a NetFlow data retrieval request, the retrieval request including address pair conditions, the address pair including a first source IP address and a first destination IP address; Using Z-Order encoding, the binary sequences corresponding to the first source IP address and the first destination IP address are interleaved bitwise to generate the target Z-value; The target Z-value is matched with the Z-value range of the data blocks corresponding to each subset of the current dataset to determine the target data block; wherein, the current dataset is an incremental data set divided according to different time windows, each subset contains NetFlow data collected within the corresponding time window, and the data block corresponding to each subset is obtained by sorting the Z-values ​​corresponding to each NetFlow data in the subset and dividing it according to a preset storage size; Based on the NetFlow data retrieval request, a retrieval is performed from the NetFlow data corresponding to the target data block to generate retrieval results.

2. The method as described in claim 1, characterized in that, The method employing Z-Order encoding, which involves interleaving the binary sequences corresponding to the first source IP address and the first destination IP address bit by bit to generate the target Z-value, includes: Based on the IP address type, the first source IP address and the first destination IP address are converted into binary sequences of the same number of bits to obtain the first sequence and the second sequence; Using the Z-Order encoding method, the binary bits at the same position in the first sequence and the second sequence are interleaved and concatenated in order from the high bit to the low bit to generate the target sequence; The target sequence is numerically transformed to generate the target Z-value.

3. The method as described in claim 1, characterized in that, Data blocks are generated in the following manner: In response to the arrival of any preset time window, determine the NetFlow data collected within the time window; wherein each piece of NetFlow data contains a second source IP address and a second destination IP address; wherein the time window includes a long-cycle time window and a short-cycle time window; For each piece of NetFlow data, Z-Order encoding is used to interleave the binary sequences corresponding to the second source IP address and the second destination IP address of the NetFlow data bit by bit to generate a Z-value. The Z-value corresponding to each NetFlow data within the time window is sorted according to a preset sorting method. The NetFlow data corresponding to each sorted Z-value is divided according to a preset storage size to generate at least one data block. Each data block contains at least one NetFlow data and the Z-value corresponding to the at least one NetFlow data.

4. The method as described in claim 3, characterized in that, After generating the data block, the process also includes: Based on the Z-value range corresponding to each data block, index information for each data block corresponding to the time window is constructed; wherein, the index key in the index information is the minimum Z-value corresponding to the NetFlow data within each data block, and the index value corresponding to the index key is the maximum Z-value corresponding to the NetFlow data within the data block. Based on the index information of each data block corresponding to each time window, generate the index information corresponding to the current dataset; The step of matching the target Z-value with the Z-value range of data blocks corresponding to each subset of the current dataset to determine the target data block includes: For each index information in the current dataset, the target Z-value is compared with the index key in each index information to determine the target index key whose index key is not greater than the target Z-value; The target Z-value is compared with the index value corresponding to the target index key to determine that the index value is not less than the target index value of the target Z-value; The target data block is determined based on the data block corresponding to the target index value.

5. The method as described in claim 4, characterized in that, The step of retrieving data from the NetFlow data corresponding to the target data block based on the NetFlow data retrieval request and generating retrieval results includes: The first source IP address in the retrieval request is matched with the second source IP address corresponding to each piece of NetFlow data in the target data block, and the first destination IP address in the retrieval request is matched with the second destination IP address corresponding to each piece of NetFlow data in the target data block. From each NetFlow data in the target data block, determine the successfully matched target NetFlow data, wherein the target NetFlow data is the NetFlow data whose second source IP address is the same as the first source IP address and whose second destination IP address is the same as the first destination IP address; The target NetFlow data is encapsulated according to a preset format to generate search results, which include the second source IP address and the second destination IP address corresponding to the target NetFlow data.

6. The method as described in claim 4, characterized in that, The retrieval request also includes time conditions, which include the start and end times of the NetFlow data. The index information also includes the time interval for each data block. The start time of the time interval is the earliest time corresponding to each NetFlow data in the data block, and the end time of the time interval is the latest time corresponding to each NetFlow data in the data block. The step of determining the target data block based on the data block corresponding to the target index value includes: For any data block corresponding to a target index value, if the first time interval corresponding to the start time and the end time of the data block is within the second time interval corresponding to the start time and the end time of the retrieval request, then the data block is determined as the target data block.

7. The method as described in claim 6, characterized in that, The step of retrieving data from the NetFlow data corresponding to the target data block based on the NetFlow data retrieval request and generating retrieval results includes: The first source IP address in the retrieval request is matched with the second source IP address corresponding to each piece of NetFlow data in the target data block, and the first destination IP address in the retrieval request is matched with the second destination IP address corresponding to each piece of NetFlow data in the target data block. From each NetFlow data in the target data block, determine the successfully matched target NetFlow data. The target NetFlow data is the NetFlow data whose second source IP address is the same as the first source IP address and whose second destination IP address is the same as the first destination IP address. The third time interval corresponding to the start time and the end time is within the second time interval corresponding to the start time and the end time of the retrieval request. The target NetFlow data is encapsulated according to a preset format to generate search results. The search results include the second source IP address, the second destination IP address, the start time, and the end time corresponding to the target NetFlow data.

8. A NetFlow data retrieval device, characterized in that, include: The acquisition module is used to acquire NetFlow data retrieval requests, the retrieval requests including address pair conditions, the address pair including a first source IP address and a first destination IP address; The generation module is used to generate the target Z-value by interleaving the binary sequences corresponding to the first source IP address and the first destination IP address bit by bit using the Z-Order encoding method. The determination module is used to match the target Z-value with the Z-value range of the data blocks corresponding to each subset of the current dataset to determine the target data block; wherein, the current dataset is an incremental data set divided according to different time windows, each subset contains NetFlow data collected within the corresponding time window, and the data block corresponding to each subset is obtained by sorting the Z-values ​​corresponding to each NetFlow data in the subset and dividing it according to a preset storage size; The retrieval module is used to retrieve data from the NetFlow data corresponding to the target data block based on the NetFlow data retrieval request and generate retrieval results.

9. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to said at least one processor, wherein: The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-7.

10. A storage medium, characterized in that, When the computer program in the storage medium is executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-7.

Images