Octree-based network attack traffic spatial index analysis method and device

Abstract

The present application discloses a network attack traffic spatial index analysis method and device based on an octree, and the method is implemented by: constructing a network traffic data set, the network traffic data set includes network traffic data to be indexed; constructing an octree structure based on the network traffic data to be indexed, mapping the network traffic data to be indexed to the corresponding nodes of the octree structure, and obtaining an octree spatial index structure; based on the octree spatial index structure, performing traffic distribution statistics and path tracking, obtaining the traffic category and path table corresponding to each node in the octree spatial index structure, and performing traffic analysis based on the traffic category and path table. In the embodiment of the present application, the network traffic data is efficiently spatially divided and indexed by constructing an octree structure, and by constructing an efficient spatial index structure, rapid retrieval, analysis and processing of large-scale network traffic data are achieved, thereby significantly improving network monitoring and management efficiency.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to an octree-based network attack traffic spatial index analysis method and device. Background Art

[0002] With the rapid development of the Internet, network traffic data has exploded. How to efficiently process and analyze these massive data has become an important challenge in the field of network management and security. Existing network traffic analysis methods mostly use traditional data structures such as hash tables, B-trees, and KD trees for indexing, but these methods have limitations in processing three-dimensional spatial data. The specific limitations are: Insufficient spatial division: When processing network traffic data, traditional methods often use simple two-dimensional or one-dimensional division methods, ignoring the spatial distribution characteristics of traffic, resulting in insufficient analysis accuracy. Low indexing efficiency: For large-scale network traffic data, traditional index structures are inefficient in data retrieval, updating, and maintenance, and it is difficult to meet real-time requirements. This makes it difficult to directly apply to network traffic data with spatial characteristics. Summary of the invention

[0003] Based on this, it is necessary to provide an octree-based network attack traffic spatial index analysis method, device, computer equipment and storage medium to address the above technical problems, so as to solve at least one problem existing in the above-mentioned prior art.

[0004] In a first aspect, a network attack traffic spatial index analysis method based on an octree is provided, comprising:

[0005] Constructing a network traffic data set, wherein the network traffic data set includes network traffic data to be indexed;

[0006] Based on the network traffic data to be indexed, an octree structure is constructed, and the network traffic data to be indexed is mapped to corresponding nodes of the octree structure to obtain an octree spatial index structure;

[0007] Based on the octree spatial index structure, traffic distribution statistics and path tracking are performed to obtain the traffic category and path table corresponding to each node in the octree spatial index structure, so as to perform traffic analysis based on the traffic category and path table.

[0008] In one embodiment, the traffic distribution statistics and path tracking based on the octree spatial index structure include:

[0009] Traversing each node of the octree structure, when the accessed node is a leaf node, counting the network traffic data in the leaf node, classifying the counted network traffic data, and determining the traffic category, wherein the traffic category includes a normal traffic probability and an abnormal traffic probability;

[0010] Determine a traffic path, the traffic path including a source IP and a destination IP, take a node corresponding to the source IP as a starting node, search for a next hop node in the octree structure based on the traffic path, and search for the next hop node again based on the found next hop node until a node corresponding to the destination IP is found, thereby generating the path table.

[0011] In one embodiment, mapping the to-be-indexed network traffic data to corresponding nodes of the octree structure includes:

[0012] Extracting the traffic characteristics corresponding to each of the network traffic data to be indexed;

[0013] Construct a mapping function for each traffic feature;

[0014] Converting the traffic characteristics into coordinate points in a multidimensional space based on the mapping function;

[0015] Based on the coordinate points, the network traffic data to be indexed is mapped to corresponding nodes of the octree structure.

[0016] In one embodiment, the traffic characteristics include: traffic size, number of data packets, average data packet size, source IP address and destination IP address, and the conversion of the traffic characteristics into coordinate points in a multidimensional space based on the mapping function includes:

[0017] Hash values ​​are calculated for the source IP address and the destination IP address respectively to obtain corresponding hash values, corresponding weight coefficients are assigned to the hash values, and modular operations are performed to obtain corresponding coordinate points; corresponding weight coefficients are assigned to the traffic size, number of data packets, and average data packet size respectively, and modular operations are performed to obtain corresponding coordinate points.

[0018] In one embodiment, after obtaining the octree spatial index structure, the following steps are performed:

[0019] When there is new network flow data, determining the spatial region to which the new network flow data belongs based on the attributes of the new network flow data;

[0020] Traversing the octree structure to determine whether there is a target node corresponding to the spatial area to which the new network traffic data belongs;

[0021] If yes, add the index information of the new network traffic data to the target node, and update the target node;

[0022] If not, a new node is created, and the index information of the new network traffic data is added to the new node.

[0023] In one embodiment, after obtaining the octree spatial index structure, the following steps are performed:

[0024] When the network traffic data changes, the index information of the node to which the network traffic data belongs is updated, or the node to which the network traffic data belongs is split or merged to update the index information;

[0025] When the network traffic data is expired or invalid, the invalid nodes in the octree structure that contain the expired or invalid network traffic data are determined; if all network traffic data contained in the invalid nodes are invalid, the invalid nodes and their corresponding child nodes are deleted; otherwise, the index information of the expired or invalid data is deleted.

[0026] In one embodiment, the network traffic data to be indexed includes spatial location information of each network traffic data point to be indexed, and constructing an octree structure based on the network traffic data to be indexed includes:

[0027] Based on the spatial location information of each network traffic data point to be indexed, construct an initial cube, wherein the initial cube includes all network traffic data points to be indexed;

[0028] Using the initial cube as the root node of the octree structure;

[0029] The space represented by the root node is recursively partitioned to generate the octree structure.

[0030] In one embodiment, the step of constructing an initial cube based on the spatial location information of each network traffic data point to be indexed includes:

[0031] In each dimension of the multidimensional space, determine the maximum value coordinate point and the minimum value coordinate point among all the network traffic data points to be indexed;

[0032] Based on the maximum value coordinate point and the minimum value coordinate point, the coordinates of each vertex of the initial cube are constructed; based on the coordinates of each vertex, the initial cube is constructed.

[0033] In one embodiment, recursively partitioning the space corresponding to the root node to generate the octree structure includes:

[0034] In each recursive partition, the space represented by the current node is divided into eight sub-cubes, each of which corresponds to a sub-node;

[0035] Determining whether each of the sub-cubes satisfies a termination condition;

[0036] If so, the child node corresponding to the sub-cube that meets the termination condition is used as a leaf node, and the corresponding traffic data is stored;

[0037] If not, the sub-cubes that do not meet the termination condition are recursively split again until the maximum recursion depth is reached or the amount of data inside the sub-cube is lower than a preset threshold, thereby obtaining the octree structure.

[0038] In a second aspect, an octree-based network attack traffic spatial index analysis device is provided, the device comprising:

[0039] A network traffic data set construction unit, used to construct a network traffic data set, wherein the network traffic data set includes network traffic data to be indexed;

[0040] An octree spatial index structure construction unit, configured to construct an octree structure based on the network traffic data to be indexed, and map the network traffic data to be indexed to corresponding nodes of the octree structure to obtain an octree spatial index structure;

[0041] The traffic analysis unit is used to perform traffic distribution statistics and path tracking based on the octree spatial index structure, obtain the traffic category and path table corresponding to each node in the octree spatial index structure, and perform traffic analysis based on the traffic category and path table.

[0042] In a third aspect, a computer device is provided, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned octree-based network attack traffic spatial index analysis method when executing the computer-readable instructions.

[0043] In a fourth aspect, a readable storage medium storing computer-readable instructions is provided. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the above-mentioned octree-based network attack traffic spatial index analysis method.

[0044] The above-mentioned network attack traffic spatial index analysis method, device, computer equipment and storage medium based on octree, and its method implementation include: constructing a network traffic data set, the network traffic data set contains the network traffic data to be indexed; based on the network traffic data to be indexed, constructing an octree structure, mapping the network traffic data to be indexed to the corresponding nodes of the octree structure, and obtaining the octree spatial index structure; based on the octree spatial index structure, performing traffic distribution statistics and path tracking, and obtaining the traffic category and path table corresponding to each node in the octree spatial index structure, so as to perform traffic analysis based on the traffic category and path table. In the embodiment of the present application, the network traffic data is efficiently spatially divided and indexed by constructing an octree structure, and by constructing an efficient spatial index structure, rapid retrieval, analysis and processing of large-scale network traffic data are achieved, thereby significantly improving the efficiency of network monitoring and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] Figure 1 This is a flow chart of a network attack traffic spatial index analysis method based on an octree in one embodiment of the present application;

[0047] Figure 2 It is a structural schematic diagram of a network attack traffic spatial index analysis device based on an octree in one embodiment of the present application;

[0048] Figure 3 It is a schematic diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] In one embodiment, if Figure 1 As shown, a network attack traffic spatial index analysis method based on an octree is provided, comprising the following steps:

[0051] In step S110, a network traffic data set is constructed, wherein the network traffic data set includes network traffic data to be indexed;

[0052] It should be noted that as long as a network attack occurs, traffic will be generated. Therefore, different application layers can be analyzed based on traffic. For example, operators need to analyze user network traffic to calculate network consumption, understand user access to other operators, and provide decision-making data support for the setting of network export interconnection links. Business application layers such as website providers use traffic analysis to understand website visitor data, such as IP addresses, browser information, etc. The security monitoring field uses traffic analysis to monitor abnormal network communications. Therefore, corresponding network traffic data can be collected according to different needs. For example, software packet capture tools such as wireshark and tcpdump can be used to capture and analyze packets in real time to collect network traffic data; or SNMP (Simple Network Management Protocol) is usually used to collect basic traffic details, such as bytes / packets, etc., to collect all network traffic data that needs to be indexed.

[0053] It should be noted that network traffic data can come from network monitoring equipment, intrusion detection systems (IDS), firewalls, anti-DDoS and other security products.

[0054] It should be noted that the collected network traffic data may be pre-processed, such as data cleaning, unifying data formats, filtering duplicate data or missing data, etc.

[0055] In step S120, an octree structure is constructed based on the network traffic data to be indexed, and the network traffic data to be indexed is mapped to corresponding nodes of the octree structure to obtain an octree spatial index structure;

[0056] In an embodiment of the present application, the network traffic data to be indexed can be spatially divided by an octree spatial indexing algorithm, and similar traffic data can be clustered together to form a spatial index structure with a hierarchical structure.

[0057] Specifically, the collected network traffic data can usually be represented in the form of three-dimensional coordinates, that is, each data has a corresponding coordinate (x, y, z). Based on the coordinates corresponding to each data, the boundary of each data in the three-dimensional space is calculated, and the maximum value coordinate and the minimum value coordinate in each dimension are found. Then, based on all the minimum value coordinates and the maximum value coordinates, an initial cube is constructed, and the initial cube is used as the root node of the octree structure for initialization, and the space represented by the root node is recursively divided. In each recursive division process, the space represented by the current node can be divided into eight sub-cubes, each sub-cube can correspond to a sub-node. If the sub-cube contains data that needs to be further refined, the sub-cube is recursively divided until the maximum recursive depth is reached or the amount of data inside the sub-cube is lower than the preset threshold, and all the spatial structures of the octree are obtained, that is, the octree structure is generated.

[0058] Then, a mapping function can be constructed for each network traffic data to be indexed, and based on the mapping function, the network traffic data to be indexed can be converted into coordinate values, and based on the coordinate values, the network traffic data to be indexed can be mapped to the corresponding nodes of the multidimensional space of the octree structure to obtain the octree space index structure. By constructing the octree space index structure, storage space can be saved, query efficiency can be improved, and data processing speed can be accelerated.

[0059] It should be noted that spatial compression technology can be introduced to process data. For example, if all network traffic in a sub-cube exhibits similar behavior patterns, these traffic data can be combined and stored to reduce the storage of duplicate data. By introducing spatial compression technology, reducing data redundancy, optimizing data storage structure, and utilizing spatial correlation, storage overhead can be reduced and index performance can be improved.

[0060] Optionally, the main task of the index construction and maintenance function is to dynamically update the octree index when the network traffic data changes to ensure the accuracy and timeliness of the index. After building the octree spatial index structure, if there is new network traffic data, a new spatial index can be added; when the network traffic data changes, such as traffic increase, decrease, or attribute change, the existing spatial index can be updated; when the network traffic data expires or is considered invalid, the corresponding invalid nodes can be deleted or the invalid or expired network traffic data can be deleted. Through an efficient index maintenance mechanism, the consistency of the octree index and the network traffic data can be maintained, providing a reliable basis for subsequent retrieval and analysis.

[0061] Optionally, during the index building and maintenance process, the balance of the octree can be checked regularly. For unbalanced index structures (such as some nodes are too deep or too shallow), balance optimization is performed through node splitting, merging, or redistribution of data. According to the distribution characteristics of network traffic data and query requirements, the recursive depth and node partitioning strategy of the octree are dynamically adjusted to optimize the performance and efficiency of the index.

[0062] Optionally, ensure the concurrency safety of index building and maintenance operations in a multi-user or multi-threaded environment. Prevent data contention and index inconsistency through locking, transaction management and other mechanisms.

[0063] In step S130, traffic distribution statistics and path tracking are performed based on the octree spatial index structure to obtain the traffic category and path table corresponding to each node in the octree spatial index structure, so as to perform traffic analysis based on the traffic category and path table.

[0064] The traffic category may specifically include the probability of a normal traffic category and the probability of an abnormal traffic category. Optionally, based on the octree spatial index structure, each node of the octree may be traversed. During the traversal process, for each accessed node, it may be detected whether the node is a leaf node. If it is a leaf node, the network traffic data volume in the node is counted, such as the number of data packets, the number of bytes, etc. If it is not a leaf node, other child nodes are continued to be accessed until the network traffic data volume in all leaf nodes is counted. Based on the statistical network traffic data, the spatial distribution characteristics of the obtained network traffic data may be analyzed to help identify traffic characteristic areas and potential security threats.

[0065] Among them, the path table records the transmission path of the network traffic in detail, including the nodes and areas passed through. When a network failure occurs, the specific location and possible causes of the failure can be quickly located by analyzing the path table. Optionally, a traffic path can be given, which may include a source IP and a destination IP, and an empty path table is initialized. Then, the node corresponding to the source IP is used as the starting node, and the path is tracked in the octree structure to determine the next hop node. If the next hop node can be found, the node is added to the empty path table, and the next hop node is continued to be determined. The above steps are repeated until the node corresponding to the destination IP is found. At this time, the path table includes all nodes on the traffic path between the source IP and the destination IP. Based on the path table, the key nodes and areas on the traffic path can be quickly located to help network troubleshooting and performance optimization work to be carried out efficiently.

[0066] The above-mentioned network attack traffic spatial index analysis method based on octree, its method implementation includes: constructing a network traffic data set, the network traffic data set contains the network traffic data to be indexed; based on the network traffic data to be indexed, constructing an octree structure, mapping the network traffic data to be indexed to the corresponding nodes of the octree structure, and obtaining the octree spatial index structure; based on the octree spatial index structure, performing traffic distribution statistics and path tracking, obtaining the traffic category and path table corresponding to each node in the octree spatial index structure, so as to perform traffic analysis based on the traffic category and path table. In the embodiment of the present application, the network traffic data is efficiently spatially divided and indexed by constructing an octree structure, and by constructing an efficient spatial index structure, rapid retrieval, analysis and processing of large-scale network traffic data are achieved, thereby significantly improving the efficiency of network monitoring and management.

[0067] In an embodiment of the present application, the network traffic data to be indexed includes spatial location information of each network traffic data point to be indexed, and constructing an octree structure based on the network traffic data to be indexed includes:

[0068] Based on the spatial location information of each network traffic data point to be indexed, construct an initial cube, wherein the initial cube includes all network traffic data points to be indexed;

[0069] Using the initial cube as the root node of the octree structure;

[0070] The space represented by the root node is recursively partitioned to generate an octree structure.

[0071] Optionally, for the network traffic data to be indexed, a multi-dimensional coordinate system can be established to determine the meaning of each dimension. For example, taking three-dimensional space as an example, the x-axis can represent the date, the y-axis can represent the traffic, and the z-axis represents the location where the traffic is generated, etc. Then, the network traffic data to be indexed is generated based on the coordinate system to generate coordinate points, and multiple network traffic data points can be obtained, and each network traffic data point has a corresponding coordinate (x, y, z). Based on the coordinates corresponding to each data, the boundary of each data in the three-dimensional space is calculated, and the maximum value coordinate and the minimum value coordinate in each dimension are found. Then, based on all the minimum value coordinates and the maximum value coordinates, an initial cube is constructed, and the initial cube is used as the root node of the octree structure for initialization, and the space represented by the root node is recursively divided. In each recursive division process, the space represented by the current node can be divided into eight sub-cubes, each of which can correspond to a sub-node. If the sub-cube contains data that needs to be further refined, the sub-cube is recursively divided until the maximum recursive depth is reached or the amount of data inside the sub-cube is lower than the preset threshold, and all the spatial structures of the octree are obtained, that is, the octree structure is generated.

[0072] In an embodiment of the present application, constructing an initial cube based on the spatial location information of each network traffic data point to be indexed includes:

[0073] In each dimension of the multidimensional space, determine the maximum value coordinate point and the minimum value coordinate point among all the network traffic data points to be indexed;

[0074] Based on the maximum value coordinate point and the minimum value coordinate point, the coordinates of each vertex of the initial cube are constructed; based on the coordinates of each vertex, the initial cube is constructed.

[0075] Optionally, in each dimension of the multidimensional space, determine the maximum value coordinate point and the minimum value coordinate point of all the network traffic data points to be indexed. For example, taking the three-dimensional space as an example, they can be recorded as , , , Then, the eight vertices of the initial cube are defined by the maximum coordinate point and the minimum coordinate point. For example, the eight vertices may be ( , , )、( , , )、( , , )、( , , )、( , , )、( , , )、( , , )、( , , ). Based on the above eight vertices, the initial cube can be obtained and the minimum coordinate position point can be stored =( , , ) and the maximum coordinate position point =( , , ), the maximum coordinate position point and the minimum coordinate position point define an axis-aligned bounding box that encloses all network traffic data.

[0076] In an embodiment of the present application, recursively partitioning the space corresponding to the root node to generate the octree structure includes:

[0077] In each recursive partition, the space represented by the current node is divided into eight sub-cubes, each of which corresponds to a sub-node;

[0078] Determining whether each of the sub-cubes satisfies a termination condition;

[0079] If so, the child node corresponding to the sub-cube that meets the termination condition is used as a leaf node, and the corresponding network traffic data is stored;

[0080] If not, the sub-cubes that do not meet the termination condition are recursively split again until the maximum recursion depth is reached or the amount of data inside the sub-cube is lower than a preset threshold, thereby obtaining the octree structure.

[0081] Optionally, the initial cube can be used as the root node of the octree structure, and the root node can be initialized. The root node may include a pointer to its child nodes (which is empty at the beginning), as well as the minimum and maximum points of the bounding box of the node. Assume that the root node is , and its corresponding bounding box in three-dimensional space is The bounding box is usually composed of a minimum point =( , , ) and a maximum point =( , , )definition.

[0082] Then, the space represented by the root node can be segmented by a recursive segmentation method. For example, in each recursive segmentation, the space represented by the current node is divided into eight sub-cubes, each sub-cube corresponds to a sub-node. Taking node N as an example, when recursively segmenting N nodes, determine whether node N meets the termination condition. If the termination condition is met, node N is marked as a leaf node and the corresponding network traffic data exists. If the termination condition is not met, the midpoint of the bounding box B of node N can be calculated, which can be calculated by the following formula:

[0083] ;

[0084] Then, the bounding box B is divided into eight sub-bounding boxes, each of which consists of the midpoint m and , For example, the first child bounding box is defined by ( , , ) to m, the second sub-bounding box is defined by ( , , )arrive( , , ) definition, and so on. Create a new child node for each child bounding box and add these child nodes to the current node N. Traverse the network flow data points in the current node N and assign each network flow data point to the corresponding child node according to the child bounding box where it is located. Repeat the above steps for each child node until all nodes are processed.

[0085] in, Indicates the minimum size threshold of the bounding box B, Indicates the maximum size threshold of the bounding box B.

[0086] It should be noted that the termination condition may specifically include: determining that the number of points contained in the node is less than a predetermined threshold τ, or the size of the bounding box is less than a predetermined threshold ε. For example, the point count threshold: |N|<τ, where |N| represents the number of points contained in the node N, and τ is the predetermined point count threshold. Bounding box size threshold: ,in is a predetermined size threshold.

[0087] In an embodiment of the present application, mapping the network traffic data to be indexed to a corresponding node of the octree structure includes:

[0088] Extracting the traffic characteristics corresponding to each of the network traffic data to be indexed;

[0089] Construct a mapping function for each traffic feature;

[0090] Converting the traffic characteristics into coordinate points in a multidimensional space based on the mapping function;

[0091] Based on the coordinate points, the network traffic data to be indexed is mapped to corresponding nodes of the octree structure.

[0092] Optionally, for each network traffic data to be indexed, traffic features can be extracted through feature extraction networks or feature engineering. It should be noted that the extracted traffic features must be able to represent the key features of network attack traffic characteristics. Specifically, they may include: traffic size ( ): The amount of data transmitted per unit time (such as the number of bytes); the number of data packets ( ): The total number of packets transmitted per unit time; the average packet size ( ): The average size of all packets in the traffic; the hash value of the source IP address ( ): The value obtained by hashing the source IP address; the hash value of the destination IP address ( ): The value obtained by hashing the destination IP address.

[0093] For each traffic feature, a mapping function can be constructed, which can be used to convert the feature value into a coordinate value in a multidimensional space. It can be understood that the mapping function needs to consider factors such as the distribution characteristics of the feature, the sparsity of the data, and the uniformity of the mapping.

[0094] In one embodiment of the present application, the traffic characteristics include: traffic size, number of data packets, average data packet size, source IP address and destination IP address, and the conversion of the traffic characteristics into coordinate points in a multidimensional space based on the mapping function includes:

[0095] Calculate hash values ​​for the source IP address and the destination IP address respectively to obtain corresponding hash values, assign corresponding weight coefficients to the hash values, and perform modulus operations to obtain corresponding coordinate points;

[0096] Corresponding weight coefficients are assigned to the flow size, the number of data packets, and the average data packet size, and a modular operation is performed to obtain corresponding coordinate points.

[0097] Optionally, characteristic values ​​such as traffic size, number of packets, average packet size, source IP address, and destination IP address can be extracted from the network traffic data. For the source IP address and the destination IP address, their hash values ​​can be calculated by a hash function, and then the corresponding weight coefficients can be assigned to them, and the coordinate points of the latter two dimensions can be obtained by modular operations. The corresponding weight coefficients can be assigned to the traffic size, number of packets, and average packet size, and modular operations can be performed to obtain the coordinate values ​​of the first three dimensions. The five calculated coordinate values ​​are then combined into a coordinate point in a five-dimensional space, that is, ( , , ). Each coordinate point represents a specific instance of network attack traffic and has a unique position in the multidimensional space. In this way, network traffic data can be mapped into the multidimensional space for further analysis and visualization.

[0098] Exemplarily, a mapping function may be constructed, where each dimension may represent a feature. The mapping function may be as follows:

[0099] =( , , );

[0100] The coordinate points are calculated as follows:

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] in, , , : Weight coefficients, used to adjust the relative importance of traffic size, number of packets, and average packet size in coordinate calculation. These coefficients can be adjusted and optimized according to actual needs. , , , , : Modulus, used to constrain the range of coordinate values, ensuring that they are in a reasonable region of the multidimensional space. The choice of modulus should be based on the range and distribution of the eigenvalues. , : The hash value of the source IP address and the destination IP address is calculated by a hash function. The choice of the hash function should ensure the uniqueness and uniform distribution of the hash value.

[0107] In an embodiment of the present application, after obtaining the octree spatial index structure, the following steps are included:

[0108] When there is new network flow data, determining the spatial region to which the new network flow data belongs based on the attributes of the new network flow data;

[0109] Traversing the octree structure to determine whether there is a target node corresponding to the spatial area to which the new network traffic data belongs;

[0110] If yes, add the index information of the new network traffic data to the target node, and update the target node;

[0111] If not, a new node is created, and the index information of the new network traffic data is added to the new node.

[0112] Optionally, when new network traffic data arrives, first determine the spatial region to which it belongs based on the attributes of the data. Traverse each node in the octree, find the node corresponding to the region, add the index information of the new network traffic data to the node, and update the node. If the node does not exist (i.e., the maximum recursive depth has not been reached and the sub-region is not empty), create a new node as needed. Add the index or summary information of the new data to the corresponding node, and update the relevant statistical information of the node (such as data volume, boundary range, etc.).

[0113] In an embodiment of the present application, after obtaining the octree spatial index structure, the following steps are included:

[0114] When the network traffic data changes, the index information of the node to which the network traffic data belongs is updated, or the node to which the network traffic data belongs is split or merged to update the index information;

[0115] When the network traffic data is expired or invalid, the invalid nodes in the octree structure that contain the expired or invalid network traffic data are determined; if all network traffic data contained in the invalid nodes are invalid, the invalid nodes and their corresponding child nodes are deleted; otherwise, the index information of the expired or invalid data is deleted.

[0116] Optionally, when the network traffic data changes, such as traffic increases, decreases, or attribute changes, the index information of the corresponding node needs to be updated. According to the specific circumstances of the data change, adjust the node's boundary range, data volume and other statistical information. If the change causes the node to need to be split or merged, perform the corresponding operation and update the index structure. When the network traffic data expires or is considered invalid, for example, the old data is cleaned up, the abnormal data is removed, etc., the corresponding index information needs to be deleted from the octree index. Traverse the octree and find the node containing invalid data. If all data in the node is invalid data, delete the node and all its child nodes; otherwise, only delete the index information of the invalid data and update the node's statistical information.

[0117] In an embodiment of the present application, the traffic distribution statistics and path tracking based on the octree spatial index structure include:

[0118] Traversing each node of the octree structure, when the accessed node is a leaf node, counting the network traffic data in the leaf node, classifying the counted network traffic data, and determining the traffic category, wherein the traffic category includes a normal traffic probability and an abnormal traffic probability;

[0119] Determine a traffic path, the traffic path including a source IP and a destination IP, take a node corresponding to the source IP as a starting node, search for a next hop node in the octree structure based on the traffic path, and search for the next hop node again based on the found next hop node until a node corresponding to the destination IP is found, thereby generating the path table.

[0120] Optionally, based on the octree spatial index structure, each node of the octree can be traversed. During the traversal process, for each visited node, it can be detected whether the node is a leaf node. If it is a leaf node, the network traffic data volume in the node is counted, such as the number of data packets, the number of bytes, etc. If it is not a leaf node, other child nodes are continuously accessed until the network traffic data volume in all leaf nodes is counted. Based on the statistical network traffic data, the spatial distribution characteristics of the network traffic data can be analyzed to help identify traffic characteristic areas and potential security threats.

[0121] Among them, a logistic regression algorithm can be used to classify the statistical network traffic data volume to determine the probability that the network traffic data is normal traffic and the probability of abnormal traffic, as shown below:

[0122] ;

[0123] in:

[0124] : The category label of the node (such as 0 for normal traffic and 1 for attack traffic).

[0125] : The feature vector of the node, which can include traffic data volume, traffic density, source / destination address distribution, etc.

[0126] , ,…, are the parameters of the model.

[0127] The goal of this formula is to calculate the feature vector of a given node The probability that the node belongs to the attack traffic category is .

[0128] Optionally, the path table records the transmission path of network traffic in detail, including the nodes and areas passed through. When a network failure occurs, the specific location and possible causes of the failure can be quickly located by analyzing the path list. The path tracking can be implemented in the following ways:

[0129] A traffic path can be given, which may include a source IP and a destination IP, and an empty path table is initialized. Then, the node corresponding to the source IP is used as the starting node, and the starting node is added to the empty path table. Then, the next hop node is searched in the octree structure according to the next hop information of the traffic path, wherein the next hop information may include the IP address or port number of the next hop. If the next hop node can be found, the found next hop node is added to the empty path table, and the next hop node is used as the starting point to continue to determine the next hop node. Otherwise, the tracking is terminated and the current path list is output. Repeat the above steps until the node corresponding to the destination IP is found. At this time, the path table includes all nodes on the traffic path between the source IP and the destination IP. Based on the path table, the key nodes and areas on the traffic path can be quickly located to help network troubleshooting and performance optimization work to be carried out efficiently.

[0130] It should be noted that after performing path tracking, the tracking effect can be optimized by combining a deep learning algorithm. Optionally, the path information obtained after path tracking is input into a pre-trained neural network model, and the next hop node of the traffic path is predicted by the neural network model to obtain a more accurate and efficient traffic path.

[0131] For example, let X be the vector representation of traffic features (such as source IP, destination IP, port number, protocol type, historical path information, etc.), and Y be the label of the next hop node. Use a neural network model to learn the mapping relationship from X to Y. The specific algorithm is as follows:

[0132] ;

[0133] in:

[0134] : Input feature vector, which contains various characteristic information of traffic.

[0135] : The weight matrix of the first layer of the neural network, used to map input features to the hidden layer.

[0136] : The bias vector of the first layer of the neural network, used to adjust the output of the hidden layer.

[0137] : Activation function, used to increase the nonlinear representation ability of the model.

[0138] : The output prediction label, indicating the predicted next hop node.

[0139] In the embodiment of the present application, the network traffic data is efficiently spatially divided and indexed by constructing an octree structure, and by constructing an efficient spatial index structure, rapid retrieval, analysis and processing of large-scale network traffic data are achieved, thereby significantly improving network monitoring and management efficiency. Through dynamic updates and concurrency control, the real-time and accuracy of the index are ensured, and the processing efficiency is improved. The innovative spatial mapping strategy is combined with spatial compression technology to effectively reduce storage overhead, reduce system resource consumption, and improve indexing performance. It is also easy to expand to larger-scale network traffic data processing scenarios, with good adaptability and flexibility.

[0140] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0141] In one embodiment, an octree-based network attack traffic spatial index analysis device is provided, and the octree-based network attack traffic spatial index analysis device corresponds to the octree-based network attack traffic spatial index analysis method in the above embodiment. Figure 2 As shown, the network attack traffic spatial index analysis device based on octree includes a network traffic data set construction unit 10, an octree spatial index structure construction unit 20 and a traffic analysis unit 30. Each functional module is described in detail as follows:

[0142] A network traffic data set construction unit 10 is used to construct a network traffic data set, wherein the network traffic data set includes network traffic data to be indexed;

[0143] An octree spatial index structure construction unit 20 is used to construct an octree structure based on the network traffic data to be indexed, and map the network traffic data to be indexed to corresponding nodes of the octree structure to obtain an octree spatial index structure;

[0144] The traffic analysis unit 30 is used to perform traffic distribution statistics and path tracking based on the octree spatial index structure, obtain the traffic category and path table corresponding to each node in the octree spatial index structure, and perform traffic analysis based on the traffic category and path table.

[0145] In one embodiment of the present application, the traffic analysis unit 30 is further configured to:

[0146] Traversing each node of the octree structure, when the accessed node is a leaf node, counting the network traffic data in the leaf node, classifying the counted network traffic data, and determining the traffic category, wherein the traffic category includes a normal traffic probability and an abnormal traffic probability;

[0147] Determine a traffic path, the traffic path including a source IP and a destination IP, take a node corresponding to the source IP as a starting node, search for a next hop node in the octree structure based on the traffic path, and search for the next hop node again based on the found next hop node until a node corresponding to the destination IP is found, thereby generating the path table.

[0148] In one embodiment of the present application, the octree spatial index structure construction unit 20 is further used for:

[0149] Extracting the traffic characteristics corresponding to each of the network traffic data to be indexed;

[0150] Construct a mapping function for each traffic feature;

[0151] Converting the traffic characteristics into coordinate points in a multidimensional space based on the mapping function;

[0152] Based on the coordinate points, the network traffic data to be indexed is mapped to corresponding nodes of the octree structure.

[0153] In an embodiment of the present application, the traffic characteristics include: traffic size, number of data packets, average data packet size, source IP address and destination IP address, and the octree space index structure construction unit 20 is further used to:

[0154] Hash values ​​are calculated for the source IP address and the destination IP address respectively to obtain corresponding hash values, corresponding weight coefficients are assigned to the hash values, and modular operations are performed to obtain corresponding coordinate points; corresponding weight coefficients are assigned to the traffic size, number of data packets, and average data packet size respectively, and modular operations are performed to obtain corresponding coordinate points.

[0155] In one embodiment of the present application, the device further includes a spatial index adding unit, which is used to:

[0156] When there is new network flow data, determining the spatial region to which the new network flow data belongs based on the attributes of the new network flow data;

[0157] Traversing the octree structure to determine whether there is a target node corresponding to the spatial area to which the new network traffic data belongs;

[0158] If yes, add the index information of the new network traffic data to the target node, and update the target node;

[0159] If not, a new node is created, and the index information of the new network traffic data is added to the new node.

[0160] In one embodiment of the present application, the device further includes a spatial index updating and deleting unit, which is used to:

[0161] When the network traffic data changes, the index information of the node to which the network traffic data belongs is updated, or the node to which the network traffic data belongs is split or merged to update the index information;

[0162] When the network traffic data is expired or invalid, the invalid nodes in the octree structure that contain the expired or invalid network traffic data are determined; if all network traffic data contained in the invalid nodes are invalid, the invalid nodes and their corresponding child nodes are deleted; otherwise, the index information of the expired or invalid data is deleted.

[0163] In one embodiment of the present application, the octree spatial index structure construction unit 20 is further used for:

[0164] Based on the spatial location information of each network traffic data point to be indexed, construct an initial cube, wherein the initial cube includes all network traffic data points to be indexed;

[0165] Using the initial cube as the root node of the octree structure;

[0166] The space represented by the root node is recursively partitioned to generate an octree structure.

[0167] In one embodiment of the present application, the octree spatial index structure construction unit 20 is further used for:

[0168] In each dimension of the multidimensional space, determine the maximum value coordinate point and the minimum value coordinate point among all the network traffic data points to be indexed;

[0169] Based on the maximum value coordinate point and the minimum value coordinate point, the coordinates of each vertex of the initial cube are constructed; based on the coordinates of each vertex, the initial cube is constructed.

[0170] In one embodiment of the present application, the octree spatial index structure construction unit 20 is further used for:

[0171] In each recursive partition, the space represented by the current node is divided into eight sub-cubes, each of which corresponds to a sub-node;

[0172] Determining whether each of the sub-cubes satisfies a termination condition;

[0173] If so, the child node corresponding to the sub-cube that meets the termination condition is used as a leaf node, and the corresponding traffic data is stored;

[0174] If not, the sub-cubes that do not meet the termination condition are recursively split again until the maximum recursion depth is reached or the amount of data inside the sub-cube is lower than a preset threshold, thereby obtaining the octree structure.

[0175] In the embodiment of the present application, the network traffic data is efficiently spatially divided and indexed by constructing an octree structure, and by constructing an efficient spatial index structure, rapid retrieval, analysis and processing of large-scale network traffic data are achieved, thereby significantly improving network monitoring and management efficiency. Through dynamic updates and concurrency control, the real-time and accuracy of the index are ensured, and the processing efficiency is improved. The innovative spatial mapping strategy is combined with spatial compression technology to effectively reduce storage overhead, reduce system resource consumption, and improve indexing performance. It is also easy to expand to larger-scale network traffic data processing scenarios, with good adaptability and flexibility.

[0176] For the specific definition of the octree-based network attack traffic spatial index analysis device, please refer to the definition of the octree-based network attack traffic spatial index analysis method above, which will not be repeated here. Each module in the above-mentioned octree-based network attack traffic spatial index analysis device can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0177] In one embodiment, a computer device is provided. The computer device may be a terminal device, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a network attack traffic space index analysis method based on an octree is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0178] In an embodiment of the present application, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the octree-based network attack traffic spatial index analysis method as described above are implemented.

[0179] In an embodiment of the application, a readable storage medium is provided, wherein the readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the octree-based network attack traffic spatial index analysis method as described above are implemented.

[0180] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0181] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0182] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A network attack traffic spatial index analysis method based on octree, characterized in that: The method comprises: Constructing a network traffic data set, wherein the network traffic data set includes network traffic data to be indexed; Based on the network traffic data to be indexed, an octree structure is constructed, and the network traffic data to be indexed is mapped to corresponding nodes of the octree structure to obtain an octree spatial index structure; Based on the octree spatial index structure, traffic distribution statistics and path tracking are performed to obtain a traffic category and a path table corresponding to each node in the octree spatial index structure, so as to perform traffic analysis based on the traffic category and the path table; The flow distribution statistics and path tracking based on the octree spatial index structure include: Traversing each node of the octree structure, when the accessed node is a leaf node, counting the network traffic data in the leaf node, classifying the counted network traffic data, and determining the traffic category, wherein the traffic category includes a normal traffic probability and an abnormal traffic probability; Determine a traffic path, the traffic path including a source IP and a destination IP, take a node corresponding to the source IP as a starting node, search for a next hop node in the octree structure based on the traffic path, and search for the next hop node again based on the found next hop node until a node corresponding to the destination IP is found, thereby generating the path table; After obtaining the octree spatial index structure, the following steps are included: When there is new network flow data, determining the spatial region to which the new network flow data belongs based on the attributes of the new network flow data; Traversing the octree structure to determine whether there is a target node corresponding to the spatial area to which the new network traffic data belongs; If yes, add the index information of the new network traffic data to the target node, and update the target node; If not, create a new node and add the index information of the new network traffic data to the new node; After obtaining the octree spatial index structure, the following steps are included: When the network traffic data to be indexed changes, the index information of the node to which the network traffic data to be indexed belongs is updated, or the node to which the network traffic data to be indexed belongs is split or merged to update the index information; When the network traffic data to be indexed is expired or invalid, determine the invalid nodes in the octree structure that contain the expired or invalid network traffic data to be indexed; if all the network traffic data to be indexed contained in the invalid nodes are invalid, delete the invalid nodes and their corresponding child nodes; otherwise, delete the index information of the expired data or invalid data.

2. The network attack traffic spatial index analysis method based on octree according to claim 1 is characterized in that: Mapping the to-be-indexed network traffic data to corresponding nodes of the octree structure includes: Extracting the traffic characteristics corresponding to each of the network traffic data to be indexed; Construct a mapping function for each traffic feature; Converting the traffic characteristics into coordinate points in a multidimensional space based on the mapping function; Based on the coordinate points, the network traffic data to be indexed is mapped to corresponding nodes of the octree structure.

3. The network attack traffic spatial index analysis method based on octree according to claim 2 is characterized in that: The traffic characteristics include: traffic size, number of data packets, average data packet size, source IP address and destination IP address, and the conversion of the traffic characteristics into coordinate points in a multidimensional space based on the mapping function includes: Calculate hash values ​​for the source IP address and the destination IP address respectively to obtain corresponding hash values, assign corresponding weight coefficients to the hash values, and perform modulus operations to obtain corresponding coordinate points; Corresponding weight coefficients are assigned to the flow size, the number of data packets, and the average data packet size, and a modular operation is performed to obtain corresponding coordinate points.

4. The network attack traffic spatial index analysis method based on octree according to claim 1 is characterized in that: The network traffic data to be indexed includes spatial location information of each network traffic data point to be indexed, and constructing an octree structure based on the network traffic data to be indexed includes: Based on the spatial location information of each network traffic data point to be indexed, construct an initial cube, wherein the initial cube includes all network traffic data points to be indexed; Using the initial cube as the root node of the octree structure; The space represented by the root node is recursively partitioned to generate an octree structure.

5. The network attack traffic spatial index analysis method based on octree according to claim 4 is characterized in that: The step of constructing an initial cube based on the spatial location information of each network traffic data point to be indexed includes: In each dimension of the multidimensional space, determine the maximum value coordinate point and the minimum value coordinate point among all the network traffic data points to be indexed; Based on the maximum value coordinate point and the minimum value coordinate point, construct the coordinates of each vertex of the initial cube; The initial cube is constructed based on the coordinates of each vertex.

6. The network attack traffic spatial index analysis method based on octree according to claim 5 is characterized in that: The recursive segmentation of the space represented by the root node to generate an octree structure includes: In each recursive partition, the space represented by the current node is divided into eight sub-cubes, each of which corresponds to a sub-node; Determining whether each of the sub-cubes satisfies a termination condition; If so, the child node corresponding to the sub-cube that meets the termination condition is used as a leaf node, and the corresponding traffic data is stored; If not, the sub-cubes that do not meet the termination condition are recursively split again until the maximum recursion depth is reached or the amount of data inside the sub-cube is lower than a preset threshold, thereby obtaining the octree structure.

7. A network attack traffic spatial index analysis device based on octree, characterized in that: The device comprises: A network traffic data set construction unit, used to construct a network traffic data set, wherein the network traffic data set includes network traffic data to be indexed; An octree spatial index structure construction unit, configured to construct an octree structure based on the network traffic data to be indexed, and map the network traffic data to be indexed to corresponding nodes of the octree structure to obtain an octree spatial index structure; A traffic analysis unit, configured to perform traffic distribution statistics and path tracking based on the octree spatial index structure, obtain a traffic category and a path table corresponding to each node in the octree spatial index structure, and perform traffic analysis based on the traffic category and the path table; The traffic analysis unit is also used for: Traversing each node of the octree structure, when the accessed node is a leaf node, counting the network traffic data in the leaf node, classifying the counted network traffic data, and determining the traffic category, wherein the traffic category includes a normal traffic probability and an abnormal traffic probability; Determine a traffic path, the traffic path including a source IP and a destination IP, take a node corresponding to the source IP as a starting node, search for a next hop node in the octree structure based on the traffic path, and search for the next hop node again based on the found next hop node until a node corresponding to the destination IP is found, thereby generating the path table; The device further includes: a spatial index adding unit, configured to: When there is new network flow data, determining the spatial region to which the new network flow data belongs based on the attributes of the new network flow data; Traversing the octree structure to determine whether there is a target node corresponding to the spatial area to which the new network traffic data belongs; If yes, add the index information of the new network traffic data to the target node, and update the target node; If not, create a new node and add the index information of the new network traffic data to the new node; The device further includes: a spatial index updating and deleting unit, configured to: When the network traffic data to be indexed changes, the index information of the node to which the network traffic data to be indexed belongs is updated, or the node to which the network traffic data to be indexed belongs is split or merged to update the index information; When the network traffic data to be indexed is expired or invalid, determine the invalid nodes in the octree structure that contain the expired or invalid network traffic data to be indexed; if all the network traffic data to be indexed contained in the invalid nodes are invalid, delete the invalid nodes and their corresponding child nodes; otherwise, delete the index information of the expired data or invalid data.

Images