A method for big data information security verification and anomaly tracing

Abstract

The present invention relates to the field of big data security technology, and in particular to a big data information security verification and exception tracing method, which obtains a data packet hash value by acquiring an original data packet and performing multiple hash processing, performs security verification according to the data packet hash value through a preset security verification rule to obtain data packet target data, stores the data packet target data according to an adaptive data storage model, extracts data packet storage data feature information and loads it into a behavior detection model to obtain data packet behavior detection data, generates a tracing request according to the data packet behavior detection data and sends it to an upstream target node, the upstream target node generates a tracing data packet according to the tracing request, transmits the tracing data packet to a source node through a dynamic network adaptation model, and the source node generates and returns the tracing data to the node issuing the tracing request, thereby realizing efficient management of big data information security verification and exception tracing in a dynamically changing network environment.

Description

Technical Field

[0001] The present invention relates to the field of big data security technology, and in particular to a big data information security verification and exception tracing method. Background Art

[0002] With the rapid development of big data, especially in the context of cloud computing, the Internet of Things and social media, the amount of data is growing exponentially, which puts higher requirements on data security management. Traditional data security management relies on static management mechanisms, usually taking post-event remediation and passive defense, resulting in insufficient response to security threats. At present, some R&D teams have focused on studying active management mechanisms for data security management, but they all focus on security verification and anomaly tracing in a fixed network architecture, which lacks effectiveness and accuracy in the face of dynamically changing network environments. In view of this, a new method is needed to achieve efficient management of big data information security verification and anomaly tracing in a dynamically changing network environment. Summary of the invention

[0003] The purpose of the present invention is to provide a big data information security verification and exception tracing method to achieve efficient management of big data information security and exception tracing in a dynamically changing network environment.

[0004] To achieve the above object, the present invention provides a big data information security verification and exception tracing method, the method comprising:

[0005] The original data packet is obtained and multiple hash processing is performed to convert the original data packet into a data packet hash value. According to the data packet hash value, security verification is performed through preset security verification rules to obtain the data packet target data.

[0006] The data packet target data is stored according to the adaptive data storage model to obtain the data packet storage data, and the data packet storage data feature information is extracted and loaded into the behavior detection model to obtain the data packet behavior detection data; the behavior detection model is a mapping relationship between the data packet preprocessing features and the data packet anomaly categories and the data packet anomaly ratio values ​​established by the multi-layer perceptron algorithm based on the historically collected data packet storage data feature information.

[0007] A tracing request is generated based on the data of packet behavior detection and sent to the upstream target node. The upstream target node generates a tracing data packet based on the tracing request, and transmits the tracing data packet to the source node through the dynamic network adaptation model. The source node generates and returns the tracing data to the node that issued the tracing request, obtains the data information in the tracing data packet, and obtains the abnormal target source.

[0008] Furthermore, the method of obtaining the original data packet and performing multiple hashing to convert the original data packet into a data packet hash value, and performing security verification according to the data packet hash value through preset security verification rules to obtain the data packet target data includes:

[0009] Perform multiple hash operations on each data field of the original data packet to generate a data packet hash value.

[0010] The data packet hash value is matched with the preset hash whitelist and hash blacklist. When the traversal result shows that the data packet hash value is in the hash whitelist, the data packet is preliminarily verified through hash matching. When the traversal result shows the hash blacklist, the data packet is intercepted and a security alarm is triggered.

[0011] The preliminary verification data packet is checked according to the port number rules to see if it meets the expected port of the transmission protocol. When the preliminary verification data packet does not meet the expected port of the transmission protocol, the data packet is marked as abnormal data, the data packet is isolated and re-verification is triggered. When the preliminary verification data packet meets the expected port of the transmission protocol, it is marked as passing the security verification and obtaining the target data of the data packet.

[0012] Furthermore, the method of performing multiple hash operations on each data field of the original data packet to generate a data packet hash value includes:

[0013] Extract the key information of the original data packet, and perform the first hash processing on the key information of the data packet through the SHA-1 algorithm to obtain the first hash data of the data packet.

[0014] The first hash value of the data packet is subjected to a secondary hash process using the SHA-512 algorithm to obtain the second hash data of the data packet, and the first hash data of the data packet and the second hash data of the data packet are combined and finally hashed using the SHA-256 algorithm to obtain the hash value of the data packet.

[0015] Further, the method of storing the data packet target data according to the adaptive data storage model to obtain the data packet storage data includes:

[0016] The storage space is divided into storage layers according to a preset number of layers, and the storage layers are divided into storage blocks according to a preset number of blocks to obtain a storage allocation matrix.

[0017] The target data of the data packet is calculated according to the storage allocation matrix layer index address calculation rule to obtain the storage layer index address, and the target data of the data packet is calculated according to the storage allocation matrix layer internal address calculation rule to obtain the storage layer internal address.

[0018] The data packet target data is stored as data packet storage data according to the storage layer index address and the address within the storage layer and the corresponding position is set to 1.

[0019] Dynamically detect the saturation rate of each layer of storage blocks in the storage allocation matrix. When the storage block storage of the idle layer reaches a preset saturation rate threshold, the idle layer is updated to the saturated layer, and another idle layer storage block is selected to replace the saturated layer storage block. The index address is updated to point the index pointing to the saturated layer storage block to the idle layer storage block.

[0020] The target data retention time of the data packet in the saturated layer storage block is recorded. When the retention time reaches the set preservation period, the storage block is released, the corresponding position is set to 0 and the saturated layer is updated to the free layer.

[0021] Furthermore, the method for storing and allocating matrix layer index address calculation rules includes:

[0022] Perform a bitwise AND operation on the data packet hash value and the 16-bit mask of all 1s shifted left by a preset number of bits, and retain the high-order part of the data packet hash value as the storage allocation matrix layer index address.

[0023] The method for storing and allocating address calculation rules in a matrix layer includes:

[0024] The data packet hash value is bitwise ANDed with a low-order mask of a preset number of bits, and the low-order portion of the data packet hash value is retained to obtain an address within the storage allocation matrix layer.

[0025] Furthermore, the method of extracting data characteristic information stored in a data packet and loading it into a behavior detection model to obtain data packet behavior detection data includes:

[0026] Extract data packet storage data feature information including data packet size, data packet time interval, source IP connection frequency, data packet flow rate, protocol type, and port type; normalize the data packet size, data packet time interval, source IP connection frequency, and data packet flow rate in the data packet storage data feature information; and perform one-hot encoding on the protocol type and port type in the data packet storage data feature information to obtain data packet preprocessing features.

[0027] The data packet preprocessing features are loaded into the behavior detection model to obtain the data packet anomaly category and data packet anomaly ratio value; the anomaly categories include: malware propagation, phishing attacks, SQL injections, cross-site scripting attacks, and denial of service attacks.

[0028] The data packet anomaly ratio value is compared with the corresponding anomaly category tolerance threshold. When the data packet anomaly ratio value is greater than the anomaly category tolerance threshold, it is determined to be an abnormal data packet and the corresponding anomaly category is used as the behavior detection result.

[0029] The behavior detection model training step includes:

[0030] Set up the input layer and input the training set ;in is the data packet preprocessing feature training set, For sample Packet preprocessing features, For sample The data packet anomaly category, N is the number of samples.

[0031] The hidden layer activation function is set to the Softmax function.

[0032] The output layer sets 5 neuron nodes to output 5 types of data packet anomalies respectively.

[0033] The model is compiled by the binary cross entropy loss function. For each type of target data packet information feature data anomaly judgment information predicted by the model, the loss is calculated as follows:

[0034] .

[0035] in, is the number of samples, is the number of packet anomaly categories, is a training set sample In the packet anomaly category The true value of is a training set sample In the packet anomaly category The predicted value of .

[0036] The Adam optimizer is used to adjust the learning rate for parameter fitting.

[0037] Use cross-validation set data to validate the model, adjust the model architecture based on the performance of the validation set data, and iteratively optimize the model parameters.

[0038] Furthermore, the method of generating a tracing request according to the data packet behavior detection data and sending it to the upstream target node, the upstream target node generating a tracing data packet according to the tracing request, transmitting the tracing data packet to the source node through the dynamic network adaptation model, the source node generating and returning the tracing data to the node issuing the tracing request, and obtaining the data information in the tracing data packet to obtain the abnormal target source includes:

[0039] According to the data packet behavior detection data, the request information matching the abnormal category is obtained from the preset abnormal category tracing request information list and sent to the upstream target node.

[0040] The network indicators of the path segments between the upstream nodes are obtained, and the network indicators of the path segments between the upstream nodes are input into the dynamic network adaptation model to obtain the tracing path through the shortest path algorithm. The tracing data packets are sliced ​​according to the tracing path to obtain the tracing data packet slice set. The frequency of tracing data packets being sent is determined according to the number of tracing data packet slices in the tracing data packet slice set. Each upstream node transmits the tracing data packet slice set forward along the tracing path in sequence according to the tracing data packet sending frequency until it reaches the source node.

[0041] The source node generates and returns the traceability data to the node that issues the traceability request, obtains the data information in the traceability data packet, and obtains the abnormal target source.

[0042] Furthermore, the network indicators of the path segment between the upstream node and the node include bandwidth, delay and load; the method of inputting the network indicators of the path segment between the upstream node and the node into the dynamic network adaptation model to obtain the tracing path through the shortest path algorithm, slicing the tracing data packet according to the tracing path to obtain the tracing data packet slice set, and determining the frequency of tracing data packet issuance according to the number of tracing data packet slices in the tracing data packet slice set includes:

[0043] The bandwidth, delay and load of the path segment between nodes are obtained by exchanging information between network nodes, and the maximum and minimum values ​​are normalized to obtain the normalized bandwidth value, delay value and load value. The path segment performance is obtained based on the normalized bandwidth value, delay value and load value. for:

[0044] .

[0045] in Is a node To Node The normalized bandwidth value of the path segment between Is a node To Node The normalized load value of the path segment between Is a node To Node The normalized delay value of the path segment between is the path segment performance.

[0046] Initialize the comprehensive path segment efficiency to zero, start from the node where the tracing request is issued, calculate the path segment efficiency of all directly connected nodes, mark the node where the tracing request is issued as visited, select the node corresponding to the minimum value of the path segment efficiency of the adjacent nodes as the iteration target node and update the comprehensive path segment efficiency according to the path segment efficiency to obtain the comprehensive path segment efficiency iteration value.

[0047] Mark the nodes directly connected to the node issuing the traceability request as visited. Starting from the iteration target node, calculate the path segment efficiencies of all directly connected nodes and repeat the next iteration target node selection and comprehensive path segment efficiency iteration value update until all nodes are traversed and the iteration is stopped. When all nodes are marked as visited, the current comprehensive path segment efficiency iteration value is used as the final comprehensive path segment efficiency. The path segments between the iteration target nodes corresponding to the final comprehensive path segment efficiency are obtained to obtain the traceability path.

[0048] Select the path with the minimum path segment efficiency in the traceability path as the bottleneck path segment, and slice the traceability data packet according to the bandwidth and load of the bottleneck path segment to obtain the number of traceability data packet slices. for:

[0049] .

[0050] in is the bandwidth of the bottleneck path segment, is the load of the bottleneck path segment, It is The size of the node tracing data packet, It is The number of slices of the node traceability data packet, Yes Round up.

[0051] The source data packet is sliced ​​according to the number of source data packet slices to obtain a source data packet slice set, and the slice transmission time is obtained according to the number of source data packet slices in the source data packet slice set, the bandwidth and load of the bottleneck path segment. for:

[0052] .

[0053] in It is The slice transmission time of the traceability data packet of each node.

[0054] The frequency of tracing data packets is obtained based on the slice transmission time and bottleneck path segment delay. for:

[0055] .

[0056] in It is The frequency of sending tracing packets from each node, is the bottleneck path segment delay.

[0057] Compared with the prior art, the present invention has the following beneficial effects: dynamic storage and efficient management of data are realized by storing data packet target data through an adaptive data storage model, abnormal behavior of data packets are recognized by building a behavior detection model through a multi-layer perceptron algorithm, the optimal tracing path is determined through a dynamic network adaptation model, and the tracing data packets are sliced ​​and the sending frequency is dynamically adjusted in combination with dynamic network indicators to realize efficient abnormal tracing. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 This is a flowchart of a big data information security verification and exception tracing method according to Example 1 of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Example 1: Figure 1 As shown, this embodiment provides a big data information security verification and exception tracing method, the method comprising:

[0062] S1. Obtain the original data packet and perform multiple hashing processes to convert the original data packet into a data packet hash value, and perform security verification according to the data packet hash value through preset security verification rules to obtain the data packet target data; the preset security verification rules are hash whitelist, hash blacklist, and port number rules set according to experience and standards.

[0063] S2. Storing the target data of the data packet according to the adaptive data storage model to obtain the data packet storage data, extracting the characteristic information of the data packet storage data and loading it into the behavior detection model to obtain the data packet behavior detection data; the behavior detection model is a mapping relationship between the data packet preprocessing characteristics and the data packet anomaly categories and the data packet anomaly ratio values ​​established by the multi-layer perceptron algorithm based on the characteristic information of the data packet storage data collected historically; by further extracting characteristic information from the data packet storage data and monitoring and analyzing these characteristics through the behavior detection model, further identifying the complex threats and intrusions that are not easy to be directly detected by the security verification rules, thereby achieving a more accurate judgment of abnormal data behavior.

[0064] S3. Generate a tracing request based on the data of packet behavior detection and send it to the upstream target node. The upstream target node generates a tracing data packet based on the tracing request, transmits the tracing data packet to the source node through the dynamic network adaptation model, and the source node generates and returns the tracing data to the node that issued the tracing request, obtains the data information in the tracing data packet, and obtains the abnormal target source.

[0065] It should be noted that the method of obtaining the original data packet and performing multiple hashing to convert the original data packet into a data packet hash value, and performing security verification according to the data packet hash value through a preset security verification rule to obtain the data packet target data includes:

[0066] S11. Perform multiple hash operations on each data field of the original data packet to generate a data packet hash value.

[0067] S12. Match the data packet hash value with the preset hash whitelist and hash blacklist. When the traversal result shows that the data packet hash value is in the hash whitelist, a preliminary verification data packet is obtained through hash matching. When the traversal result shows that it is in the hash blacklist, the data packet is intercepted and a security alarm is triggered. The hash whitelist is a hash value list containing known legitimate data packets preset based on experience, and the hash blacklist is a hash value list containing known malicious data packets preset based on experience.

[0068] S13. Check whether the preliminary verification data packet conforms to the expected port of the transmission protocol according to the port number rule. When the preliminary verification data packet does not conform to the expected port of the transmission protocol, mark the data packet as abnormal data, isolate the data packet and trigger re-verification. When the preliminary verification data packet conforms to the expected port of the transmission protocol, mark it as passing the security verification and obtaining the target data of the data packet.

[0069] This step verifies whether the data packet complies with predefined security standards and rules, and initially identifies its legitimacy and security. For example, the system checks whether the data packet comes from a known secure source or matches a known secure data packet pattern, thereby ensuring that the source and content of the data packet meet security requirements. For data packets that do not meet standards or rules, the system directly intercepts and blocks their further transmission, preventing potential security threats or malicious attacks from entering the network.

[0070] It should be noted that the method of performing multiple hash operations on each data field of the original data packet to generate a data packet hash value includes:

[0071] S111. Extract key information of the original data packet, perform a first hash process on the key information of the data packet through the SHA-1 algorithm to obtain the first hash data of the data packet; extract key information of the original data packet such as the timestamp, payload, target IP address and port number of the extracted data; SHA-1 is a hash function that can generate a hash value with a length of 160 bits, which is used to ensure the initial integrity and consistency of the key information of the data packet.

[0072] S112, the first hash value of the data packet is subjected to a secondary hashing process through the SHA-512 algorithm to obtain the second hash data of the data packet, the first hash data of the data packet and the second hash data of the data packet are combined and finally hashed through the SHA-256 algorithm to obtain the data packet hash value. The SHA-512 algorithm provides a longer 512-bit hash value, which can improve the anti-collision property of the data, reduce the probability of different data generating the same hash value, and enhance the security of the system. Different key information of data packets may generate the same hash value after hashing, resulting in hash collision. If two different data blocks generate the same hash value, it will be impossible to distinguish or verify the authenticity of the data, reducing the effectiveness of data integrity protection. A hash function with a low collision probability can more effectively resist certain attacks, such as pre-image attacks, thereby enhancing the security of the system. This embodiment can effectively reduce the collision probability by combining the SHA-1 algorithm and the SHA-512 algorithm. On this basis, re-hashing through the SHA-256 algorithm can retain the anti-collision property of the original data while adding an additional security layer, thereby improving the overall security.

[0073] It should be noted that the method of storing the data packet target data according to the adaptive data storage model to obtain the data packet storage data includes:

[0074] S21. Divide the storage space into storage layers according to a preset number of layers and divide the storage layers into storage blocks according to a preset number of blocks to obtain a storage allocation matrix.

[0075] S22. Calculate the target data of the data packet according to the storage allocation matrix layer index address calculation rule to obtain the storage layer index address, and calculate the target data of the data packet according to the storage allocation matrix layer internal address calculation rule to obtain the storage layer internal address.

[0076] S23. Store the data packet target data as data packet storage data according to the storage layer index address and the storage layer internal address and set the corresponding position to 1.

[0077] S24. Dynamically detect the saturation rate of each layer of storage blocks in the storage allocation matrix. When the storage block storage of the idle layer reaches a preset saturation rate threshold, the idle layer is updated to the saturated layer, and another idle layer storage block is selected to replace the saturated layer storage block. The index address is updated to point the index pointing to the saturated layer storage block to the idle layer storage block.

[0078] S25. Record the retention time of the target data of the data packet in the saturated layer storage block. When the retention time reaches the set preservation period, release the storage block, set the corresponding position to 0 and update the saturated layer to the free layer.

[0079] It should be noted that the method for storing and allocating matrix layer index address calculation rules includes:

[0080] S221, performing a bitwise AND operation on the data packet hash value and the 16-bit mask of all 1s shifted left by a preset number of bits, and retaining the high-order part of the data packet hash value as the storage allocation matrix layer index address.

[0081] S222, the method for storing and allocating address calculation rules in the matrix layer includes:

[0082] The data packet hash value is bitwise ANDed with a low-order mask of a preset number of bits, and the low-order portion of the data packet hash value is retained to obtain an address within the storage allocation matrix layer.

[0083] The preset number of bits is set by the length of the storage block. When the preset number of bits is k, it means that the length of each storage block is byte.

[0084] The k used in this embodiment is 8, that is, the 16-bit mask 0xFFFF of all 1s is shifted left by 8 bits and then ANDed with the data packet, and the high-order part of the data packet hash value is retained to obtain the storage allocation matrix layer index address. For example: when the target data packet information is 0xABCDEF12, the mask is calculated by shifting 8 bits to the left to obtain the mask 0xFF00, and then the hash value and the mask are ANDed and the high-order part is retained, that is, 0xABCDEF12&0xFF00=0xAB00 to obtain the storage allocation matrix layer index address of 0xAB00; the mask within the layer is masked according to the lower 8 bits to obtain the mask 0xFF, and then the hash value and the mask are ANDed and the low-order part is retained, that is, 0xABCDEF12&0xFF=0x12 to obtain the storage allocation matrix layer address of 0x12, that is, record at address 0x12 in layer 0xAB00.

[0085] It should be noted that the method of extracting data characteristic information stored in a data packet and loading it into a behavior detection model to obtain data packet behavior detection data includes:

[0086] S26, extracting data packet storage data feature information including data packet size, data packet time interval, source IP connection frequency, data packet flow rate, protocol type, and port type, normalizing the data packet size, data packet time interval, source IP connection frequency, and data packet flow rate in the data packet storage data feature information, and performing one-hot encoding on the protocol type and port type in the data packet storage data feature information to obtain data packet preprocessing features; the normalization process, for example, uses the minimum and maximum value normalization method to normalize the data, and the one-hot encoding of the protocol type and port type, for example, the protocol type: , , , port type: , , , thereby converting it into a set of binary features.

[0087] S27. Load the data packet preprocessing features into the behavior detection model to obtain the data packet anomaly category and the data packet anomaly ratio value; the anomaly category includes: malware propagation, phishing attack, SQL injection, cross-site scripting attack, and denial of service attack.

[0088] S28, comparing the data packet anomaly ratio value with the corresponding anomaly category tolerance threshold, and when the data packet anomaly ratio value is greater than the anomaly category tolerance threshold, determining it as an abnormal data packet and taking the corresponding anomaly category as the behavior detection result.

[0089] For example: Packet size Bytes, using UDP protocol, packet time interval Seconds, packet flow rate Packets per minute, source IP connection frequency Packets per minute, port number is port 80, and the normalized data obtained after minimum and maximum normalization is the data packet size , Packet time interval , Packet Traffic Rate , Source IP connection frequency , one-hot encoding , , that is, the data packet preprocessing characteristics are ,Load the data into the anomaly monitoring model to obtain the output data packet anomaly category and data packet anomaly ratio values: malware propagation , phishing attack , SQL injection , cross-site scripting attack , denial of service attack , when the anomaly category tolerance threshold is malware propagation , phishing attack , SQL injection , cross-site scripting attack , denial of service attack A denial of service attack occurs when Exceeding its threshold , and the behavior detection result is a denial of service attack.

[0090] The behavior detection model training step includes:

[0091] A1. Set the input layer and input the training set ;in is the data packet preprocessing feature training set, For sample Packet preprocessing features, For sample The data packet anomaly category, N is the number of samples.

[0092] A2. The hidden layer activation function is set to the Softmax function.

[0093] A3. The output layer sets 5 neuron nodes to output 5 types of data packet anomalies respectively.

[0094] A4. Compile the model through the binary cross entropy loss function. For each type of target data packet information feature data anomaly judgment information predicted by the model, the loss is calculated as follows:

[0095] .

[0096] in, is the number of samples, is the number of packet anomaly categories, is a training set sample In the packet anomaly category The true value of is a training set sample In the packet anomaly category The predicted value of .

[0097] A5. Use the Adam optimizer to adjust the learning rate for parameter fitting.

[0098] A6. Use the cross-validation set data to validate the model, adjust the model architecture based on the validation set data performance, and iteratively optimize the model parameters.

[0099] It should be noted that the method of generating a tracing request based on the data packet behavior detection data and sending it to the upstream target node, the upstream target node generating a tracing data packet based on the tracing request, transmitting the tracing data packet to the source node through the dynamic network adaptation model, the source node generating and returning the tracing data to the node issuing the tracing request, and obtaining the data information in the tracing data packet to obtain the abnormal target source includes:

[0100] S31. According to the data packet behavior detection data, the request information matching the abnormal category is obtained from the preset abnormal category tracing request information list and sent to the upstream target node; the preset abnormal category tracing request information list is set based on experience to facilitate the information needed to trace the abnormal behavior; different request information is set for each abnormal category, which is conducive to efficiently obtaining key data related to the corresponding abnormal behavior, thereby accelerating the location and analysis of the abnormal source. According to the detected abnormal category, the corresponding tracing request information is obtained, for example, the malware propagation request information includes requesting the target node to transmit the file transfer log, requesting to check whether there is a file upload or download record, and the denial of service attack request information includes requesting the target node traffic data and requesting the target server load data.

[0101] S32. Obtain network indicators of the path segments between upstream nodes and nodes, input the network indicators of the path segments between upstream nodes and nodes into the dynamic network adaptation model to obtain the tracing path through the shortest path algorithm, slice the tracing data packets according to the tracing path to obtain the tracing data packet slice set, determine the frequency of tracing data packet issuance according to the number of tracing data packet slices in the tracing data packet slice set, and each upstream node transmits the tracing data packet slice set forward along the tracing path in sequence until it reaches the source node according to the tracing data packet issuance frequency; monitor network indicators in real time through the dynamic network adaptation model and dynamically adjust the transmission path of the tracing data packet to ensure that when encountering network congestion, the data packet can bypass the bottleneck or faulty node and continue to be transmitted along the optimal path, and further dynamically adjust the number of data packet slices and the issuance frequency according to the optimal path network status to ensure the efficiency and accuracy of the tracing process.

[0102] S33. The source node generates and returns the traceability data to the node that issues the traceability request, obtains the data information in the traceability data packet, and obtains the abnormal target source.

[0103] It should be noted that the network indicators of the path segment between the upstream node and the node include bandwidth, delay and load; the method of inputting the network indicators of the path segment between the upstream node and the node into the dynamic network adaptation model to obtain the tracing path through the shortest path algorithm, slicing the tracing data packet according to the tracing path to obtain the tracing data packet slice set, and determining the frequency of tracing data packet issuance according to the number of tracing data packet slices in the tracing data packet slice set includes:

[0104] S321, obtaining the bandwidth, delay and load of the path segment between nodes through network node exchange information and performing maximum and minimum value normalization processing to obtain the normalized bandwidth value, delay value and load value, and obtaining the path segment efficiency according to the normalized bandwidth value, delay value and load value for:

[0105] .

[0106] in Is a node To Node The normalized bandwidth value of the path segment between Is a node To Node The normalized load value of the path segment between Is a node To Node The normalized delay value of the path segment between is the path segment performance.

[0107] The path segment efficiency is used to measure the quality of the path segments between nodes. When the path segment bandwidth is higher, the load is lower, and the delay is smaller, the path segment efficiency is higher, that is, it is more suitable for the transmission of data packets.

[0108] S322. Initialize the comprehensive path segment efficiency to zero, start from the node where the tracing request is issued, calculate the path segment efficiency of all directly connected nodes, mark the node where the tracing request is issued as visited, select the node corresponding to the minimum value of the path segment efficiency of the adjacent nodes as the iteration target node and update the comprehensive path segment efficiency according to the path segment efficiency to obtain the comprehensive path segment efficiency iteration value.

[0109] S323. Mark the nodes directly connected to the node issuing the traceability request as visited. Starting from the iteration target node, calculate the path segment efficiencies of all directly connected nodes and repeat the next iteration target node selection and comprehensive path segment efficiency iteration value update until all nodes are traversed and the iteration is stopped. When all nodes are marked as visited, the current comprehensive path segment efficiency iteration value is used as the final comprehensive path segment efficiency. The path segments between the iteration target nodes corresponding to the final comprehensive path segment efficiency are obtained to obtain the traceability path.

[0110] By calculating the efficiency of the path segments and selecting the most efficient path segment through the shortest path algorithm, it is ensured that the traceability data is transmitted along the optimal path, thus ensuring the high efficiency of traceability.

[0111] S324: Select the path with the minimum path segment efficiency in the traceability path as the bottleneck path segment, and slice the traceability data packet according to the bandwidth and load of the bottleneck path segment to obtain the number of traceability data packet slices. for:

[0112] .

[0113] in is the bandwidth of the bottleneck path segment, is the load of the bottleneck path segment, It is The size of the node tracing data packet, It is The number of slices of the node traceability data packet, Yes Round up.

[0114] For example: the size of a packet at an upstream node ,Bandwidth of bottleneck path segment ,load , then the number of slices indivual.

[0115] S325, slicing the source data packet according to the number of source data packet slices to obtain a source data packet slice set, and obtaining the slice transmission time according to the number of source data packet slices in the source data packet slice set, the bandwidth and load of the bottleneck path segment. for:

[0116] .

[0117] in It is The slice transmission time of the traceability data packet of each node.

[0118] For example: When the size of a packet at an upstream node , number of slices Available Second.

[0119] S326: Obtain the frequency of tracing data packets sent out based on the slice transmission time and bottleneck path segment delay for:

[0120] .

[0121] in It is The frequency of sending tracing packets from each node, is the bottleneck path segment delay.

[0122] For example: When the slice transfer time is Seconds, bottleneck path segment delay seconds, the frequency of tracing data packets is , that is, a traceability data packet slice is sent every 0.98 seconds.

[0123] The bottleneck path segment is the path segment with the lowest efficiency in the traceability path, and its bandwidth is limited. When the path segment load is high, directly sending large data packets will cause network congestion or even packet loss. Slicing can decompose large data packets into smaller data packets, and dynamically adjust the number of slices according to the actual network conditions, thereby improving transmission stability and efficiency. At the same time, during the traceability process, network delay and bandwidth will affect the transmission speed of data packets. If the interval between slice issuance is small, that is, the issuance frequency is too high, the number of slices transmitted per second will exceed the bandwidth of the bottleneck path segment, causing network congestion and packet loss. If the issuance frequency is too low, the transmission efficiency will be low. Adjusting the issuance frequency according to the actual network conditions can ensure that the slices can be seamlessly connected in the network, adapt to changes in network bandwidth and delay, and maximize the utilization efficiency of network resources, thereby achieving efficient data transmission and low latency.

[0124] Finally, it should be noted that: Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for big data information security verification and anomaly tracing, characterized in that: The method comprises: Obtaining the original data packet and performing multiple hashing processes to convert the original data packet into a data packet hash value, and performing security verification according to the data packet hash value through preset security verification rules to obtain the data packet target data; The target data packet is stored according to the adaptive data storage model to obtain data packet storage data, and the characteristic information of the data packet storage data is extracted and loaded into the behavior detection model to obtain the data packet behavior detection data; the behavior detection model is a mapping relationship between the data packet preprocessing characteristics and the data packet anomaly category and the data packet anomaly ratio value established by the multi-layer perceptron algorithm based on the characteristic information of the data packet storage data collected historically; Generate a tracing request based on the data of packet behavior detection and send it to the upstream target node. The upstream target node generates a tracing data packet based on the tracing request, transmits the tracing data packet to the source node through the dynamic network adaptation model, and the source node generates and returns the tracing data to the node that issued the tracing request, obtains the data information in the tracing data packet, and obtains the abnormal target source; The method for storing the data packet target data according to the adaptive data storage model to obtain the data packet storage data comprises: Dividing the storage space into storage layers according to a preset number of layers and dividing the storage layers into storage blocks according to a preset number of blocks to obtain a storage allocation matrix; The target data of the data packet is calculated according to the storage allocation matrix layer index address calculation rule to obtain the storage layer index address; Calculate the target data of the data packet according to the address calculation rule in the storage allocation matrix layer to obtain the address in the storage layer; Store the data packet target data as data packet storage data according to the storage layer index address and the storage layer internal address and set the corresponding position to 1; Dynamically detect the saturation rate of each layer of storage blocks in the storage allocation matrix, and when the storage of the storage block of the idle layer reaches a preset saturation rate threshold, update the idle layer to the saturated layer, select another idle layer storage block to replace the saturated layer storage block, and update the index address to point the index pointing to the saturated layer storage block to the idle layer storage block; The target data retention time of the data packet in the saturated layer storage block is recorded. When the retention time reaches the set preservation period, the storage block is released, the corresponding position is set to 0 and the saturated layer is updated to the free layer.

2. A method for big data information security verification and abnormality tracing according to claim 1, characterized in that: The method of obtaining the original data packet and performing multiple hash processing to convert the original data packet into a data packet hash value, and performing security verification according to the data packet hash value through a preset security verification rule to obtain the data packet target data includes: Perform multiple hash operations on each data field of the original data packet to generate a data packet hash value; The data packet hash value is matched with the preset hash whitelist and hash blacklist. When the traversal result shows that the data packet hash value is in the hash whitelist, the data packet is initially verified through hash matching. When the traversal result shows that it is in the hash blacklist, the data packet is intercepted and a security alarm is triggered. The preliminary verification data packet is checked according to the port number rules to see if it meets the expected port of the transmission protocol. When the preliminary verification data packet does not meet the expected port of the transmission protocol, the data packet is marked as abnormal data, the data packet is isolated and re-verification is triggered. When the preliminary verification data packet meets the expected port of the transmission protocol, it is marked as passing the security verification and obtaining the target data of the data packet.

3. A method for big data information security verification and abnormality tracing according to claim 2, characterized in that: The method of performing multiple hash operations on each data field of the original data packet to generate a data packet hash value comprises: Extract key information of the original data packet, and perform a first hash process on the key information of the data packet using the SHA-1 algorithm to obtain first hash data of the data packet; The first hash value of the data packet is subjected to a secondary hash process using the SHA-512 algorithm to obtain the second hash data of the data packet, and the first hash data of the data packet and the second hash data of the data packet are combined and finally hashed using the SHA-256 algorithm to obtain the hash value of the data packet.

4. A method for big data information security verification and abnormality tracing according to claim 1, characterized in that: The storage allocation matrix layer index address calculation rules include: Perform a bitwise AND operation on the data packet hash value and the 16-bit mask of all 1s shifted left by a preset number of bits, and retain the high-order part of the data packet hash value as the storage allocation matrix layer index address; The address calculation rules in the storage allocation matrix layer include: The data packet hash value is bitwise ANDed with a low-order mask of a preset number of bits, and the low-order portion of the data packet hash value is retained to obtain an address within the storage allocation matrix layer.

5. A method for big data information security verification and abnormality tracing according to claim 4, characterized in that: The method of extracting data characteristic information stored in a data packet and loading it into a behavior detection model to obtain data packet behavior detection data includes: Extracting data packet storage data feature information including data packet size, data packet time interval, source IP connection frequency, data packet flow rate, protocol type, and port type, normalizing the data packet size, data packet time interval, source IP connection frequency, and data packet flow rate in the data packet storage data feature information, and performing one-hot encoding on the protocol type and port type in the data packet storage data feature information to obtain data packet preprocessing features; Loading the data packet preprocessing features into the behavior detection model to obtain data packet anomaly categories and data packet anomaly ratio values; the anomaly categories include: malware propagation, phishing attacks, SQL injections, cross-site scripting attacks, and denial of service attacks; The data packet abnormality ratio value is compared with the corresponding abnormal category tolerance threshold. When the data packet abnormality ratio value is greater than the abnormal category tolerance threshold, it is determined as an abnormal data packet and the corresponding abnormal category is used as the behavior detection result; The behavior detection model training step includes: Set up the input layer and input the training set ;in is the data packet preprocessing feature training set, For sample Packet preprocessing features, For sample The packet anomaly category, is the number of samples; The hidden layer activation function is set to the Softmax function; The output layer sets 5 neuron nodes to output 5 types of data packet anomalies respectively; The model is compiled by the binary cross entropy loss function. For each type of target data packet information feature data anomaly judgment information predicted by the model, the loss is calculated as follows: ； in, is the number of samples, is the number of packet anomaly categories, is a training set sample In the packet anomaly category The true value of is a training set sample In the packet anomaly category The predicted value of Use the Adam optimizer to adjust the learning rate for parameter fitting; Use cross-validation set data to validate the model, adjust the model architecture based on the performance of the validation set data, and iteratively optimize the model parameters.

6. A method for big data information security verification and abnormality tracing according to claim 1, characterized in that: The method of generating a tracing request according to the data packet behavior detection data and sending it to the upstream target node, the upstream target node generating a tracing data packet according to the tracing request, transmitting the tracing data packet to the source node through the dynamic network adaptation model, the source node generating and returning the tracing data to the node issuing the tracing request, and obtaining the data information in the tracing data packet to obtain the abnormal target source includes: According to the data packet behavior detection data, the request information matching the abnormal category is obtained from the preset abnormal category tracing request information list and sent to the upstream target node; Obtain the network indicators of the path segments between the upstream nodes and the nodes, input the network indicators of the path segments between the upstream nodes and the nodes into the dynamic network adaptation model to obtain the tracing path through the shortest path algorithm, slice the tracing data packets according to the tracing path to obtain the tracing data packet slice set, determine the frequency of tracing data packets being sent according to the number of tracing data packet slices in the tracing data packet slice set, and each upstream node transmits the tracing data packet slice set forward in sequence along the tracing path until it reaches the source node according to the tracing data packet sending frequency; The source node generates and returns the traceability data to the node that issues the traceability request, obtains the data information in the traceability data packet, and obtains the abnormal target source.

7. A method for big data information security verification and abnormality tracing according to claim 6, characterized in that: The network indicators of the path segment between the upstream node and the node include bandwidth, delay and load; the method of inputting the network indicators of the path segment between the upstream node and the node into the dynamic network adaptation model to obtain the tracing path through the shortest path algorithm, slicing the tracing data packet according to the tracing path to obtain the tracing data packet slice set, and determining the frequency of tracing data packet issuance according to the number of tracing data packet slices in the tracing data packet slice set includes: The bandwidth, delay and load of the path segment between nodes are obtained by exchanging information between network nodes, and the maximum and minimum values ​​are normalized to obtain the normalized bandwidth value, delay value and load value. The path segment performance is obtained based on the normalized bandwidth value, delay value and load value. for: ； in Is a node To Node The normalized bandwidth value of the path segment between Is a node To Node The normalized load value of the path segment between Is a node To Node The normalized delay value of the path segment between is the path segment effectiveness; Initialize the comprehensive path segment efficiency to zero, start from the node where the traceability request is issued, calculate the path segment efficiency of all directly connected nodes, mark the node where the traceability request is issued as visited, select the node corresponding to the minimum value of the path segment efficiency of the adjacent nodes as the iteration target node, and update the comprehensive path segment efficiency according to the path segment efficiency to obtain the comprehensive path segment efficiency iteration value; Mark the nodes directly connected to the node issuing the traceability request as visited, start from the iteration target node, calculate the path segment effectiveness of all directly connected nodes and repeat the next iteration target node selection and comprehensive path segment effectiveness iteration value update until all nodes are traversed and the iteration is stopped. When all nodes are marked as visited, the current comprehensive path segment effectiveness iteration value is used as the final comprehensive path segment effectiveness, and the path segments between each iteration target node corresponding to the final comprehensive path segment effectiveness are obtained to obtain the traceability path; Select the path with the minimum path segment efficiency in the traceability path as the bottleneck path segment, and slice the traceability data packet according to the bandwidth and load of the bottleneck path segment to obtain the number of traceability data packet slices. for: ； in is the bandwidth of the bottleneck path segment, is the load of the bottleneck path segment, It is The size of the node tracing data packet, It is The number of slices of the node traceability data packet, Yes Round up; The source data packet is sliced ​​according to the number of source data packet slices to obtain a source data packet slice set, and the slice transmission time is obtained according to the number of source data packet slices in the source data packet slice set, the bandwidth and load of the bottleneck path segment. for: ； in It is The slice transmission time of the traceability data packet of each node; The frequency of tracing data packets is obtained based on the slice transmission time and bottleneck path segment delay. for: ； in It is The frequency of sending tracing packets from each node, is the bottleneck path segment delay.

Images