Distributed load balancing data encryption storage method based on SDN

Abstract

The application relates to the technical field of cyberspace security, in particular to a distributed load balancing data encryption storage method based on SDN. On the basis of the original industrial internet network, the topology structure is not changed, data is directly transmitted to an SDN processing device in a series connection mode or by using a switch port mirroring mode, firstly, when the data is directly transmitted in the industrial internet device, the data can be captured by the SDN processing device; then, according to a flow, the data is transmitted to an SDN-based log encryption storage server group through a log storage network-related infrastructure; after the SDN-based log encryption storage server group completes encryption storage, relevant index bases are transmitted to a log query server; finally, the log query server can provide full-amount network data analysis support for other security unified risk management related systems. The application solves the technical problem of difficult collection of length-biased small data packets under the condition of large network flow of the existing industrial internet.

Description

Technical Field

[0001] This invention relates to the field of cyberspace security technology, specifically to a method for encrypted storage of distributed load-balanced data based on SDN. Background Technology

[0002] With the development of the Industrial Internet, numerous network attacks have emerged. To effectively analyze these attacks, identify security vulnerabilities, and upgrade equipment, it is necessary to collect network data packets from the network flow. However, the Industrial Internet often contains a large number of short data packets (small packets), making traditional data collection methods inefficient and prone to data packet omissions due to insufficient network card throughput.

[0003] At the same time, due to the large number of data packets contained in the data stream, how to effectively query these data packets is also a problem that needs to be solved.

[0004] In addition, in accordance with high-level network security protection requirements, files in persistent storage must be stored in an encrypted manner to address the problem of data leakage caused by the loss of storage. Summary of the Invention

[0005] The purpose of this invention is to provide a distributed load balancing data encryption and storage method based on SDN, which aims to solve the technical problem of difficulty in collecting short data packets under the large network traffic of the existing industrial Internet, and to complete the encryption and storage during the collection process.

[0006] To achieve the above objectives, this invention provides a method for encrypted storage of distributed load-balanced data based on SDN, comprising the following steps:

[0007] After data acquisition, the data flow is mirrored.

[0008] Enter the load balancing management process;

[0009] Simultaneously, continuous data storage is used to assist the process;

[0010] The network data distributed storage process is initiated, and the data packet files that have completed the stage of data storage are encrypted using a random key;

[0011] After the distributed storage process for network data is completed, the log query server begins auditing the log data.

[0012] The mirrored data process is implemented by connecting an SDN processing device in series with the network that needs to capture data or by using the mirroring function of a switch port, in order to acquire data in the transmission link of industrial Internet devices.

[0013] Specifically, the process of the mirrored data flow involves the data packets being copied after passing through the SDN processing device. If the current connection mode is serial mode, the SDN processing device will send the copied data packets back to the original network through the other network interface to enter the next process; if the connection mode is port mirroring mode, it will directly enter the next process.

[0014] Before the network data distributed storage process is started, a load balancing management process is needed to perform server pressure statistics, and a continuous data storage auxiliary process is used to improve reading efficiency.

[0015] In the load balancing management process, the main indicator of server stress data is server memory utilization, and the secondary indicators are server CPU utilization and server disk utilization. The stress values ​​are calculated using the server stress formula and then sorted and statistically analyzed.

[0016] The data continuous storage auxiliary process uses a four-tuple storage historical record temporary database to provide an access interface to the network data distributed storage process, and periodically clears outdated historical records during execution.

[0017] The network data distributed storage process includes the following steps:

[0018] Perform preprocessing to obtain the pointer to the data packet in memory;

[0019] Extract the quadruple from the pointer;

[0020] Obtain the hash value based on the quadruple;

[0021] The server for storing data packets is initially selected based on a Bloom filter method using hash values.

[0022] The available servers for storage are determined based on the stress statistics collected during the load balancing management process.

[0023] The data packets are stored in the storable server, and counting is performed simultaneously in the continuous data storage auxiliary process;

[0024] The storable server reads data packets into its local memory and feeds back information about the current server load.

[0025] A storable server stores data packets from memory into a storage device for encrypted storage;

[0026] When the number of stored files accumulates to a specified time or a specified quantity, the data packet is encrypted using a random key.

[0027] The random key, the current storage server identifier, and the four-tuple information are sent to the log query server for storage, and a fast query index is generated.

[0028] The process of the log query server performing log data auditing includes the following steps:

[0029] Select the data to query based on the index information;

[0030] The encrypted data packet is retrieved from the storage device used for encrypted storage based on the index association;

[0031] Decrypt the encrypted data packet;

[0032] The log query server merges the decrypted data packets and returns them to the requester.

[0033] This invention provides a distributed load balancing data encryption and storage method based on SDN. Without altering the topology of the existing industrial internet network, data is directly transmitted to the SDN processing device via serial connection or port mirroring on a switch. First, when data is transmitted directly through industrial internet devices, it can be captured by the SDN processing device. Then, according to the process, data is transmitted via log storage network infrastructure to an SDN-based log encryption and storage server group. After encryption and storage by the SDN-based log encryption and storage server group, the relevant index is transmitted to a log query server. Finally, the log query server can provide full network data analysis support for other security and unified risk management systems. This invention solves the technical problem of difficulty in collecting short data packets under high network traffic conditions in the existing industrial internet. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram illustrating the problem-solving approach and overall architecture of the SDN-based distributed load balancing data encryption and storage method of the present invention.

[0036] Figure 2 This is a flowchart illustrating a distributed load balancing data encryption and storage method based on SDN according to the present invention. Detailed Implementation

[0037] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0038] This invention provides a method for encrypted storage of distributed load balancing data based on SDN, comprising the following steps:

[0039] S1: Start mirroring the data flow after data acquisition;

[0040] S2: Entering the load balancing management process;

[0041] S3: Simultaneously use continuous data storage to assist the process;

[0042] S4: The network data distributed storage process is initiated, and the data packet files that have completed the stage of data storage are encrypted using a random key;

[0043] S5: After the distributed storage process for network data is completed, the log query server begins to audit the log data.

[0044] Please refer to the detailed architecture and process diagram. Figure 1 and Figure 2 Regarding the industrial internet data rapid acquisition and secure storage architecture proposed in this invention, as follows: Figure 1 As shown in the diagram, firstly, when data is directly transmitted between industrial internet devices, it can be captured by SDN processing devices. Then, using the process proposed in this invention, the data is transmitted via log storage network infrastructure to an SDN-based encrypted log storage server group. Afterward, the SDN-based encrypted log storage server group completes encrypted storage and transmits the relevant index data to a log query server. Finally, the log query server can provide full-scale network data analysis support for other security-related unified risk management systems.

[0045] The present invention will be further described below with reference to specific implementation steps:

[0046] The data acquisition, rapid storage, load balancing, and query processes of this invention, along with their interrelationships, comprise five main processes: mirrored data process, distributed network data storage process, continuous data storage auxiliary process, load balancing management process, and log query process. The interrelationships between these processes are shown in the diagram below. Figure 2 As shown.

[0047] 1. The mirrored data process is mainly used to acquire data in the transmission link of industrial Internet devices. It can be implemented by connecting the SDN processing device in series to the network where data needs to be captured or by using the port mirroring function of the switch.

[0048] 1.1 First, the mirroring data process can only proceed after data transfer between devices has begun.

[0049] 1.2 When data packets transmitted between industrial internet devices pass through the SDN processing device, the processor quickly begins the data replication operation.

[0050] 1.3 First, the SDN processing device reads data from the network interface.

[0051] 1.4 Subsequently, the SDN processing device quickly copies the read data into memory. At this time, the copied data is the structure data in the SDN framework.

[0052] After step 1.5, the SDN processing device needs to confirm the current connection's operating mode. If it's in serial mode, it needs to send data packets, so it needs to proceed to step [1.61], complete step [1.62], and then proceed to step [1.7]. If it's in port mirroring mode, it directly proceeds to step [1.7].

[0053] The 1.61 SDN processing device quickly places the copied data packets into the data packet sending queue of the network interface on the other side of the original network.

[0054] 1.62 The SDN processing device sends data packets from each sending queue, thereby minimizing the impact on the original network during data replication.

[0055] 1.7 The SDN processing device's data mirroring process notifies the network data distributed storage process to begin processing.

[0056] 2. Before the distributed data storage process begins, the load balancing management process must be completed first. The log query server, in addition to providing log data querying and retrieval, also acts as the load balancer for the entire SDN-based encrypted log storage server group.

[0057] 2.1 First, the log query server collects stress data from all servers in the SDN-based log encryption storage server group. Its primary metric is server memory utilization, and its secondary metrics are server CPU utilization and server disk utilization.

[0058] 2.2 Based on the server pressure formula = Memory pressure weight * Server memory utilization + CPU pressure weight * Server CPU utilization + Disk pressure weight * Server disk utilization, the current server pressure is sorted. Furthermore, to ensure the continuous storage of data packets with the same four-tuple (source IP address, destination IP address, source port, and destination port) for a longer period, the weight of memory pressure is increased in this invention to ensure sufficient data packet cache space.

[0059] 2.3 Place server stress statistics into the query interface for use in the distributed network data storage process.

[0060] 3. In addition, in order to ensure that data packets can be stored continuously under certain server pressure to improve reading efficiency, this invention also proposes to use a data continuous storage auxiliary process to achieve two-level pressure load balancing management, on the premise of using load balancing management.

[0061] 3.1 Once the distributed network data storage process confirms the destination storage server for the current data packet, it needs to place the four-tuple hash, destination storage server, and storage count information from the current data packet into a temporary database of the four-tuple storage history records that have already been stored. The structure of the stored data is as follows:

[0062] Quadruple Hash Destination storage server Storage count

[0063] 3.2 The quadruple storage historical record temporary database provides an access interface for distributed network data storage processes.

[0064] 3.3 To ensure that historical records are kept relatively recent, while alleviating pressure on temporary storage and improving the efficiency of historical record access, this invention uses a method of periodically clearing outdated historical records to solve the above problems. During process execution, this step is performed periodically, directly reducing the storage count by a certain value.

[0065] 3.4 When the storage count decreases to less than 0, delete the historical record corresponding to the four-tuple hash.

[0066] 4. After the log query server collects and generates stress statistics for the SDN-based encrypted log storage server group, the distributed network data storage process will begin.

[0067] 4.1 After the network data distributed storage process of the SDN processing device starts, the data packets need to be preprocessed first to extract the network data packet pointers from the original SDN framework processing structure data in the data packets copied in the data mirroring process.

[0068] 4.2 Extract the quadruple of the data packet from the data packet pointer.

[0069] 4.3 Based on the quadruple information, use the Hash method to obtain the Hash value: Hash{source IP address + destination IP address + source port + destination port}.

[0070] 4.4 In the Bloom filter-based method, the server where the data packets are to be stored is first selected based on the hash value.

[0071] 4.5 Subsequently, obtain the load statistics of the target data storage server from the load balancing management thread (related to [2.3]).

[0072] 4.5.1 If the pressure on the target data storage server is less than the set pressure threshold, proceed to [4.6].

[0073] 4.5.2 If the target data storage server pressure is greater than or equal to the set pressure threshold, then query whether the quadruple of the current data packet is in the recent activity history of the data continuous storage auxiliary process (related to [3.2]). This step is mainly used to enable the storage server to focus on processing recently active data connections when the storage server is close to high pressure, reducing the problem of other scattered network data streams causing a surge in storage pressure on a single server, so as to save the data packets corresponding to the quadruple within a continuous time period.

[0074] 4.5.2.1 If the quadruple of the currently processed data packet is in the recent activity history, then it is directly accessed [4.6].

[0075] 4.5.2.2 If the quadruple of the currently processed packet is not in the recent activity history, the Bloom filter method is used to recalculate the hash to identify the next server that can be stored, and then jump to [4.5].

[0076] 4.6 After selecting a server with low storage pressure, the current data packet is sent to the server with low storage pressure. In the continuous data storage auxiliary process, the corresponding storage count is incremented by one based on the quadruple hash and the destination storage server information.

[0077] 4.7 Subsequently, when the selected SDN-based log encryption storage server receives the transmitted network data packet, it quickly reads the current data packet from the network card into memory and promptly sends feedback on the current server pressure information to the log data query (related to [2.1]).

[0078] 4.8 Subsequently, the packet saving thread of the SDN-based log encryption storage service server saves the network packets that need to be processed in memory to the storage of the current SDN-based log encryption storage service server.

[0079] 4.9 Subsequently, when the files stored by the SDN-based log encryption storage server accumulate to a specified time or a specified number, the filename of the stored file will be changed, and the data packet file that has completed the stage of data storage will be encrypted using a random key.

[0080] 4.10 The SDN-based encrypted log storage server sends the random key, current storage server identifier, and four-tuple information of the continuous, phased storage data packets to the log query server for storage.

[0081] 4.11 After receiving the random key, the current storage server identifier, and the four-tuple information, the log query server records the random key, the current storage server identifier, and the four-tuple information in its database and generates a fast query index so that other processes can quickly obtain the data and perform related security analysis operations.

[0082] 5. Once the distributed storage of network data is complete, log data auditing can begin.

[0083] 5.1 First, the packet analysis program or staff can select the data to be queried from the log query server based on the index information.

[0084] 5.2 Subsequently, the log query server pulls encrypted data packets from the SDN-based log encryption storage server group associated with the selected index information.

[0085] After version 5.3, the log query server will retrieve and decrypt the completed encrypted data packets.

[0086] 5.4 Log query server merges decrypted data packets.

[0087] 5.5 The log query server returns the decrypted data packet to the requester.

[0088] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A method for SDN-based distributed load balancing data encryption preservation, characterized in that, Includes the following steps: After data acquisition, the data flow is mirrored. The load balancing management process is initiated, in which the log query server acts as the load balancing pressure manager for the SDN-based log encrypted storage server group. It collects the pressure data of each server in the log encrypted storage server group, sorts the servers according to the server pressure formula, and provides the server pressure statistics data to the network data distributed storage process. The main indicator of the server pressure data is the server memory utilization rate, and the secondary indicators are the server CPU utilization rate and the server disk utilization rate. The mirrored data packets are preprocessed to obtain a pointer to the data packets in memory, and a four-tuple is extracted from the pointer, wherein the four-tuple includes the source IP address, the destination IP address, the source port, and the destination port. The hash value is obtained based on the quadruple; Simultaneously, continuous data storage is used to assist the process; The network data distributed storage process is initiated, and the data packet files that have completed the stage of data storage are encrypted using a random key; Based on the hash value, a Bloom filter method is used to initially select the target server to store the current data packet; Obtain the pressure statistics of the target server. When the pressure of the target server is less than the set pressure threshold, the target server is identified as a storable server. When the pressure of the target server is greater than or equal to the set pressure threshold, check whether the quadruple of the current data packet exists in the recent activity history of the data continuous storage auxiliary process. If it exists, the target server is identified as a storable server. If it does not exist, calculate the hash again based on the Bloom filter method to confirm the next target server until all storable servers are selected. The current data packet is sent to the storable server, and the corresponding storage count is increased according to the quadruple hash and the destination storage server information in the data continuous storage auxiliary process. The storable server reads the received data packets into its local memory and feeds back the current server load information. Then, it stores the data packets in memory into a storage device for encrypted storage. When the stored files accumulate to a specified time or a specified number, the data packet files that have completed the stage of data storage are encrypted using a random key. The random key, the current storage server identifier, and the four-tuple information are sent to the log query server for storage, and a fast query index is generated. After the distributed storage process for network data is completed, the log query server begins auditing the log data. During log data auditing, the data to be queried is selected according to the fast query index, and the encrypted data packets are pulled from the SDN-based log encryption storage server group according to the index association. The encrypted data packets are decrypted, and the log query server merges the decrypted data packets and returns them to the requester.

2. The SDN-based distributed load balancing data encryption and storage method as described in claim 1, characterized in that, The image data process is implemented by connecting the SDN processing device in series with the network that needs to capture data or by using the port mirroring function of the switch, and is used to obtain data in the transmission link of industrial Internet devices.

3. The SDN-based distributed load balancing data encryption and storage method as described in claim 2, characterized in that, The process of the image data flow is as follows: after the data packets flow through the SDN processing device, a copy operation is performed. If the current connection mode is serial mode, the SDN processing device sends the copied data packets back to the original network through the other network interface to enter the next process; if the connection mode is port mirroring mode, it directly enters the next process.

4. The SDN-based distributed load balancing data encryption and storage method as described in claim 1, characterized in that, The data continuous storage auxiliary process uses a quadruple storage historical record temporary database to provide an access interface to the network data distributed storage process, and periodically clears outdated historical records during execution.

Images