Method and system for remotely mirroring traffic

Abstract

The invention discloses a method and system for remotely mirroring traffic, and the method comprises the steps: at a traffic import end, transferring the traffic imported into a network card to a first tunnel network card through a first traffic mirroring program; the first tunnel program segments the traffic received by the first tunnel virtual network card and re-packages the traffic into new UDP protocol messages one by one, and the new UDP protocol messages are sent to the traffic output end through the first tunnel physical network card; at a flow output end, a second tunnel program receives the message intercepted by the UDP port, decapsulates the message and sends the message to a second tunnel virtual network card, so that the message is captured by a second flow mirror image program; and transferring the traffic of the second tunnel network card to the output network card through the second traffic mirroring program. According to the method, the traffic torsion capability under the complex network condition is provided, so that the traffic import end and the traffic output end can be at any position and do not depend on the virtualization technology used by the traffic import end and the traffic output end, an intermediate transmission network does not have any special requirement, and the performance of remote mirroring can be improved.

Description

Technical Field

[0001] The invention relates to a method and system for remote mirroring traffic, belonging to the field of security and shooting range. Background Art

[0002] Traffic mirroring refers to forwarding the inbound and outbound traffic of a network card to another network card intact, regardless of the length of the source data, the format, or how it is encapsulated. This technology is supported by mainstream hardware devices such as routers, switches, and firewalls. In the fields of cross-range shooting, security probes, etc., when the traffic of a network card is mirrored to network cards in different physical locations, the two network cards need to cross a complex network environment and cannot be directly connected to the same hardware device. A technology that can support remote mirroring of traffic is needed to meet this requirement.

[0003] The current solution for implementing remote traffic mirroring generally uses OVS to couple common protocols such as GRE and VXLAN to build a network tunnel and mirror the traffic to another segment through the tunnel. This implementation has the following problems:

[0004] 1. Use a common protocol to build a tunnel. The protocol has high characteristics and is easy to identify and intercept. For example, the public cloud will intercept GRE traffic by default, and the vxlan protocol requires vxlan switch support;

[0005] 2. If the mirrored traffic uses the default MTU encapsulation, after adding the tunnel message header, each intermediate transmission device needs to perform superframe processing, which has great limitations;

[0006] 3. In the existing cases, all remote port mirroring is performed for OpenStack virtual machines, so OVS can be used to encapsulate and decapsulate the tunnel protocol. If it is not an OpenStack virtual machine, OVS is a heavy software with high requirements for operating system resources and difficult maintenance;

[0007] 4. OVS is based on Hypervior to forward port traffic, which needs to be operated outside the virtual machine. However, the customer site usually provides a ready-made machine, which usually does not have Hypervior permissions or is simply a physical machine. Traffic mirroring operations can only be performed within the operating system, and the existing solutions will not be able to meet this requirement. Summary of the invention

[0008] Purpose of the invention: In view of the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method and system for remote mirroring traffic to improve the versatility of remote mirroring.

[0009] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides a method for remotely mirroring traffic, comprising the following steps:

[0011] At the traffic import end, the traffic imported into the network card is transferred to the first tunnel virtual network card through the first traffic mirroring program;

[0012] At the traffic import end, the first tunnel program divides the traffic received by the first tunnel virtual network card and re-encapsulates it into new UDP protocol messages one by one, and sends it to the traffic output end through the first tunnel physical network card;

[0013] At the flow output end, the second tunnel program listens to the UDP port of the second tunnel physical network card, decapsulates the intercepted message, restores it to the source message, and sends it to the second tunnel virtual network card, so that it is captured by the second flow mirror program;

[0014] At the traffic output end, the traffic captured from the second tunnel virtual network card is transferred to the traffic output network card through the second traffic mirroring program.

[0015] In a specific embodiment, the first traffic mirroring program is installed by a server of a device that needs to mirror traffic. If the device that needs to mirror traffic cannot install a general operating system to deploy application programs, a server is added to the back end of the device to deploy the first traffic mirroring program.

[0016] In a specific implementation example, a server is added at the front end of the traffic receiving device to deploy the second traffic mirroring program.

[0017] In the specific implementation example, it is recommended to configure a dual network card on the server where the first / second traffic mirroring program is deployed, one network card is used for input / output traffic, and the other network card is used for a traffic reversal tunnel.

[0018] In a specific embodiment, when a server deploying the first / second traffic mirroring program configures a network card, it is necessary to add logic for filtering tunnel traffic in the first / second traffic mirroring program and consider whether the bandwidth upper limit can be met.

[0019] Furthermore, the first tunnel program uses a communicative three-layer header to re-encapsulate the original message according to the set rules, encapsulates the fragment size and sequence number in the fifth layer, and then uses the UDP protocol to encapsulate the slices one by one; the second tunnel program uses the same rules to decapsulate and reassemble the fragments before sending them to the second tunnel virtual network card.

[0020] Furthermore, the first tunnel program encrypts data above the fourth layer using an RSA algorithm, and public and private keys are pre-embedded in the first tunnel program and the second tunnel program.

[0021] In a second aspect, the present invention provides a system for remotely mirroring traffic, comprising:

[0022] The first traffic mirroring program module is deployed at the traffic import end and is used to transfer the traffic of the import network card to the first tunnel virtual network card; the second traffic mirroring program module is deployed at the traffic output end and is used to capture the traffic of the second tunnel virtual network card and transfer it to the output network card;

[0023] The first tunnel program module is deployed at the traffic import end, and is used to split the traffic received by the first tunnel virtual network card and re-encapsulate it into new UDP protocol messages one by one, and send it to the traffic output end through the first tunnel physical network card;

[0024] And, the second tunnel program module is deployed at the traffic output end, used to listen to the UDP port of the second tunnel physical network card, decapsulate the intercepted message, restore it to the source message, and then send it to the second tunnel virtual network card, so as to be captured by the second traffic mirroring program module.

[0025] Furthermore, the first tunnel program uses a communicative three-layer header to re-encapsulate the original message according to the set rules, encapsulates the fragment size and sequence number in the fifth layer, and then uses the UDP protocol to encapsulate the slices one by one; the second tunnel program uses the same rules to decapsulate and reassemble the fragments before sending them to the second tunnel virtual network card.

[0026] Furthermore, the first tunnel program module encrypts the slice data using an RSA algorithm, and public and private keys are pre-embedded in the first tunnel program module and the second tunnel program module.

[0027] Beneficial effects: The method and system for remote mirroring traffic provided by the present invention provide traffic reversal capabilities under complex network conditions, so that the traffic import and export ends can be at any location, independent of the virtualization technology they use, and there are no special requirements for the intermediate transmission network. In addition, when tunnel traffic passes through the public network, non-plaintext transmission can be used, which improves the security of traffic transmission. Compared with the prior art, the present invention has the following advantages:

[0028] 1. The present invention uses pure software to achieve traffic transmission, can be lightly deployed using binary files, has high versatility, has low requirements for the operating environment, does not require the purchase of professional equipment, does not require the deployment of complex software, and has lower operating and maintenance costs.

[0029] 2. Mirror traffic uses the standard UDP protocol to re-encapsulate data packets, which can penetrate any network environment to adapt to various network restriction policies below the fourth layer, such as public networks, firewalls, and proxies.

[0030] 3. Use standard MTU for re-encapsulation, without the need for intermediate network devices to support and configure frame expansion.

[0031] 4. The present invention uses a private protocol to re-encapsulate data and can be further equipped with asymmetric encryption, which improves security when the traffic crosses the public network. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic structural diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments.

[0034] like Figure 1 As shown, a method for remote mirroring traffic disclosed in an embodiment of the present invention involves a traffic import end and a traffic output end, wherein the network card at the traffic import end is used to receive the mirrored traffic, and the network card at the traffic output end is used to pass the traffic to the monitoring probe, and a network tunnel is established between the two ends to reverse the mirrored traffic. Specifically, at the traffic import end, any traffic imported into the network card is transferred to the first tunnel virtual network card through the first traffic mirroring program; the first tunneling program divides and re-encapsulates any traffic received by the first tunnel virtual network card into a new UDP protocol message, and sends it to the traffic output end through the first tunnel physical network card; at the traffic output end, the second tunneling program listens to the UDP port of the second tunnel physical network card, decapsulates the intercepted message, restores it to the source message, and sends it to the second tunnel virtual network card, so that it is captured by the second traffic mirroring program; the traffic captured from the second tunnel virtual network card is transferred to the traffic output network card through the second traffic mirroring program.

[0035] The following describes in detail the two main components of the traffic mirroring program and the tunneling program involved in the solution of the embodiment of the present invention.

[0036] 1. Traffic Mirroring Program

[0037] The traffic mirroring program is a pure software program, including the first traffic mirroring program at the traffic import end and the second traffic mirroring program at the traffic output end. The first traffic mirroring program can usually be installed with the mirrored server at the traffic import end. If the device that needs to mirror the traffic cannot install a general operating system to deploy the application, a server with any specifications and configuration can be added to the back end of the traffic generating device to deploy the first traffic mirroring program; the second traffic mirroring program can also be installed with the device at the traffic output end. For special devices, a server can be added to the front end of the traffic receiving device to deploy the second traffic mirroring program.

[0038] The function of the traffic mirroring program is single. It only transfers the data of one network card of the device to another network card in full. At the traffic import end, it transfers the imported network card traffic to the tunnel virtual network card (such as tap network card, local loopback network card), and at the traffic output end, it transfers the traffic of the tunnel virtual network card to the output network card. The program logic is simple and the CPU overhead is low. It can be implemented using the python scapy library.

[0039] Traffic import and export servers should be configured with dual network cards, one for input / output traffic and the other for traffic reverse tunnel. If conditions limit the use of only one network card, you need to add logic to filter tunnel traffic in the traffic mirroring program. This will not increase CPU overhead too much, but it should be noted that tunnel traffic will occupy the traffic bandwidth of the network card, because the physical network card carries both the import / export traffic and tunnel traffic of the business, and you need to consider whether the bandwidth limit can be met.

[0040] 2. Tunnel Program

[0041] The function of the tunnel is to re-encapsulate the source data packet, re-encapsulate it with a communicable three-layer header, and send the traffic to the other end. The tunnel program includes the first tunnel program at the traffic import end and the second tunnel program at the traffic output end. The first tunnel program is used to re-encapsulate the mirror traffic, and the second tunnel program is used to decapsulate the mirror traffic. Because the UDP header length is only 8 bytes, in order to ensure the performance of the pipeline, the traffic tunnel considers using the UDP protocol as the transport layer protocol. At the traffic import end and the traffic output end, a UDP port (such as UDP port 3000) is started to listen to the messages sent by the other end.

[0042] In the operating systems at both ends, configure the local IP address and UDP port that the tunnel program listens to on the physical network card of the traffic tunnel, configure the IP address and UDP port number of the other end, establish a tunnel relationship between the two ends, and create a tunnel virtual network card for the tunnel program and the traffic mirror program to send and receive traffic.

[0043] At the traffic import end, the first tunnel program will re-encapsulate the message received from the traffic tunnel virtual network card and forward it to the other end, that is, the UDP3000 port at the traffic output end; at the traffic output end, the second tunnel program will decapsulate the UDP3000 message received and send it to the traffic tunnel virtual network card, so that it will be captured by the traffic mirroring program.

[0044] The configuration files for the tunnel programs at both ends are listed below.

[0045] First, the tunnel program creates a virtual network card:

[0046] ip tuntap add dev tap0 mod tap

[0047] ip link set tap0 up

[0048] Assume that the tunnel physical network card IP address of the first traffic mirroring program is 100.100.100.1, and the first tunnel program is configured as follows:

[0049] local_ip:100.100.100.1

[0050] local_port:3000

[0051] dest_ip:100.100.100.2

[0052] dest_port:3001

[0053] mirror_tap:tap0

[0054] The second tunnel program creates a virtual network card:

[0055] ip tuntap add dev tap0 mod tap

[0056] ip link set tap0 up

[0057] Assume that the tunnel physical network card IP address of the second traffic mirroring program is 100.100.100.2, the second tunnel program configuration:

[0058] local_ip:100.100.100.2

[0059] local_port:3001

[0060] dest_ip:100.100.100.1

[0061] dest_port:3000

[0062] mirror_tap:tap0

[0063] The configuration files for the traffic mirroring program are listed below.

[0064] Assume that the NIC alias for traffic import is eth0, and the configuration of the first traffic mirror program is:

[0065] src_port:eth0

[0066] dest_port:tap0

[0067] Assuming that the NIC alias for traffic output is eth0, the configuration of the second traffic mirror program is:

[0068] src_port:tap0

[0069] dest_port:eth0

[0070] Considering the limitation of MTU (maximum transmission unit), the mirrored message needs to be fragmented. The first tunnel program needs to fragment the original message according to certain rules. In order to reduce PPS, the slice cannot be too small. Usually, the MTU of the network device is 1500, and the slice size is 1500 minus the UDP header overhead minus the overhead described by the slice sequence number. The fragment description can be regarded as the fifth layer encapsulation. Considering the uniqueness and statelessness, the slice description protocol is constructed using the control bit + timestamp + sequence bit. The control bit describes whether to slice, the timestamp provides uniqueness, and the sequence number describes the number of slices. The second tunnel program on the other end uses the same rules to decapsulate and reassemble the fragments before sending them to the second tunnel virtual network card. This part of the logic calculation is relatively large and can be implemented in C language.

[0071] If the mirrored traffic needs to pass through the public network, security needs to be considered. Since the tunnel slicing method is a private protocol, it is equivalent to a certain encryption of the message. If it is still not enough, the data above the fourth layer can be encrypted using the RSA algorithm. The public and private keys are pre-embedded in the tunnel program and decrypted before decapsulation. Encryption and decryption will bring a large CPU overhead. It is necessary to pay attention to the PPS of the mirrored message to confirm the CPU overhead.

[0072] Based on the same inventive concept, another embodiment of the present invention discloses a system for remotely mirroring traffic, including: a first traffic mirroring program module, deployed at a traffic import end, for transferring traffic imported into a network card to a first tunnel virtual network card; a second traffic mirroring program module, deployed at a traffic output end, for capturing traffic of a second tunnel virtual network card and transferring it to an output network card; a first tunnel program module, deployed at a traffic import end, for dividing traffic received by the first tunnel virtual network card and re-encapsulating it one by one into new UDP protocol packets, and sending them to the traffic output end through a first tunnel physical network card; and a second tunnel program module, deployed at a traffic output end, for listening to a UDP port of a second tunnel physical network card, decapsulating the intercepted packets, restoring them to source packets, and sending them to the second tunnel virtual network card, so that they are captured by the second traffic mirroring program module. Furthermore, the first tunnel program re-encapsulates the original message using a communicative layer 3 header, fragments the original message according to the set rules, and encapsulates the fragment size and sequence number in the fifth layer, and then uses the UDP protocol to encapsulate the slices one by one; the second tunnel program uses the same rules to decapsulate and reassemble the slices before sending them to the second tunnel virtual network card. Furthermore, the first tunnel program module encrypts the slice data using the RSA algorithm, and the public and private keys are pre-embedded in the first tunnel program module and the second tunnel program module.

Claims

1. A method for remotely mirroring traffic, characterized in that: The steps include: At the traffic import end, the traffic imported into the network card is transferred to the first tunnel virtual network card through the first traffic mirroring program; At the traffic import end, the first tunnel program divides the traffic received by the first tunnel virtual network card and re-encapsulates it into new UDP protocol messages one by one, and sends it to the traffic output end through the first tunnel physical network card; At the flow output end, the second tunnel program listens to the UDP port of the second tunnel physical network card, decapsulates the intercepted message, restores it to the source message, and sends it to the second tunnel virtual network card, so that it is captured by the second flow mirroring program; At the traffic output end, the traffic captured from the second tunnel virtual network card is transferred to the traffic output network card through the second traffic mirroring program.

2. A method for remotely mirroring traffic according to claim 1, characterized in that: The first traffic mirroring program is installed by the server of the device that needs to mirror the traffic. If the device that needs to mirror the traffic cannot install a general operating system to deploy the application program, a server is added to the back end of the device to deploy the first traffic mirroring program.

3. A method for remotely mirroring traffic according to claim 1, characterized in that: A server is added at the front end of the traffic receiving device to deploy the second traffic mirroring program.

4. A method for remotely mirroring traffic according to claim 1, characterized in that: For servers that deploy the primary / secondary traffic mirroring program, it is recommended to configure dual network cards, one for input / output traffic and the other for the traffic reversal tunnel.

5. A method for remotely mirroring traffic according to claim 1, characterized in that: When deploying a server with the primary / secondary traffic mirroring program and configuring a network card, you need to add the logic of filtering tunnel traffic to the primary / secondary traffic mirroring program and consider whether the bandwidth limit can be met.

6. A method for remotely mirroring traffic according to claim 1, characterized in that: The first tunnel program re-encapsulates the original message using a communicative layer 3 header, fragments the original message according to a set rule, encapsulates the fragment size and sequence number in the fifth layer, and then uses the UDP protocol to encapsulate the fragments one by one; The second tunnel program uses the same rules to decapsulate and reassemble the fragments before sending them to the second tunnel virtual network card.

7. A method for remotely mirroring traffic according to claim 6, characterized in that: The first tunnel program encrypts data above layer 4 using the RSA algorithm, and the public and private keys are pre-embedded in the first tunnel program and the second tunnel program.

8. A system for remotely mirroring traffic, characterized in that: include: A first traffic mirroring program module is deployed at the traffic import end and is used to transfer the traffic imported into the network card to the first tunnel virtual network card; The second traffic mirroring program module is deployed at the traffic output end and is used to capture the traffic of the second tunnel virtual network card and transfer it to the output network card; The first tunnel program module is deployed at the traffic import end, and is used to split the traffic received by the first tunnel virtual network card and re-encapsulate it into new UDP protocol messages one by one, and send it to the traffic output end through the first tunnel physical network card; And, the second tunnel program module is deployed at the traffic output end, used to listen to the UDP port of the second tunnel physical network card, decapsulate the intercepted message, restore it to the source message, and then send it to the second tunnel virtual network card, so as to be captured by the second traffic mirroring program module.

9. A system for remotely mirroring traffic according to claim 8, characterized in that: The first tunnel program re-encapsulates the original message using a communicative layer 3 header, fragments the original message according to a set rule, encapsulates the fragment size and sequence number in the fifth layer, and then uses the UDP protocol to encapsulate the fragments one by one; The second tunnel program uses the same rules to decapsulate and reassemble the fragments before sending them to the second tunnel virtual network card.

10. A system for remotely mirroring traffic according to claim 8, characterized in that: The first tunnel program module encrypts the slice data using the RSA algorithm, and the public and private keys are pre-embedded in the first tunnel program module and the second tunnel program module.

Images