Traffic mirroring method and device, electronic equipment and storage medium

Abstract

The application discloses a traffic mirroring method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining the to-be-mirrored packet data stored in the packet matching storage memory of a traffic mirroring processing tool; determining the number of mirroring demand systems matched by the to-be-mirrored packet data; and performing traffic mirroring processing on the to-be-mirrored packet data in the packet matching storage memory according to the number of mirroring demand systems matched by the to-be-mirrored packet data. The technical scheme of the embodiment of the application can save the memory resources occupied by traffic mirroring, improve the efficiency of traffic mirroring, and further improve the overall service performance of traffic mirroring processing.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a traffic mirroring method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the widespread adoption of computer technology and the development of remote information processing applications, data communication technology has emerged, enabling data transfer between computers and between computers and terminals. Traffic mirroring technology is a fundamental and crucial technique in data processing. Also known as shadow traffic, traffic mirroring filters network traffic based on multiple different dimensions before sending it to the user.

[0003] Traditional network traffic mirroring methods involve copying the filtered traffic that needs to be mirrored one or more times and sending them to a third-party system, depending on the business requirements. Copying traffic usually requires creating a memory pool in software or requesting additional memory from the network interface. Then, the original packet content to be sent is copied into the newly allocated memory, and finally, the packet format is customized according to the needs of different systems before being sent to each third-party system.

[0004] In the process of realizing this invention, the inventors discovered the following defects in the prior art: the above-mentioned traffic mirroring method requires additional memory allocation when copying traffic, and performs packet copying on the additional memory allocation, which not only consumes a lot of memory resources, but also reduces the efficiency of traffic mirroring, thereby affecting the overall business performance of the system. Summary of the Invention

[0005] This invention provides a traffic mirroring method, apparatus, electronic device, and storage medium, which can save memory resources occupied by traffic mirroring and improve the efficiency of traffic mirroring, thereby improving the overall business performance of traffic mirroring processing.

[0006] In a first aspect, embodiments of the present invention provide a traffic mirroring method, including:

[0007] Obtain the packet matching data of the traffic mirroring tool stored in memory to be mirrored;

[0008] Determine the number of mirroring requirement systems that match the message data to be mirrored;

[0009] Based on the number of mirroring requirement systems that match the message data to be mirrored, traffic mirroring processing is performed on the message data to be mirrored in the message matching storage memory.

[0010] The embodiment of the application acquires the to-be-mirrored processing packet data stored in the packet matching storage memory of the traffic mirroring processing tool, determines the number of mirroring demand systems matched by the to-be-mirrored processing packet data, and then selects the traffic mirroring strategy corresponding to the number of mirroring demand systems matched by the to-be-mirrored processing packet data, so as to perform traffic mirroring processing on the to-be-mirrored processing packet data in the packet matching storage memory, thereby solving the problem that the existing traffic mirroring method needs to additionally apply for memory for packet copying when mirroring traffic, saving the memory resources occupied by traffic mirroring, improving the efficiency of traffic mirroring, and further improving the overall business performance of traffic mirroring processing.

[0011] Optionally, before the to-be-mirrored processing packet data stored in the packet matching storage memory of the traffic mirroring processing tool is acquired, the method further comprises:

[0012] Acquiring an original network packet, and extracting a target packet feature field from the original network packet;

[0013] Matching the target packet feature field of the original network packet according to a packet matching strategy, to filter the to-be-mirrored processing packet data from the original network packet.

[0014] The above technical solution determines which packet data in the original network packet needs to be mirrored by matching the received original network packet according to user rules before mirroring processing, thereby improving the efficiency and accuracy of traffic mirroring processing.

[0015] Optionally, the traffic mirroring processing of the to-be-mirrored processing packet data in the packet matching storage memory according to the number of mirroring demand systems matched by the to-be-mirrored processing packet data comprises:

[0016] In a case where the number of mirroring demand systems matched by the to-be-mirrored processing packet data is one, updating the reference count identifier matched by the packet matching storage memory to a default value;

[0017] Sending the to-be-mirrored processing packet data stored in the packet matching storage memory to a packet sending queue for traffic mirroring processing.

[0018] The above technical solution, when the number of mirroring demand systems matched by the to-be-mirrored processing packet data is one, can reasonably use and release the memory space of the packet matching storage memory by introducing the application of the reference count identifier, thereby improving the usage rate and utilization rate of the packet matching storage memory.

[0019] Optionally, the traffic mirroring processing of the to-be-mirrored processing packet data in the packet matching storage memory according to the number of mirroring demand systems matched by the to-be-mirrored processing packet data comprises:

[0020] In a case where the number of the mirroring demand systems matched by the to-be-mirrored packet data is multiple, the reference count identifier of the packet matching storage memory is updated according to the number of the mirroring demand systems;

[0021] The mirroring association strategy matched by the to-be-mirrored packet data is acquired, wherein the mirroring association strategy comprises a system hit strategy and a packet mirroring processing strategy;

[0022] The mirroring association strategy is added to the packet matching storage memory;

[0023] The to-be-mirrored packet data is sequentially subjected to traffic mirroring processing according to the mirroring association strategy stored in the packet matching storage memory.

[0024] The above technical solution, when the number of the mirroring demand systems matched by the to-be-mirrored packet data is multiple, adds multiple mirroring association strategies, reduces the mirroring times of the to-be-mirrored packet data, and thus reduces the packet matching storage memory overhead.

[0025] Optionally, the adding of the mirroring association strategy to the packet matching storage memory comprises:

[0026] The mirroring association strategy index information is generated according to the mirroring association strategy;

[0027] The mirroring association strategy is stored in a set storage area, and the mirroring association strategy index information is added to the packet matching storage memory.

[0028] The above technical solution, by introducing the application of the mirroring association strategy index information, when the mirroring association strategy does not exceed the set storage capacity of the packet matching storage memory, the mirroring association strategy index can be directly stored in the packet matching storage memory; when the mirroring association strategy exceeds the set storage capacity of the packet matching storage memory, the mirroring association strategy index information can be stored in the form of index, which not only can store more mirroring association strategy information in the packet matching storage memory, but also can save the memory space of the packet matching storage memory, and improves the flexibility of storing the mirroring association strategy.

[0029] Optionally, the adding of the mirroring association strategy index information to the packet matching storage memory comprises:

[0030] A target memory structure in the packet matching storage memory for storing the mirroring association strategy index information is determined;

[0031] The mirroring association strategy index information is added to the target memory structure in the packet matching storage memory;

[0032] The target memory structure includes a memory head structure or a memory tail structure.

[0033] The above technical solution introduces the application of the target memory structure, and the storage area of the mirror association strategy index information can be selected according to actual conditions, thereby improving the flexibility of mirror association strategy index information storage.

[0034] Optionally, the adding of the mirror association strategy index information into the target memory structure of the packet matching storage memory includes:

[0035] Determining a dynamic priority of each of the mirror association strategy index information matched with the traffic mirror;

[0036] According to the dynamic priority of each of the mirror association strategy index information matched with the traffic mirror, each of the mirror association strategy index information is sorted;

[0037] The sorted mirror association strategy index information is added into the target memory structure of the packet matching storage memory.

[0038] The above technical solution sets different priorities for mirror demand systems with different processing capabilities, thereby improving the working efficiency of traffic mirror processing and realizing the personalized processing mode of traffic mirror.

[0039] In a second aspect, the embodiment of the present application further provides a traffic mirror device, which includes:

[0040] A to-be-mirrored packet data acquisition module is configured to acquire to-be-mirrored packet data stored in a packet matching storage memory of a traffic mirror processing tool;

[0041] A mirror demand system quantity determination module is configured to determine the number of mirror demand systems matched with the to-be-mirrored packet data;

[0042] A traffic mirror processing module is configured to perform traffic mirror processing on the to-be-mirrored packet data in the packet matching storage memory according to the number of mirror demand systems matched with the to-be-mirrored packet data.

[0043] In a third aspect, the embodiment of the present application further provides an electronic device, which includes:

[0044] At least one processor; and

[0045] A memory in communication connection with the at least one processor; wherein

[0046] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the traffic mirroring method according to any one of the embodiments of the present application.

[0047] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores computer instructions for enabling a processor to perform the traffic mirroring method according to any one of the embodiments of the present application when executed by the processor.

[0048] It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the present application or to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0050] Figure 1 is a flowchart of a traffic mirroring method provided by the embodiments of the present application;

[0051] Figure 2 is a flowchart of a traffic mirroring method provided by the embodiments of the present application;

[0052] Figure 3 is a flowchart of a traffic mirroring method provided by the embodiments of the present application;

[0053] Figure 4 is a schematic diagram of an internal structure of a traffic mirroring processing tool provided by the embodiments of the present application;

[0054] Figure 5 is a flowchart of a traffic mirroring method provided by the embodiments of the present application;

[0055] Figure 6 is a schematic diagram of a traffic mirroring device provided by the embodiments of the present application;

[0056] Figure 7 is a schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0057] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application by persons skilled in the art. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should belong to the protection scope of the present application.

[0058] It should be noted that the terms "first", "second", "original" and "target" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be exchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] Figure 1 is a flowchart of a traffic mirroring method provided by an embodiment of the present application. The embodiment can be applicable to the case of traffic mirroring processing based on the memory resources of stored packets. The method can be executed by a traffic mirroring device, which can be realized in the form of hardware and / or software. The traffic mirroring device can be configured on an electronic device, which can be a desktop computer, a notebook computer and many other devices having traffic mirroring processing function, or a server, etc. The device type of the electronic device is not limited by the present application. As shown in Figure 1 The method comprises the following steps.

[0060] S110, obtaining the packet data to be mirrored and processed stored in the packet matching storage memory of the traffic mirroring processing tool.

[0061] The traffic mirror processing is processing of copying and sending network traffic, etc. The traffic mirror processing tool can be understood as a tool having a function of mirror processing network traffic, for example, it can be a Data Plane Development Kit (DPDK). The packet matching storage memory is a memory used by the traffic mirror processing tool to store received packets, for example, for the DPDK, the packet matching storage memory can be a packet memory mbuf structure (struct rte_mbuf) loaded by the DPDK from a network card interface. The to-be-mirror-processed packet data is packet data that needs to be processed by one or more mirrors.

[0062] In the embodiment of the present application, network traffic in the network can be collected by the traffic acquisition tool, and when the traffic acquisition tool identifies that any network traffic needs to be processed by a mirror, the network traffic is stored in the packet matching storage memory of the traffic mirror processing tool. Further, the to-be-mirror-processed packet data stored in the packet matching storage memory of the traffic mirror processing tool can be acquired to process the to-be-mirror-processed packet data by traffic mirror processing.

[0063] S120, determining the number of mirror requirement systems matched with the to-be-mirror-processed packet data.

[0064] The mirror requirement system can be understood as a third-party system that receives the to-be-mirror-processed packet data according to business needs.

[0065] Correspondingly, after the to-be-mirror-processed packet data stored in the packet matching storage memory of the traffic mirror processing tool is acquired, the number of third-party systems having mirror requirements for the to-be-mirror-processed packet data can be further determined.

[0066] S130, according to the number of mirror requirement systems matched with the to-be-mirror-processed packet data, processing the to-be-mirror-processed packet data by traffic mirror processing in the packet matching storage memory.

[0067] In the embodiment of the present application, the number of mirror requirement systems matched with the to-be-mirror-processed packet data is different, and the corresponding traffic mirror processing mode is also different. Therefore, according to the number of mirror requirement systems matched with the to-be-mirror-processed packet data, the number of times of mirror processing required by the to-be-mirror-processed packet data can be determined, so that according to the number of times of mirror processing, the to-be-mirror-processed packet data is processed by the corresponding number of times of mirror processing based on the packet matching storage memory, and the mirrored traffic is sent to a next node.

[0068] Exemplarily, when the number of mirror demand systems matched by the to-be-mirrored packet data is one, the to-be-mirrored packet data can be sent to the packet sending queue for flow mirroring processing, and the packet matching storage memory is emptied and the memory is released; when the number of mirror demand systems matched by the to-be-mirrored packet data is multiple, the multiple to-be-mirrored packet data can be processed in order according to the demand, and then sent to the packet sending queue for flow mirroring processing by the packet matching storage memory, and when the entire mirror processing operation is completed, the packet matching storage memory is emptied and the memory is released.

[0069] Therefore, the flow mirroring method provided by the embodiment of the application does not need to apply additional memory for the network packet received by the flow mirroring processing tool, thereby saving a large amount of memory resources. Meanwhile, the flow mirroring method does not need to copy the network packet stored in the packet matching storage memory of the network packet received from the storage to the additionally applied memory, and can complete the flow mirroring processing operation only by using the packet matching storage memory, thereby avoiding the copying process of the data packet, improving the efficiency of flow mirroring, and further improving the overall service performance of flow mirroring processing.

[0070] The embodiment of the application acquires the to-be-mirrored packet data stored in the packet matching storage memory of the flow mirroring processing tool, determines the number of mirror demand systems matched by the to-be-mirrored packet data, and then selects the flow mirroring strategy corresponding to the number of mirror demand systems matched by the to-be-mirrored packet data, thereby performing flow mirroring processing on the to-be-mirrored packet data in the packet matching storage memory, solving the problem that the existing flow mirroring method needs to apply additional memory for packet copying when mirroring flow, saving the memory resources occupied by flow mirroring, improving the efficiency of flow mirroring, and further improving the overall service performance of flow mirroring processing.

[0071] Figure 2 is a flow chart of a flow mirroring method provided by the embodiment of the application, and the embodiment is further optimized and extended based on the above-mentioned embodiment, introduces the application mode of the mirror association strategy, and simultaneously gives multiple specific optional implementation modes of flow mirroring processing based on the mirror association strategy. As shown in Figure 2 The method comprises the following steps of:

[0072] S210, acquiring an original network data packet, and extracting a target packet feature field from the original network data packet.

[0073] The original network data packet can be understood as a data packet of original network traffic data obtained by the traffic mirror processing tool directly from the network. The target message feature field can be understood as a feature field used to describe a message to be processed. Exemplarily, the feature field can include, but is not limited to, a five-tuple (including a source IP address, a source port, a destination IP address, a destination port and a transport layer protocol) in the message, an application layer protocol and a message load and the like related to a message feature.

[0074] In the embodiment of the present application, the traffic mirror processing tool can collect network traffic, obtain data packets of original network traffic data from the network, and extract relevant target message feature fields in the original network data packets according to requirements, to determine whether the original network data packets are data packets to be mirror processed.

[0075] S220, matching the target message feature fields of the original network data packets according to a message matching strategy, to filter and obtain the to-be-mirror-processed message data from the original network data packets.

[0076] The message matching strategy is a strategy for filtering and screening messages according to the target message feature fields, and can be used to determine whether any target message needs to be mirror processed, a number of mirror processing, and a mirror demand system receiving the mirror message after mirror processing, and the like.

[0077] Specifically, the traffic mirror processing tool can filter and screen the target message feature fields in the collected original network data packets according to the message matching strategy, to filter and screen the messages to be mirror processed and sent from the original network data packets as the to-be-mirror-processed message data, and store the to-be-mirror-processed message data in the message matching storage memory of the traffic mirror processing tool.

[0078] It should be noted that the message matching strategy can be a strategy for filtering and screening messages according to five-tuple, application layer protocol, message load features and the like, or a strategy for filtering and screening messages according to more complex regular rules and multiple different dimension combinations, and the like. The embodiment of the present application does not limit the specific content of the message matching strategy.

[0079] The above technical solution determines which message data in the original network data packets need to be mirror processed according to user rules before mirror processing, thereby improving the efficiency and accuracy of traffic mirror processing.

[0080] S230, obtaining the to-be-mirror-processed message data stored in the message matching storage memory of the traffic mirror processing tool.

[0081] S240, judging whether the number of mirror image demand systems matched by the to-be-mirrored packet data is one, if yes, executing S250; otherwise, executing S270.

[0082] S250, updating the reference count identifier matched by the packet matching storage memory to a default value.

[0083] The reference count identifier can be understood as an identifier recording the reference times of the data packet stored in the packet matching storage memory, and can be used to support the use and release operation of any piece of packet matching storage memory. The default value refers to a value set in advance, and can be used to represent the default number of references of the data stored in the packet matching storage memory. For example, the default value can be set to 1, indicating that the number of mirror image operations of the data packet stored in the packet matching storage memory is 1, and the specific value of the default value is not limited in the present application.

[0084] For example, when the reference count identifier is not 0, it indicates that the packet matching storage memory is still in use and cannot release the data packet stored therein; when the reference count identifier is 0, it indicates that the mirror image processing of the data packet stored in the packet matching storage memory is completed, and the data in the memory area can be emptied and the memory can be released.

[0085] Specifically, when the number of mirror image demand systems matched by the to-be-mirrored packet data is one, it indicates that the to-be-mirrored packet data needs to be mirrored once and sent to the matched mirror image demand system, and the reference count identifier matched by the packet matching storage memory can be updated to the default value, such as 1.

[0086] S260, sending the to-be-mirrored packet data stored in the packet matching storage memory to the packet sending queue for flow mirroring processing.

[0087] The packet sending queue refers to a queue in which the to-be-mirrored packet data is arranged in order.

[0088] Correspondingly, the to-be-mirrored packet data stored in the packet matching storage memory can be obtained according to the reference count identifier, and the to-be-mirrored packet data can be sent to the packet sending queue for flow mirroring processing and sending. For example, the to-be-mirrored packet data can be processed according to the sending requirement of the corresponding mirror image demand system, and then sent to the to-be-mirrored packet data. Optionally, the sending requirement can include but is not limited to filtering fields of the to-be-mirrored packet data or other processing rules configured according to business requirements.

[0089] The above technical scheme, when the number of mirror image demand systems matched by the to-be-mirrored packet data is one, the application of introducing the reference count identifier can reasonably use and release the memory space of the packet matching storage memory, and improves the use rate and utilization rate of the packet matching storage memory.

[0090] Figure 3 A flowchart of a traffic mirroring method provided by an embodiment of the present application is shown in FIG. 1. In a specific example, as shown in FIG. 1, taking DPDK as a traffic mirroring processing tool, when the number of mirror image demand systems matched by the to-be-mirrored packet data is one, the corresponding traffic mirroring method can include the following steps: Figure 3

[0091] Step one, judging whether the current packet needs to be mirrored.

[0092] Step two, after determining that the current packet needs to be mirrored once, increasing the reference times of mbuf to avoid that the memory is released after being sent once.

[0093] Step three, pushing the mbuf where the current packet is located to a packet sending queue for mirror image processing.

[0094] S270, updating the reference count identifier matched by the packet matching storage memory according to the number of mirror image demand systems.

[0095] In the embodiment, when the number of mirror image demand systems matched by the to-be-mirrored packet data is multiple, it indicates that the to-be-mirrored packet data needs to be mirrored multiple times, and the multiple mirror image processing packet data are sent to the matched mirror image demand systems respectively. At this time, the reference count identifier matched by the packet matching storage memory can be updated according to the number of mirror image demand systems, and the updated value can be the number of mirror image demand systems.

[0096] S280, obtaining a mirror image association strategy matched by the to-be-mirrored packet data; wherein the mirror image association strategy includes a system hit strategy and a packet mirror image processing strategy.

[0097] ​In the embodiment of the present application, the mirror association policy can be a policy of mirroring and sending the to-be-mirrored packet data, which can include, for example, a policy of determining the receiver of the mirrored packet data, a policy of determining the rule of reference count identification operation, and a policy of specific mirroring operation. The system hit policy refers to a policy of determining the mirror demand system matched with the to-be-mirrored packet data. For example, when the network ID (ID) of the mirror demand system and the field content required for filtering the to-be-mirrored packet data are different, the corresponding hit policy needs to be formulated according to the information corresponding to each mirror demand system. The packet mirroring processing policy can be a policy of mirroring the to-be-mirrored packet data, which can be, for example, a policy of determining the required field content of the to-be-mirrored packet data according to the business requirement of the mirror demand system.

[0098] Specifically, when the number of mirror demand systems matched with the to-be-mirrored packet data is multiple, the system hit policy and the packet mirroring processing policy matched with the to-be-mirrored packet data can be acquired, so as to determine the receiver of the mirrored packet data, the rule of reference count identification operation, and the policy content of specific mirroring operation according to the system hit policy and the packet mirroring processing policy, thereby realizing different mirroring processing modes of different mirror demand systems for the to-be-mirrored packet data.

[0099] For example, when the number of mirror demand systems is three, and the IDs of the three mirror demand systems are ID01, ID02, and ID03, the corresponding three hit policies are: the hit policy 1 corresponds to ID01, and the corresponding packet mirroring processing policy is to filter the field content required for ID01 from the to-be-mirrored packet data, and to add the filtered field required for ID01 to the tail of the to-be-mirrored packet; the hit policy 2 corresponds to ID02, and the corresponding packet mirroring processing policy is to filter the field content required for ID02 from the to-be-mirrored packet data, and to add the filtered field required for ID02 to the tail of the to-be-mirrored packet; and the hit policy 3 corresponds to ID03, and the corresponding packet mirroring processing policy is to filter the field content required for ID03 from the to-be-mirrored packet data, and to add the filtered field required for ID03 to the tail of the to-be-mirrored packet. Optionally, the field content required for ID01, ID02, and ID03 can be the same or different, and the embodiment of the present application does not limit this.

[0100] For example, when the number of mirror demand systems is three, the reference count identifier is updated to 3 according to the number of mirror demand systems, and the reference count identifier is decremented by 1 each time the to-be-mirrored packet data is mirrored once and sent to the corresponding mirror demand system. When the to-be-mirrored packet data is sent three times, the reference count identifier is decremented to 0, indicating that the to-be-mirrored packet data has completed all mirroring and sending processing, and the data in the packet matching memory area storing the to-be-mirrored packet data can be emptied and the memory is released.

[0101] S290, add the mirror association policy to the packet matching memory.

[0102] In the embodiment of the present application, by adding the mirror association policy to the packet matching memory, when the traffic mirroring is performed, the mirror association policy matched by the to-be-mirrored packet data can be directly obtained based on the packet matching memory, and the to-be-mirrored packet data is sequentially processed based on the obtained mirror association policy matched by the to-be-mirrored packet data in the packet matching memory. That is, the above traffic mirroring method does not need to apply additional memory according to the number of mirror demand systems, and does not need to copy the to-be-mirrored packet data to the additional memory according to the number of mirror demand systems, so as to mirror the to-be-mirrored packet data based on the additional memory.

[0103] Optionally, adding the mirror association policy to the packet matching memory can include: generating mirror association policy index information according to the mirror association policy; storing the mirror association policy to a set storage area, and adding the mirror association policy index information to the packet matching memory.

[0104] The mirror association policy index information can be index information for searching the mirror association policy, for example, can be an address for storing the mirror association policy or an ID identifier of the mirror association policy. The set storage area can be a storage area in the device for storing the mirror association policy.

[0105] It can be understood that if the mirror association policy data volume is not large, it will not cause the packet matching memory to overflow, and the mirror association policy can be directly stored in the packet matching memory. However, if the number of mirror demand systems is large or the mirror association policy data volume is large, it will often cause the packet matching memory to overflow. Therefore, in order to avoid the packet matching memory from overflowing, the mirror association policy index information for searching the mirror association policy can be generated according to the mirror association policy, and the mirror association policy can be stored in the set storage area of the non-packet matching memory, and only the mirror association policy index information can be stored in the packet matching memory.

[0106] The technical scheme of the embodiment of the application introduces the application of mirror association policy index information, when the mirror association policy does not exceed the set storage capacity of the packet matching storage memory, the mirror association policy index can be directly stored in the packet matching storage memory; when the mirror association policy exceeds the set storage capacity of the packet matching storage memory, the mirror association policy index information can be stored in the form of index, which not only can store more mirror association policy information in the packet matching storage memory, but also can save the memory space of the packet matching storage memory, and improves the flexibility of storing the mirror association policy.

[0107] Optionally, adding the mirror association policy index information to the packet matching storage memory can include: determining a target memory structure in the packet matching storage memory for storing the mirror association policy index information; and adding the mirror association policy index information to the target memory structure of the packet matching storage memory; wherein the target memory structure can include a memory head structure or a memory tail structure.

[0108] The target memory structure can be a memory structure for storing the mirror association policy index information in the packet matching storage memory, for example, can be a memory structure other than the internal data structure and the packet content in the packet matching storage memory, which can be controlled and used by a user. The memory head structure is a head structure in the memory structure of the packet matching storage memory. The memory tail structure is a tail structure in the memory structure of the packet matching storage memory.

[0109] For example, taking DPDK as a traffic mirror processing tool, the packet matching storage memory can be generally divided into four storage areas: internal data structure, packet content, and two pieces of memory which can be controlled and used by a user. The two pieces of memory which can be controlled and used by a user are distributed in the head structure and the tail structure of the memory area, wherein the size of the head structure is determined by the storage framework or the protocol, and the size space is determined; the size of the tail structure is negatively correlated with the length of the packet, that is, the shorter the length of the packet, the larger the tail structure, and the longer the length of the packet, the smaller the tail structure.

[0110] Figure 4 The embodiment of the application provides an internal structure diagram of a traffic mirror processing tool. In a specific example, as shown in FIG. 1, the packet matching storage memory is divided into four parts: internal data structure, packet content, and two pieces of memory which can be controlled and used by a user. Figure 4As shown, the mbuf memory structure in the DPDK can include four parts: a first part mbuf structure, used to store the internal data structure of the mbuf; a second part headroom; a third part data room, used to store the message content, i.e., the network data frame content, the third part is also called rte_pktmbuf_mtod(mbuf), and the length is rte_pktmbuf_pktlen(mbuf); and a fourth part tailroom. The use of the second part and the fourth part is controlled by the user. Optionally, if the network data frame needs to be mirrored multiple times, the corresponding mirror association strategy index information can be stored in the headroom part to avoid memory overflow problems caused by limited storage capacity of the tailroom.

[0111] The technical scheme of the embodiment of the application can introduce the application of the target memory structure, can select the storage area of the mirror association strategy index information according to the actual situation, and improves the flexibility of the mirror association strategy index information storage.

[0112] Optionally, adding the mirror association strategy index information to the target memory structure of the message matching storage memory can include: determining the traffic mirror dynamic priority matched by each mirror association strategy index information; sorting each mirror association strategy index information according to the traffic mirror dynamic priority matched by each mirror association strategy index information; and adding the sorted mirror association strategy index information to the target memory structure of the message matching storage memory.

[0113] The traffic mirror dynamic priority can be a parameter for describing the priority level of each mirror demand system when the traffic mirror processing tool performs multiple traffic mirror processing tasks.

[0114] In the embodiment of the application, in the process of simultaneously performing multiple traffic mirror processing tasks, the traffic mirror dynamic priority matched by each mirror association strategy index information can also be determined according to reference information such as the weight of the mirror demand system, the mirror traffic characteristics, or the total amount of mirror traffic data, and the traffic mirror dynamic priority can determine the order of mirror processing of the mirror demand system. Further, each mirror association strategy index information is sorted according to the traffic mirror dynamic priority matched by each mirror association strategy index information, and the sorted mirror association strategy index information is added to the target memory structure of the message matching storage memory. Accordingly, when performing traffic mirroring, each mirror association strategy index information can be obtained in order according to the order of the mirror association strategy index information, so as to obtain the matched mirror association strategy according to each mirror association strategy index information, and then perform traffic mirroring on the to-be-mirrored message data according to the mirror association strategy.

[0115] That is, the mirror demand system corresponding to the mirror association strategy index information with high priority can be preferentially processed; the mirror demand system corresponding to the mirror association strategy index information with low priority needs to wait for the processing of the mirror association strategy index information with high priority to be completed before being processed.

[0116] The technical scheme of the embodiment of the application improves the work efficiency of traffic mirror processing by setting different priorities for mirror demand systems with different processing capabilities, and realizes a personalized processing mode of traffic mirror.

[0117] S2100, sequentially processing the traffic mirror of the to-be-mirrored processing packet data according to the mirror association strategy stored in the packet matching storage memory.

[0118] Specifically, the traffic mirror processing tool extracts a plurality of mirror association strategy information stored in the packet matching storage memory in a certain order, determines the network ID of the corresponding mirror demand system or the corresponding mirror processing strategy information such as the field to be filtered according to the system hit strategy and the packet mirror processing strategy in each mirror association strategy information, and sequentially mirrors and sends the packet data to the corresponding mirror demand system.

[0119] The above technical scheme, when the number of mirror demand systems matched by the to-be-mirrored processing packet data is multiple, adds a plurality of mirror association strategies, reduces the mirror times of the to-be-mirrored processing packet data, and thereby reduces the overhead of the packet matching storage memory.

[0120] The embodiment of the application serializes the to-be-mirrored processing packet data required by different mirror demand systems to the non-packet structure area of the packet matching storage memory, realizes that the memory for storing data packets can directly meet the customized mirror business of multiple systems, and improves the work efficiency of traffic mirror processing.

[0121] In order to more clearly describe the technical scheme provided by the embodiment of the application, in one specific example, the DPDK is taken as an example of a traffic mirror processing tool, and the traffic mirror method can first perform multi-dimensional strategy matching on the network frame received by the DPDK and stored in the mbuf, filter out the packets that need to be mirrored, and then directly use the current mbuf for mirroring. When a packet needs to be mirrored multiple times, the hit multiple strategies are stored in the headroom part of the mbuf, and the packet is sent after the business processing is completed. Figure 5 A flowchart of a traffic mirror method provided by the embodiment of the application is shown in FIG. 1. Figure 5 As shown in FIG. 1, the traffic mirror method can include the following processes:

[0122] Step one, multiple dimensions of network packet on DPDK are matched: filter traffic, extract data frame source and destination IP address information, source and destination port information and message load and other message characteristic fields; the matched message content is matched to obtain the current matching successful message, that is, the message needs to be mirrored.

[0123] Step two, judge whether the current matching successful message needs to be mirrored multiple times.

[0124] Step three, if the current matching successful message needs to be mirrored only once, the current mbuf is directly increased in reference count and pushed to the packet sending queue for traffic mirroring.

[0125] Step four, if the current matching successful message needs to be mirrored multiple times, the matched policy content is stored in the header part of mbuf, and after the current message is processed, it is pushed to the packet sending queue according to the policy information for traffic mirroring.

[0126] The above technical solution develops the traffic mirroring capability of the multi-service system based on DPDK, solves the performance bottleneck and excessive occupation of memory resources of mirroring multiple traffic, not only saves the memory resources occupied by traffic mirroring, but also avoids the operation of copying multiple messages, thereby improving the efficiency of traffic mirroring.

[0127] It should be noted that any arrangement and combination of the technical features in the above embodiments also belongs to the protection scope of the present application.

[0128] Figure 6 is a structural schematic diagram of a traffic mirroring device provided by an embodiment of the present application. The present embodiment can be applied to the scene of traffic mirroring processing in a multi-system mirroring scene according to user demand, and the present embodiment does not make specific limitation to this. As shown in the figure, Figure 6 The traffic mirroring device includes a to-be-mirrored message data acquisition module 310, a mirror demand system quantity determination module 320, and a traffic mirroring processing module 330.

[0129] The to-be-mirrored message data acquisition module 310 is used to acquire the to-be-mirrored message data stored in the message matching storage memory of the traffic mirroring processing tool; the mirror demand system quantity determination module 320 is used to determine the quantity of mirror demand systems matched by the to-be-mirrored message data; and the traffic mirroring processing module 330 is used to perform traffic mirroring processing on the to-be-mirrored message data in the message matching storage memory according to the quantity of mirror demand systems matched by the to-be-mirrored message data.

[0130] The embodiment of the present application acquires the to-be-mirrored packet data stored in the packet matching storage memory of the traffic mirroring processing tool, determines the number of mirroring demand systems matched by the to-be-mirrored packet data, and then selects the traffic mirroring strategy corresponding to the number of mirroring demand systems matched by the to-be-mirrored packet data, so as to perform traffic mirroring processing on the to-be-mirrored packet data in the packet matching storage memory, thereby solving the problem that the existing traffic mirroring method needs to additionally apply memory for packet copying when mirroring traffic, saving the memory resources occupied by traffic mirroring, improving the efficiency of traffic mirroring, and further improving the overall business performance of traffic mirroring processing.

[0131] Optionally, the traffic mirroring device further comprises: a target packet feature field extraction module, configured to acquire an original network packet and extract a target packet feature field of the original network packet; and a to-be-mirrored packet data screening module, configured to match the target packet feature field of the original network packet according to a packet matching strategy to screen to-be-mirrored packet data from the original network packet.

[0132] Optionally, the traffic mirroring processing module 330 specifically comprises: a first reference count identifier unit, configured to update the reference count identifier matched by the packet matching storage memory to a default value in a case where the number of mirroring demand systems matched by the to-be-mirrored packet data is one; and a first traffic mirroring processing unit, configured to send the to-be-mirrored packet data stored in the packet matching storage memory to a packet sending queue for traffic mirroring processing.

[0133] Optionally, the traffic mirroring processing module 330 specifically comprises: a second reference count identifier unit, configured to update the reference count identifier matched by the packet matching storage memory according to the number of mirroring demand systems in a case where the number of mirroring demand systems matched by the to-be-mirrored packet data is multiple; a mirroring association strategy acquisition unit, configured to acquire a mirroring association strategy matched by the to-be-mirrored packet data; wherein the mirroring association strategy comprises a system hit strategy and a packet mirroring processing strategy; a mirroring association strategy adding unit, configured to add the mirroring association strategy to the packet matching storage memory; and a second traffic mirroring processing unit, configured to sequentially perform traffic mirroring processing on the to-be-mirrored packet data according to the mirroring association strategy stored in the packet matching storage memory.

[0134] Optionally, the mirroring association strategy adding unit is specifically configured to: generate mirroring association strategy index information according to the mirroring association strategy; store the mirroring association strategy in a set storage area, and add the mirroring association strategy index information to the packet matching storage memory.

[0135] Optionally, the mirror association strategy adding unit is specifically configured to: determine a target memory structure in the packet matching storage memory for storing the mirror association strategy index information; and add the mirror association strategy index information into the target memory structure in the packet matching storage memory; wherein the target memory structure comprises a memory header structure or a memory tail structure.

[0136] Optionally, the mirror association strategy adding unit is specifically configured to: determine a target memory structure in the packet matching storage memory for storing the mirror association strategy index information; and add the mirror association strategy index information into the target memory structure in the packet matching storage memory; wherein the target memory structure comprises a memory header structure or a memory tail structure.

[0137] The traffic mirroring device provided by the embodiment can execute the traffic mirroring method provided by any of the embodiments, and has the corresponding function modules and beneficial effects.

[0138] Figure 7 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0139] As shown in Figure 7 The electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which are communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0140] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0141] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the traffic mirroring method described in embodiments of the present application.

[0142] That is, the traffic mirroring processing tool acquires the to-be-mirrored packet data stored in the packet matching storage memory; determines the number of mirroring demand systems matched by the to-be-mirrored packet data; and performs traffic mirroring processing on the to-be-mirrored packet data in the packet matching storage memory according to the number of mirroring demand systems matched by the to-be-mirrored packet data.

[0143] In some embodiments, the traffic mirroring method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the traffic mirroring method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the traffic mirroring method as described in embodiments of the present application by any other appropriate means, such as by means of firmware.

[0144] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0145] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package and partially on a remote machine or entirely on a remote machine or server.

[0146] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0147] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0148] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0149] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0150] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in a different order, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0151] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalents, and / or alternatives come within the scope of the present disclosure as recited by the claims.

Claims

1. A traffic mirroring method, characterized in that, include: Obtain the packet matching data of the traffic mirroring tool stored in memory to be mirrored; Determine the number of mirroring requirement systems that match the message data to be mirrored; When the number of mirroring requirement systems that match the message data to be mirrored is one, the message data to be mirrored is sent to the packet sending queue for traffic mirroring processing, and the message matching storage memory is cleared and the memory is released. When there are multiple mirroring request systems whose message data to be mirrored matches, the multiple message data to be mirrored are processed in sequence according to the request, and then sent from the message matching storage memory to the packet sending queue for traffic mirroring. After all mirroring operations are completed, the message matching storage memory is cleared and the memory is released.

2. The method according to claim 1, characterized in that, Before obtaining the packet data to be mirrored stored in the packet matching storage memory of the traffic mirroring processing tool, the method further includes: Obtain raw network data packets and extract target packet feature fields from the raw network data packets; The target message feature fields of the original network data packets are matched according to the message matching strategy to filter out the message data to be mirrored from the original network data packets.

3. The method according to claim 1, characterized in that, Adding the mirror association strategy to the packet matching storage memory includes: Generate mirror association strategy index information based on the described mirror association strategy; The mirror association strategy is stored in a designated storage area, and the mirror association strategy index information is added to the message matching storage memory.

4. The method according to claim 3, characterized in that, Adding the mirror association policy index information to the packet matching storage memory includes: Determine the target memory structure in the message matching storage memory used to store the mirror association policy index information; Add the mirror association policy index information to the target memory structure of the message matching storage memory; The target memory structure includes a memory header structure or a memory tail structure.

5. The method according to claim 4, characterized in that, Adding the mirror association policy index information to the target memory structure of the packet matching storage memory includes: Determine the dynamic priority of traffic mirroring that matches the index information of each of the aforementioned mirroring association strategies; Based on the dynamic priority of traffic mirroring matched by the mirror association policy index information, sort the mirror association policy index information. The sorted image association strategy index information is added to the target memory structure of the message matching storage memory.

6. A flow mirroring device, characterized in that, include: The module for acquiring message data to be mirrored is used to acquire message data to be mirrored stored in the memory of the traffic mirroring tool. The module for determining the number of mirroring requirement systems is used to determine the number of mirroring requirement systems that match the message data to be mirrored. The traffic mirroring module is used to perform traffic mirroring on the packet data to be mirrored in the packet matching storage memory according to the number of mirroring requirement systems matched with the packet data to be mirrored. The traffic mirroring processing module specifically includes: The first reference counting identifier unit is used to send the message data to be mirrored to the packet sending queue for traffic mirroring when it is determined that the number of mirroring requirement systems matched by the message data to be mirrored is one, and at the same time clear the message matching storage memory and release the memory. The second reference counting identifier unit is used to process multiple mirroring request systems according to the requirements when it is determined that the number of mirroring request systems matching the message data to be mirrored is multiple. Then, the message data to be mirrored is sent from the message matching storage memory to the packet sending queue for traffic mirroring processing. After all mirroring processing operations are completed, the message matching storage memory is cleared and the memory is released.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the traffic mirroring method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the traffic mirroring method according to any one of claims 1-5.

Images