A distributed DPI data processing method and system

Abstract

The application discloses a distributed DPI data processing method and system, and is applied to a distributed DPI data processing system, wherein the distributed DPI data processing system comprises a traffic shunt, at least one signaling analysis server, at least one signaling synthesis server, and multiple user plane data processing servers. The application can make each user plane data processing server only receive and process valid signaling plane-user plane association data, without receiving and processing signaling plane-user plane association data irrelevant to the server, thereby improving processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of mobile communication network data processing technology, and in particular to a distributed DPI data processing method and system. Background Technology

[0002] In mobile communication DPI systems, it is usually necessary to process signaling plane traffic and user plane traffic separately, and to perform user association and tag addition on user plane traffic based on user identification information, location information and user plane tunnel information obtained from parsing signaling plane traffic.

[0003] The existing DPI data processing flow is as follows: 1) Externally input traffic enters the switch or traffic splitter (hereinafter referred to as traffic splitter). 2) The traffic splitter outputs signaling plane traffic to signaling parsers 1-n and user plane traffic to user plane data processing 1-n based on the signaling plane traffic characteristics and user plane traffic characteristics. Here, 1-n is multiple signaling parsers or multiple user plane data processors. 3) Signaling parsing 1-n: Parse mobile communication signaling (e.g., 4G / 5G), and output internal messages to signaling synthesis 1-n; 4) Signaling synthesis 1-n generates signaling plane-user plane associated data (hereinafter also called signaling plane-user plane associated message). The associated data index is: destination IP address + TEID (Tunnel Endpoint Identifier). The associated data content includes: 3 codes (IMSI / SUPI, MSISDN / GPSI, IMEI / PEI), location information, etc. 5) Signaling synthesis 1-n: Output signaling plane-user plane association data to the user plane data processing server. The signaling synthesis server or traffic splitter needs to copy the signaling plane-user plane association data multiple times to the user plane data processing server. For example, if there are 10 user plane data processing servers, the signaling plane-user plane association data is copied 10 times and output to each user plane data processing server respectively; that is to say, each user plane data processing server needs to store the full amount of signaling plane-user plane association data. 6) User plane data processing 1-n: Receive internal messages from the signaling synthesis server and create a signaling plane-user plane association table based on the signaling plane-user plane association data; 7) User plane data processing 1-n: Receive user plane data (GTP-U message, GPRS Tunneling Protocol User Plane), query the signaling plane-user plane association table, and add tags such as 3-code and location information to the user plane data (e.g., add a TLV tag to the end of the message). 8) User plane data processing 1-n: The processed tagged packets are output to the backend system through the traffic splitter.

[0004] Based on the above method, each user plane data processing server (1-n) load balances user plane data (GTP-U packets) according to certain rules (such as source and destination IP addresses). However, no specific design processing is performed for signaling plane-user plane associated data. The associated data cannot be load balanced and distributed by the traffic splitter according to the same rules. The user plane data processing server needs to receive and process all signaling plane-user plane associated data and create all signaling plane-user plane association tables. For example, if there are 10 user plane data processing servers, each processing an average of 1 / 10 of the user plane data, 9 / 10 of the fully stored signaling plane-user plane association tables will be invalid entries, with only 1 / 10 being valid entries.

[0005] Each user plane data processing server (1-n) stores and processes the entire signaling plane-user plane association table, which not only wastes a lot of memory, but also affects the efficiency of adding, deleting, modifying and querying association table entries as the number of users increases, thereby increasing DPI data processing latency.

[0006] Therefore, it is necessary to provide a distributed DPI data processing scheme that enables signaling plane-user plane associated data and corresponding user plane data to be stably distributed to the same user plane data processing server, and to adaptively process IP fragmented packets and a large number of small associated packets. Summary of the Invention

[0007] The purpose of this invention is to provide a distributed DPI data processing method and system, which enables each user plane data processing server to only receive and process valid signaling plane-user plane associated data, without needing to receive and process signaling plane-user plane associated data unrelated to its own server, thereby improving processing efficiency.

[0008] To achieve the above objectives, the following technical solution is adopted: A distributed DPI data processing method is applied to a distributed DPI data processing system, wherein the distributed DPI data processing system includes a traffic splitter, at least one signaling parsing server, at least one signaling synthesis server, and multiple user plane data processing servers, and includes the following steps: S1: The signaling parsing server parses the mobile communication signaling, obtains user identification information, and generates internal signaling based on the parsing results, which is then sent to the signaling synthesis server. S2: The signaling synthesis server associates and synthesizes internal signaling to generate signaling plane-user plane association data. The signaling plane-user plane association data includes a user plane association index and user tag information corresponding to the user plane association index. The user plane association index includes the destination IP address and tunnel endpoint identifier TEID. S3: The signaling synthesis server uses the destination IP address and tunnel endpoint identifier TEID as load sharing parameters, generates load sharing keywords using a preset unified HASH algorithm, encapsulates the load sharing keywords and signaling plane-user plane associated data into an internal data packet, and sends the internal data packet to the traffic splitter. S4: The traffic splitter extracts the load sharing keyword from the internal data packets, determines the target user plane data processing server from multiple user plane data processing servers based on the load sharing keyword and the preset server mapping rules, and forwards the signaling plane-user plane associated data to the target user plane data processing server. S5: The traffic splitter receives GTP-U user plane data corresponding to the signaling plane-user plane associated data, extracts the destination IP address and tunnel endpoint identifier (TEID) from the GTP-U user plane data, uses the extracted destination IP address and tunnel endpoint identifier (TEID) as load sharing parameters, generates user plane load sharing keywords using a unified HASH algorithm, and forwards the GTP-U user plane data to the corresponding target user plane data processing server according to the user plane load sharing keywords and preset server mapping rules. S6: The target user plane data processing server establishes, updates, or deletes the signaling plane-user plane association table based on the received signaling plane-user plane association data, and performs association processing on the received GTP-U user plane data based on the signaling plane-user plane association table. Specifically, the same load-sharing keyword is generated for signaling plane-user plane associated data and GTP-U user plane data that have the same destination IP address and tunnel endpoint identifier (TEID), so that the signaling plane-user plane associated data and its corresponding GTP-U user plane data are assigned to the same target user plane data processing server.

[0009] Furthermore, the internal data packet includes an Ethernet header, a private packet header, and a payload; the Ethernet header includes a private Ethernet type for identifying the internal data packet; the private packet header includes at least a payload type field, a payload length field, and a load-sharing keyword field, wherein the payload type field is used to identify internal signaling packets, signaling plane-user plane associated packets, or IP fragmentation forwarding packets, and the load-sharing keyword field is used to carry load-sharing keywords generated by a unified HASH algorithm.

[0010] Furthermore, the traffic splitter identifies non-IP fragmented GTP-U user plane data based on protocol type, port, and IP protocol header fragmentation field. After identifying the GTP-U user plane data, the traffic splitter prioritizes using the GTP-U protocol load balancing type that includes the destination IP address and tunnel endpoint identifier TEID, rather than using the load balancing type based on the IP 5-tuple, and generates user plane load balancing keywords through a unified HASH algorithm.

[0011] Furthermore, in S1, the user identification information includes at least one of the following: International Mobile Subscriber Identity (IMSI) or Permanent Subscriber Identity (SUPI), Mobile Subscriber Number (MSISDN) or General Public Subscriber Identity (GPSI), and International Mobile Equipment Identity (IMEI) or Permanent Equipment Identity (PEI). When the signaling parsing server extracts user identification information from mobile communication signaling, it uses at least one of these user identification information as HASH input to generate internal signaling load sharing keywords and sends internal signaling in a one-to-one manner. When the signaling parsing server does not extract user identification information, it sends internal signaling in a one-to-many manner so that the signaling synthesis server can perform multi-interface signaling association and synthesis.

[0012] Furthermore, when the signaling synthesis server sends the signaling plane-user plane associated data: Retrieve multiple signaling plane-user plane associated packets in batches from the internal queue; Extract the load sharing keywords from each signaling plane-user plane associated message; Multiple signaling plane-user plane related messages are categorized according to load sharing keywords; At least two signaling plane-user plane associated messages with the same load sharing keyword are concatenated, and the length of the concatenated message is controlled to be less than the preset maximum transmission unit (MTU) value. Batch transmission of signaling plane-user plane associated messages after classification and splicing.

[0013] Furthermore, it also includes an IP fragmentation data processing step, which includes: The traffic splitter identifies IP fragmented packets and uses the source IP address and destination IP address of the IP fragmented packets as initial load sharing parameters to distribute IP fragmented packets belonging to the same IP session to the same first user plane data processing server. The first user plane data processing server establishes IP fragmentation tracking information based on the first fragment packet, and associates the intermediate fragment packets and the last fragment packets with the first fragment packet to obtain the tunnel endpoint identifier (TEID) in the first fragment packet. The first user plane data processing server uses the destination IP address and tunnel endpoint identifier TEID as load sharing parameters, uses a unified HASH algorithm to generate fragmentation load sharing keywords, encapsulates the IP fragmentation packets and fragmentation load sharing keywords into IP fragmentation forwarding packets and sends them to the traffic splitter. The traffic splitter sends IP fragment forwarding packets to the target user plane data processing server corresponding to the fragment load sharing keyword, based on the fragment load sharing keyword.

[0014] Furthermore, the user tag information includes at least one of user identification information and location information. The location information includes at least one of Tracking Area Identifier (TAI), Evolved Universal Terrestrial Radio Access Network Cell Global Identifier (ECGI), and New Radio Interface Cell Global Identifier (NR-CGI). The target user plane data processing server queries the signaling plane-user plane association table, adds the queried user tag information to the GTP-U user plane data in Type-Length-Value (TLV) format, and outputs the user plane data with added user tag information to the backend system via a traffic splitter.

[0015] A distributed DPI data processing system includes a traffic splitter, at least one signaling parsing server, at least one signaling synthesis server, and multiple user plane data processing servers. The signaling parsing server is configured to parse mobile communication signaling, obtain user identification information, and generate internal signaling based on the parsing results; The signaling synthesis server is configured to correlate and synthesize internal signaling to generate signaling plane-user plane associated data. The signaling plane-user plane associated data includes a user plane association index and user tag information. The user plane association index includes a destination IP address and a tunnel endpoint identifier (TEID). Using the destination IP address and the tunnel endpoint identifier (TEID) as load sharing parameters, a preset unified hash algorithm is used to generate load sharing keywords. The load sharing keywords and the signaling plane-user plane associated data are then encapsulated into internal data packets and sent to the traffic splitter. The traffic splitter is configured to extract load-sharing keywords from internal data packets, forward signaling plane-user plane associated data to the target user plane data processing server according to the load-sharing keywords and preset server mapping rules; and extract the destination IP address and tunnel endpoint identifier TEID from GTP-U user plane data, generate user plane load-sharing keywords using a unified HASH algorithm, and forward GTP-U user plane data to the corresponding target user plane data processing server according to the user plane load-sharing keywords and preset server mapping rules. Each of the user plane data processing servers is configured to establish, update, or delete a signaling plane-user plane association table based on the received signaling plane-user plane association data, and to perform association processing on the received GTP-U user plane data based on the signaling plane-user plane association table. The signaling plane-user plane associated data and its corresponding GTP-U user plane data are allocated to the same target user plane data processing server based on the same load sharing parameters, unified HASH algorithm and preset server mapping rules.

[0016] By adopting the above solution, the beneficial effects of the present invention are: 1) Both the signaling plane-user plane associated data and the corresponding GTP-U user plane data use the destination IP address and tunnel endpoint identifier (TEID) as load-sharing parameters, and a unified hash algorithm is used to generate load-sharing keywords, enabling them to be stably distributed to the same user plane data processing server. Therefore, it is unnecessary to copy and send every piece of signaling plane-user plane associated data to all user plane data processing servers; 2) By avoiding the broadcasting and multiple copies of signaling plane-user plane associated data, the amount of internal message exchange between the traffic splitter, signaling synthesis server and user plane data processing server can be reduced, thereby reducing internal network bandwidth usage and data processing latency. 3) Each user plane data processing server only needs to store the signaling plane-user plane association table entries corresponding to the user plane data processed by this server, which can significantly reduce the number of invalid association table entries, reduce server memory usage and DPI system hardware costs; 4) The traffic splitter and user plane data processing server perform special processing on IP fragmented packets (including signaling and data). For IP fragmented user plane data (GTP-U packets, the middle and last fragments of outer IP fragmented packets), even if TEID is missing, it can still be accurately distributed to the expected user plane data processing server. 5) When sending signaling plane-user plane associated messages, the signaling synthesis server improves the efficiency of network card sending and receiving of associated messages by classifying and splicing messages through load sharing keywords. Attached Figure Description

[0017] Figure 1 This is an architecture diagram of the system of the present invention; Figure 2 This is a flowchart of DPI data processing in a specific embodiment of the present invention; Figure 3 This is a flowchart of the load sharing process for a flow splitter in a specific embodiment of the present invention; Figure 4 This is a flowchart of a signaling synthesis processor sending a signaling plane-user plane associated message in a specific embodiment of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0019] Reference Figures 1 to 4 As shown, the present invention provides a distributed DPI data processing method, which is applied to a distributed DPI data processing system. The distributed DPI data processing system includes a traffic splitter, at least one signaling parsing server, at least one signaling synthesis server, and multiple user plane data processing servers.

[0020] This distributed DPI data processing system processes six main types of data through its traffic splitter: 1) Signaling: Mobile communication signaling (excluding IP fragmented messages). Message characteristics are determined by the protocol and port. Examples of message characteristics: 4G S1AP signaling uses SCTP protocol port 36412, 5G NGAP signaling uses SCTP protocol port 38412, GTPv2 signaling uses UDP protocol port 2123, and PFCP signaling uses UDP protocol port 8805. Output load balancing types include: source IP, destination IP, and source-destination IP. 2) User plane data (non-IP fragmentation): GTP-U packets (non-outer IP fragmentation). The packet characteristics are determined by the protocol, port, and IP protocol header fragmentation field. Packet characteristics: UDP protocol, port 2152, IP protocol header fragmentation field indicates that the packet is not fragmented; Output load balancing types include: destination IP, TEID, destination IP + TEID, which are various combinations of GTP-U protocol tuples (destination IP, TEID); 3) IP Fragmentation Data: All IP fragmented packets (including signaling and user plane data). The packet characteristics are determined by the protocol, port, and IP protocol header fragmentation field. Packet characteristics: The IP protocol header fragmentation field identifies packet fragments (first fragment, middle fragment, last fragment); Output load balancing types include: source IP, destination IP, and source-destination IP. 4) Internal signaling; Signaling parsing server -> Signaling merging server, message characteristics are determined by Ethernet type, message characteristics: for example, Ethernet type 0x5565 or 0x5566, 0x5565 is a 1-to-1 transmission message, 0x5566 is a 1-to-many transmission message; Signaling merging server -> Signaling parsing server, message characteristics are determined by Ethernet type, message characteristics: for example, Ethernet type 0x5567; Output load balancing type includes: load balancing keywords in the private message header; 5) Internal data; including signaling plane-user plane associated messages, IP fragmentation forwarding messages, user plane data processing server or signaling synthesis server -> user plane data processing server, message characteristics are determined by Ethernet type, message characteristics: for example, Ethernet type 0x5568; output load sharing type includes: load sharing keywords in private message headers; 6) User plane data (tagged): GTP-U tunnels can be stripped as needed; output load sharing types include various combinations of IP quintuples.

[0021] For internal signaling and internal data, the message encapsulation format is as follows: Ethernet – Private_Header – Payload, where Ethernet uses a private Ethernet type: such as 0x5565 / 0x5566 / 0x557 / 0x5568.

[0022] Private_Header includes: Payload_Type, 2 bytes, includes internal signaling messages, signaling plane-user plane associated messages, and user plane forwarding messages; Payload_Len, 2 bytes, represents the payload length. Loadshare Key: 4 bytes, the value output by a unified hash algorithm based on the output load sharing type. The unified hash algorithm is used by the traffic splitter, signaling parsing, signaling synthesis, and user plane data processing. The input to the hash algorithm is determined by the load sharing type. If the load sharing type is (IP + TEID), the input is (IP + TEID); if the load sharing type is (source IP + destination IP), the input is (source IP + destination IP). The output bit width of the hash algorithm is configurable from 8 to 32 bits; a larger bit width results in more even load sharing.

[0023] Specifically, the distributed DPI data processing method of the present invention is as follows: S1: The signaling parsing server parses the mobile communication signaling, obtains user identification information, and generates internal signaling based on the parsing results, which is then sent to the signaling synthesis server.

[0024] S2: The signaling synthesis server associates and synthesizes internal signaling to generate signaling plane-user plane associated data. The signaling plane-user plane associated data includes a user plane association index and user tag information corresponding to the user plane association index. The user plane association index includes the destination IP address and tunnel endpoint identifier (TEID).

[0025] S3: The signaling synthesis server uses the destination IP address and tunnel endpoint identifier TEID as load sharing parameters, generates load sharing keywords using a preset unified HASH algorithm, encapsulates the load sharing keywords and signaling plane-user plane associated data into an internal data packet, and sends the internal data packet to the traffic splitter.

[0026] S4: The traffic splitter extracts the load sharing keyword from the internal data packets, determines the target user plane data processing server from multiple user plane data processing servers based on the load sharing keyword and the preset server mapping rules, and forwards the signaling plane-user plane associated data to the target user plane data processing server. S5: The traffic splitter receives GTP-U user plane data corresponding to the signaling plane-user plane associated data, extracts the destination IP address and tunnel endpoint identifier (TEID) from the GTP-U user plane data, uses the extracted destination IP address and tunnel endpoint identifier (TEID) as load sharing parameters, generates user plane load sharing keywords using a unified HASH algorithm, and forwards the GTP-U user plane data to the corresponding target user plane data processing server according to the user plane load sharing keywords and preset server mapping rules. S6: The target user plane data processing server establishes, updates, or deletes the signaling plane-user plane association table based on the received signaling plane-user plane association data, and performs association processing on the received GTP-U user plane data based on the signaling plane-user plane association table. Specifically, the same load-sharing keyword is generated for signaling plane-user plane associated data and GTP-U user plane data that have the same destination IP address and tunnel endpoint identifier (TEID), so that the signaling plane-user plane associated data and its corresponding GTP-U user plane data are assigned to the same target user plane data processing server.

[0027] The internal data packet includes an Ethernet header, a private header, and a payload. The Ethernet header includes a private Ethernet type for identifying the internal data packet. The private header includes at least a payload type field, a payload length field, and a load-sharing keyword field. The payload type field is used to identify internal signaling packets, signaling plane-user plane associated packets, or IP fragmentation forwarding packets. The load-sharing keyword field is used to carry load-sharing keywords generated by a unified hash algorithm.

[0028] The traffic splitter identifies non-IP fragmented GTP-U user plane data based on protocol type, port, and IP protocol header fragmentation field. After identifying GTP-U user plane data, the traffic splitter prioritizes using GTP-U protocol load balancing types that include destination IP address and tunnel endpoint identifier TEID, rather than using load balancing types based on IP 5-tuples, and generates user plane load balancing keywords through a unified HASH algorithm.

[0029] In S1, the user identification information includes at least one of the following: International Mobile Subscriber Identity (IMSI) or Permanent Subscriber Identity (SUPI), Mobile Subscriber Number (MSISDN) or General Public Subscriber Identity (GPSI), and International Mobile Equipment Identity (IMEI) or Permanent Equipment Identity (PEI). When the signaling parsing server extracts user identification information from mobile communication signaling, it uses at least one of the user identification information as HASH input to generate internal signaling load sharing keywords and sends internal signaling in a one-to-one manner. When the signaling parsing server does not extract user identification information, it sends internal signaling in a one-to-many manner so that the signaling synthesis server can perform multi-interface signaling association and synthesis.

[0030] When the signaling synthesis server sends signaling plane-user plane associated data: Retrieve multiple signaling plane-user plane associated packets in batches from the internal queue; Extract the load sharing keywords from each signaling plane-user plane associated message; Multiple signaling plane-user plane related messages are categorized according to load sharing keywords; At least two signaling plane-user plane associated messages with the same load sharing keyword are concatenated, and the length of the concatenated message is controlled to be less than the preset maximum transmission unit (MTU) value. Batch transmission of signaling plane-user plane associated messages after classification and splicing.

[0031] It also includes an IP fragmentation data processing step, which includes: The traffic splitter identifies IP fragmented packets and uses the source IP address and destination IP address of the IP fragmented packets as initial load sharing parameters to distribute IP fragmented packets belonging to the same IP session to the same first user plane data processing server. The first user plane data processing server establishes IP fragmentation tracking information based on the first fragment packet, and associates the intermediate fragment packets and the last fragment packets with the first fragment packet to obtain the tunnel endpoint identifier (TEID) in the first fragment packet. The first user plane data processing server uses the destination IP address and tunnel endpoint identifier TEID as load sharing parameters, uses a unified HASH algorithm to generate fragmentation load sharing keywords, encapsulates the IP fragmentation packets and fragmentation load sharing keywords into IP fragmentation forwarding packets and sends them to the traffic splitter. The traffic splitter sends IP fragment forwarding packets to the target user plane data processing server corresponding to the fragment load sharing keyword, based on the fragment load sharing keyword.

[0032] The user tag information includes at least one of user identification information and location information. The location information includes at least one of Tracking Area Identifier (TAI), Evolved Universal Terrestrial Radio Access Network Cell Global Identifier (ECGI), and New Radio Interface Cell Global Identifier (NR-CGI). The target user plane data processing server queries the signaling plane-user plane association table, adds the queried user tag information to the GTP-U user plane data in Type-Length-Value (TLV) format, and outputs the user plane data with added user tag information to the backend system via a traffic splitter.

[0033] Specific Implementation Example 1: Signaling Processing Flow Reference Figure 2 As shown, it specifically includes: 1) When external traffic enters the traffic splitter, it matches the external input packets according to the splitting rules: signaling, user plane data (non-IP fragmented), and IP fragmented data. The traffic splitter prioritizes matching IP fragmented data, and signaling with IP fragments belongs to IP fragmented data. In this step, if the signaling (specifically non-IP fragmented signaling, which will not be emphasized later) is successfully matched, assuming the configured output load balancing type is: source and destination IP, the traffic splitter uses a unified HASH algorithm to calculate the load balancing keyword based on the source and destination IPs, and uses this load balancing keyword to select a signaling parsing server. 2) The traffic splitter outputs signaling to a signaling parsing server; 3) The signaling parsing server parses the signaling and extracts information such as the three codes (IMSI / SUPI, MSISDN / GPSI, IMEI / PEI); it constructs internal signaling (private messages). If the three codes are available, it uses one of the three codes to calculate the load sharing keyword and sends messages one-to-one; if the three codes are not available, it sends messages one-to-many. 4) The traffic splitter forwards or replicates packets to the signaling merging server based on the Ethernet type and load balancing keywords; 5) The signaling synthesis server receives internal signaling, performs multi-interface association and synthesis of signaling, and obtains information such as (IP address + TEID), 3 codes (IMSI / SUPI, MSISDN / GPSI, IMEI / PEI), and location (e.g., TAI, ECGI, NR-CGI). If it is the first time the 3 codes are obtained, the signaling synthesis server constructs internal signaling (private message) and forwards the internal signaling to the signaling resolution server through the traffic splitter. 6) The traffic splitter forwards packets to the signaling parsing server based on the Ethernet type and load balancing keywords; 7) The signaling synthesis server has obtained enough signaling plane-user plane association information, constructs a private message, and assuming that the configured output load sharing type is (destination IP + TEID), it calculates the load sharing keyword based on (destination IP + TEID) using a unified HASH algorithm and sends the signaling plane-user plane association data to the traffic splitter. 8) The traffic splitter forwards packets to the user plane data processing server based on the Ethernet type and load balancing keywords; the user plane data processing server creates / updates / deletes the signaling plane-user plane association table; 9) External traffic enters the traffic splitter. According to the splitting rules, it matches external input packets: signaling, user plane data (non-IP fragmented), and IP fragmented data. In this step, the user plane data (non-IP fragmented) is successfully matched. Assuming the configured output load balancing type is (destination IP + TEID), the load balancing keyword is calculated using a unified HASH algorithm based on (destination IP + TEID). This load balancing keyword is then used to select a user plane data processing server. Since the signaling plane-user plane associated data also uses the same load balancing keyword, the associated data and related user plane data will be sent to the same user plane data processing server, avoiding broadcasting or multiple copies of the associated data. 10) The user plane data processing server receives the message, looks up the association table, and marks the 3-code and location information into the message, such as adding a TLV tag to the end of the message; 11) The user plane data processing server forwards tagged packets to the traffic splitter; 12) The flow splitter outputs the processed tagged message to the backend system; 13) When external traffic enters the traffic splitter, it matches the external input packets according to the splitting rules: signaling, user plane data (non-IP fragmented), and IP fragmented data. In this step, the IP fragmented data is successfully matched. Assuming the configured output load balancing type is: source and destination IP, the traffic splitter uses a unified HASH algorithm to calculate the load balancing keyword based on the source and destination IP, and uses the load balancing keyword to select a user plane data processing server. 14) The traffic splitter forwards IP fragmented data to the user plane data processing server. For IP fragmented packets, only the first fragment carries protocol and port information. For IP fragmented packets of signaling and user plane data, they are all processed by the user plane data processing server. The user plane data processing server performs IP fragmentation and reassembly for signaling and IP fragmentation tracking for user plane data based on the characteristics of the first fragment. Non-first fragment packets (middle fragments, tail fragments) are associated with the first fragment packet to obtain the TEID in the first fragment packet. The load sharing keyword can be calculated through (destination IP + TEID) to ensure that fragmented packets of the same IP session are distributed to the same user plane data processing server. 15) For IP fragmentation (user plane data), the user plane data processing server constructs internal packets and sends them to the traffic splitter; 16) The traffic splitter forwards packets to the user plane data processing server based on the Ethernet type and load balancing keywords; the user plane data processing server removes the internal private data headers, queries the association table, and tags the packets; 17) The user plane data processing server sends tagged IP fragmented packets to the traffic splitter; 18) The traffic splitter outputs the processed IP fragmented packets to the backend system; 19) The user plane data processing server forwards the signaling message for successful reassembly to the traffic splitter; 20) The traffic splitter recognizes the signaling and forwards it to the signaling parsing server. The subsequent processing is consistent with the signaling input from the outside.

[0034] In this embodiment, the flow splitter has three important characteristics: 1) Load balancing types: IP 5-tuple (source IP, destination IP, source port, destination port, protocol) combination, GTP-U protocol 2-tuple (destination IP, TEID) combination; 2) The HASH algorithm for generating load sharing keywords based on load sharing type is a unified algorithm for the entire DPI data processing system; 3) Supports processing of Layer 2 private protocols. For specific Ethernet type packets (such as 0x5565 / 0x5566 / 0x5567 / 0x5568), it parses the specific private protocol header and directly uses the load sharing keyword in the packet.

[0035] In a specific embodiment, refer to Figure 3 As shown, the load sharing process of the flow splitter is as follows: 1) The traffic splitter identifies the Ethernet protocol type and processes them into three categories: standard Ethernet types (such as IPv4, IPv6, VLAN, MPLS, etc.), specific Ethernet types (such as x05565), and other Ethernet types. 2) For standard Ethernet types, a unified HASH algorithm is used to generate load sharing keywords based on the configured load sharing type (combination of IP 5-tuples, combination of GTP-U 2-tuples); based on the load sharing keywords, one output port is selected from multiple ports in the output port group. 3) For a specific Ethernet type, parse the private protocol header and directly extract the load sharing keyword; based on the load sharing keyword, select one output port from multiple ports in the output port group; 4) For other Ethernet types, generate a random load-sharing keyword based on the current time; select an output port from multiple ports in the output port group based on the load-sharing keyword.

[0036] For signaling merging servers, if a signaling plane-user plane associated message contains only one signaling plane-user plane associated index (destination IP + TEID), it will result in the transmission of a large number of small messages, significantly impacting the processing efficiency of the sending and receiving network interface cards (NICs). When sending signaling plane-user plane associated messages, the signaling merging processor concatenates messages according to load-sharing keywords, combining messages with the same load-sharing keywords into a larger message, thus improving the processing efficiency of the sending and receiving NICs.

[0037] In a specific embodiment, refer to Figure 4 As shown, the process by which the signaling synthesis processor sends signaling plane-user plane associated messages is as follows: 1) Receive a certain number of signaling plane-user plane associated messages in batches from the internal queue. The number is configurable (e.g., 64). 2) Extract load-sharing keywords from each message; 3) Classify the received packets in batches using load-sharing keywords; 4) Concatenate messages with the same load sharing keyword, and ensure that the length of the concatenated message is less than the configured MTU (Maximum Transmission Unit) value; the smaller the value range of the load sharing keyword (8-32 bits), the higher the concatenation success rate. 5) Send the processed messages in batches.

[0038] In summary, to enable signaling plane-user plane associated messages and related user plane messages (GTP-U messages) to enter the same user plane data processing server, the present invention mainly makes the following improvements: 1) Customize the load balancing type of the GTP-U protocol for the traffic splitter: various combinations of GTP-U protocol tuples (destination IP + TEID); after the packet is identified as GTP-U protocol, the GTP-U load balancing type is used first; 2) The traffic splitter and user plane data processing server perform special processing on IP fragmented packets (including signaling and data). For IP fragmented user plane data (GTP-U packets, the middle and last fragments of the outer IP fragmented packets), even if the TEID is missing, it can still be accurately distributed to the expected user plane data processing server. 3) When constructing the signaling plane-user plane associated message, the signaling synthesis server uses a unified HASH algorithm (consistent with the traffic splitter algorithm) to generate load-sharing keywords, so that the associated message and GTP-U message load are distributed to the same server; 4) When sending signaling plane-user plane associated messages, the signaling synthesis server improves the efficiency of network card sending and receiving of associated messages by classifying and splicing messages through load sharing keywords.

[0039] In addition, a distributed DPI data processing system is provided, including a traffic splitter, at least one signaling parsing server, at least one signaling synthesis server, and multiple user plane data processing servers. The signaling parsing server is configured to parse mobile communication signaling, obtain user identification information, and generate internal signaling based on the parsing results; The signaling synthesis server is configured to correlate and synthesize internal signaling to generate signaling plane-user plane associated data. The signaling plane-user plane associated data includes a user plane association index and user tag information. The user plane association index includes a destination IP address and a tunnel endpoint identifier (TEID). Using the destination IP address and the tunnel endpoint identifier (TEID) as load sharing parameters, a preset unified hash algorithm is used to generate load sharing keywords. The load sharing keywords and the signaling plane-user plane associated data are then encapsulated into internal data packets and sent to the traffic splitter. The traffic splitter is configured to extract load-sharing keywords from internal data packets, forward signaling plane-user plane associated data to the target user plane data processing server according to the load-sharing keywords and preset server mapping rules; and extract the destination IP address and tunnel endpoint identifier TEID from GTP-U user plane data, generate user plane load-sharing keywords using a unified HASH algorithm, and forward GTP-U user plane data to the corresponding target user plane data processing server according to the user plane load-sharing keywords and preset server mapping rules. Each of the user plane data processing servers is configured to establish, update, or delete a signaling plane-user plane association table based on the received signaling plane-user plane association data, and to perform association processing on the received GTP-U user plane data based on the signaling plane-user plane association table. The signaling plane-user plane associated data and its corresponding GTP-U user plane data are allocated to the same target user plane data processing server based on the same load sharing parameters, unified HASH algorithm and preset server mapping rules.

[0040] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A distributed DPI data processing method, applied to a distributed DPI data processing system, wherein, The distributed DPI data processing system includes a traffic splitter, at least one signaling parsing server, at least one signaling synthesis server, and multiple user plane data processing servers. Its features include the following steps: S1: The signaling parsing server parses the mobile communication signaling, obtains user identification information, and generates internal signaling based on the parsing results, which is then sent to the signaling synthesis server. S2: The signaling synthesis server associates and synthesizes internal signaling to generate signaling plane-user plane association data. The signaling plane-user plane association data includes a user plane association index and user tag information corresponding to the user plane association index. The user plane association index includes the destination IP address and tunnel endpoint identifier TEID. S3: The signaling synthesis server uses the destination IP address and tunnel endpoint identifier TEID as load sharing parameters, generates load sharing keywords using a preset unified HASH algorithm, encapsulates the load sharing keywords and signaling plane-user plane associated data into an internal data packet, and sends the internal data packet to the traffic splitter. S4: The traffic splitter extracts the load sharing keyword from the internal data packets, determines the target user plane data processing server from multiple user plane data processing servers based on the load sharing keyword and the preset server mapping rules, and forwards the signaling plane-user plane associated data to the target user plane data processing server. S5: The traffic splitter receives GTP-U user plane data corresponding to the signaling plane-user plane associated data, extracts the destination IP address and tunnel endpoint identifier (TEID) from the GTP-U user plane data, uses the extracted destination IP address and tunnel endpoint identifier (TEID) as load sharing parameters, generates user plane load sharing keywords using a unified HASH algorithm, and forwards the GTP-U user plane data to the corresponding target user plane data processing server according to the user plane load sharing keywords and preset server mapping rules. S6: The target user plane data processing server establishes, updates, or deletes the signaling plane-user plane association table based on the received signaling plane-user plane association data, and performs association processing on the received GTP-U user plane data based on the signaling plane-user plane association table. Specifically, the same load-sharing keyword is generated for signaling plane-user plane associated data and GTP-U user plane data that have the same destination IP address and tunnel endpoint identifier (TEID), so that the signaling plane-user plane associated data and its corresponding GTP-U user plane data are assigned to the same target user plane data processing server.

2. The distributed DPI data processing method according to claim 1, characterized in that, The internal data packet includes an Ethernet header, a private packet header, and a payload; the Ethernet header includes a private Ethernet type for identifying the internal data packet; the private packet header includes at least a payload type field, a payload length field, and a load-sharing keyword field, wherein the payload type field is used to identify internal signaling packets, signaling plane-user plane associated packets, or IP fragmentation forwarding packets, and the load-sharing keyword field is used to carry load-sharing keywords generated by a unified HASH algorithm.

3. The distributed DPI data processing method according to claim 1, characterized in that, The traffic splitter identifies non-IP fragmented GTP-U user plane data based on protocol type, port, and IP protocol header fragmentation field. After identifying GTP-U user plane data, the traffic splitter prioritizes using GTP-U protocol load balancing types that include destination IP address and tunnel endpoint identifier TEID, rather than using load balancing types based on IP 5-tuples, and generates user plane load balancing keywords through a unified HASH algorithm.

4. The distributed DPI data processing method according to claim 1, characterized in that, In S1, the user identification information includes at least one of the following: International Mobile Subscriber Identity (IMSI) or Permanent Subscriber Identity (SUPI), Mobile Subscriber Number (MSISDN) or General Public Subscriber Identity (GPSI), and International Mobile Equipment Identity (IMEI) or Permanent Equipment Identity (PEI). When the signaling parsing server extracts user identification information from mobile communication signaling, it uses at least one of the user identification information as HASH input to generate internal signaling load sharing keywords and sends internal signaling in a one-to-one manner. When the signaling parsing server does not extract user identification information, it sends internal signaling in a one-to-many manner so that the signaling synthesis server can perform multi-interface signaling association and synthesis.

5. The distributed DPI data processing method according to claim 1, characterized in that, When the signaling synthesis server sends signaling plane-user plane associated data: Retrieve multiple signaling plane-user plane associated packets in batches from the internal queue; Extract the load sharing keywords from each signaling plane-user plane associated message; Multiple signaling plane-user plane related messages are categorized according to load sharing keywords; At least two signaling plane-user plane associated messages with the same load sharing keyword are concatenated, and the length of the concatenated message is controlled to be less than the preset maximum transmission unit (MTU) value. Batch transmission of signaling plane-user plane associated messages after classification and splicing.

6. The distributed DPI data processing method according to claim 1, characterized in that, It also includes an IP fragmentation data processing step, which includes: The traffic splitter identifies IP fragmented packets and uses the source IP address and destination IP address of the IP fragmented packets as initial load sharing parameters to distribute IP fragmented packets belonging to the same IP session to the same first user plane data processing server. The first user plane data processing server establishes IP fragmentation tracking information based on the first fragment packet, and associates the intermediate fragment packets and the last fragment packets with the first fragment packet to obtain the tunnel endpoint identifier (TEID) in the first fragment packet. The first user plane data processing server uses the destination IP address and tunnel endpoint identifier TEID as load sharing parameters, uses a unified HASH algorithm to generate fragmentation load sharing keywords, encapsulates the IP fragmentation packets and fragmentation load sharing keywords into IP fragmentation forwarding packets and sends them to the traffic splitter. The traffic splitter sends IP fragment forwarding packets to the target user plane data processing server corresponding to the fragment load sharing keyword, based on the fragment load sharing keyword.

7. The distributed DPI data processing method according to claim 1, characterized in that, The user tag information includes at least one of user identification information and location information. The location information includes at least one of Tracking Area Identifier (TAI), Evolved Universal Terrestrial Radio Access Network Cell Global Identifier (ECGI), and New Radio Interface Cell Global Identifier (NR-CGI). The target user plane data processing server queries the signaling plane-user plane association table, adds the queried user tag information to the GTP-U user plane data in Type-Length-Value (TLV) format, and outputs the user plane data with added user tag information to the backend system via a traffic splitter.

8. A distributed DPI data processing system, characterized in that, It includes a traffic splitter, at least one signaling parsing server, at least one signaling synthesis server, and multiple user plane data processing servers; The signaling parsing server is configured to parse mobile communication signaling, obtain user identification information, and generate internal signaling based on the parsing results; The signaling synthesis server is configured to associate and synthesize internal signaling to generate signaling plane-user plane association data. The signaling plane-user plane association data includes user plane association index and user tag information. The user plane association index includes destination IP address and tunnel endpoint identifier TEID. Using the destination IP address and tunnel endpoint identifier TEID as load sharing parameters, a pre-defined unified HASH algorithm is used to generate load sharing keywords; And encapsulate the load sharing keywords and signaling plane-user plane associated data into an internal data packet and send it to the traffic splitter; The traffic splitter is configured to extract load sharing keywords from internal data packets and forward signaling plane-user plane associated data to the target user plane data processing server according to the load sharing keywords and preset server mapping rules. It also extracts the destination IP address and tunnel endpoint identifier TEID from the GTP-U user plane data, generates user plane load sharing keywords using a unified HASH algorithm, and forwards the GTP-U user plane data to the corresponding target user plane data processing server according to the user plane load sharing keywords and preset server mapping rules. Each of the user plane data processing servers is configured to establish, update, or delete a signaling plane-user plane association table based on the received signaling plane-user plane association data, and to perform association processing on the received GTP-U user plane data based on the signaling plane-user plane association table. The signaling plane-user plane associated data and its corresponding GTP-U user plane data are allocated to the same target user plane data processing server based on the same load sharing parameters, unified HASH algorithm and preset server mapping rules.

Images