Method, system and computer program product for protocol parsing for network security

Abstract

A method for protocol parsing for network security is provided. The method may include: receiving a plurality of packets by a packet capture system; parsing lower layer data from each packet; and transmitting a corresponding payload of each corresponding packet to at least one first queue. A routing system may route the corresponding payload of each corresponding packet to a corresponding second queue of a plurality of second queues based on the corresponding protocol of each corresponding packet. A corresponding protocol parser node of the parsing system may parse higher layer data from the corresponding payload of each corresponding packet from each corresponding second queue. The packet capture system may transmit the lower layer data of each packet to a third queue, and the parsing system may transmit the higher layer data of each packet to the third queue. A system and a computer program product are also disclosed.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application No. 17 / 232,738, filed on April 16, 2021, the disclosure of which is hereby incorporated by reference in its entirety. Technical Field

[0003] The subject matter of the present disclosure relates generally to methods, systems, and products for protocol parsing, and in non-limiting embodiments or aspects, to methods, systems, and computer program products for protocol parsing for network security. Background Art

[0004] Some network monitoring systems (e.g., intrusion detection systems, deep packet inspection tools, etc.) use highly coupled parsers and analyzers (e.g., parsers and analyzers embedded together in executable code and / or libraries (e.g., dynamic link libraries (DLLs), etc.)) to read various traffic, and the parsers and analyzers are bound to capture nodes. For example, network monitoring systems (e.g., Arkime / Moloch, Suricata, Zeek / Bro, etc.) utilize multiple protocol parsers to analyze network packets. However, due to such coupling between capture nodes and their parsers / analyzers (e.g., libraries), a relatively large amount of overhead (e.g., computing resources, network resources, etc.) may be involved in building, extending, and / or deploying new parsers and analyzers. For example, when a single change is made to one parser, all other parsers may be inadvertently affected because the parsers may share the same resources.

[0005] In addition, some network monitoring systems may lack the ability to aggregate network metadata, such as lower and higher layer protocol and payload information, from multiple instances of protocol parsers developed by multiple vendors. Moreover, network monitoring systems may lack the ability to represent custom / vendor specific metadata fields in a unified format or common fields. Due to such deficiencies, storage devices for metadata may be underutilized, detection and prevention of network intrusions may be inaccurate, and / or a comprehensive forensic investigation of a network attack may be difficult or impossible to achieve. For example, if multiple vendors develop different parsers for the same high-level (e.g., application layer) protocol (e.g., Hypertext Transfer Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), Domain Name System (DNS), etc.), and if multiple instances of these parsers are deployed to parse network traffic, the parsers may generate network metadata that includes vendor-specific custom fields that vary from vendor to vendor. There may be redundancy in the values ​​of the parsed data generated by the vendor-specific protocol parsers, but the names of the fields of the parsed data may be different because such names may be defined by each vendor. When the parsed data is stored in such heterogeneous formats in a database, the storage may be poorly utilized due to redundancy in the parsed data (e.g., storage is wasted by storing redundant values ​​in multiple fields). In addition, the heterogeneous format of the parsed data (e.g., custom fields) may create difficulties for intrusion detection, prevention, and / or investigation systems to analyze the parsed data from network traffic to identify network anatomy, as such systems would need to have custom logic to read the custom / vendor-specific metadata. Furthermore, the network monitoring system may not provide support for aggregating and encoding the parsed network metadata (e.g., which is stored in heterogeneous formats) into a common formatting / fields.

[0006] In addition, scaling such network monitoring systems may be challenging. For example, a protocol parser analyzes each network packet to extract information from it that can be used for intrusion prevention, intrusion detection, forensic investigation, etc. Such systems may encounter relatively few problems at low data rates (e.g., the number of packets received per second), but as the amount of traffic received by the system (e.g., its server) increases, problems may arise. Such problems may include packet delays, packet drops / packet losses, operating system (OS) bottlenecks, etc. For example, because the protocol parser is tightly coupled to the server, the protocol parser shares the server's computing resources (e.g., processor and / or memory resources), etc., which may cause bottlenecks. In addition, such problems cannot be easily solved without additional and usually expensive hardware costs (e.g., adding more servers, such as high-performance servers, etc.). In addition, bottlenecks may exist in many places and may be highly coupled to the resources being used by the corresponding application. In addition, some network monitoring systems may lack the ability to provision individual additional protocol parsers to cope with increased data rates (e.g., without provisioning the entire server instance, including unnecessary additional analyzers and / or parsers). Summary of the invention

[0007] Accordingly, improved methods, systems, and computer program products for protocol parsing for network security are provided.

[0008] According to non-limiting embodiments or aspects, a method for protocol parsing for network security is provided. In some non-limiting embodiments or aspects, a method for protocol parsing for network security may include receiving a plurality of packets by a packet capture system. The packet capture system may parse at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets. The packet capture system may transmit the corresponding payload of each corresponding packet in the plurality of packets to at least one first queue. The routing system may route the corresponding payload of each corresponding packet in the plurality of packets to the corresponding second queue in the plurality of second queues based on the corresponding protocol of the corresponding packet. The corresponding protocol parser node of the parsing system may parse at least one of session layer data, presentation layer data, or application layer data from the corresponding payload of each corresponding packet in each of the plurality of second queues. The packet capture system may transmit at least one of data link layer data, network layer data, or transport layer data of each of the plurality of packets to a third queue. The parsing system may transmit at least one of session layer data, presentation layer data, or application layer data of each of the plurality of packets to a third queue.

[0009] In some non-limiting embodiments or aspects, the packet capture system may include at least one first server. Additionally or alternatively, the routing system may include at least one second server. Additionally or alternatively, the parsing system may include a plurality of third servers. Additionally or alternatively, each respective protocol parser node may include a respective third server of the plurality of third servers.

[0010] In some non-limiting embodiments or aspects, each of the at least one first server may include a packet capture subsystem, a lower layer parser subsystem, and a protocol identifier subsystem. Additionally or alternatively, receiving the plurality of packets may include receiving the plurality of packets by the packet capture subsystem. Additionally or alternatively, parsing at least one of data link layer data, network layer data, or transport layer data from each packet may include parsing at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets by the lower layer parser subsystem. Additionally or alternatively, transmitting the respective payload of each respective packet of the plurality of packets to the at least one first queue may include determining the respective protocol of each respective packet of the plurality of packets by the protocol identifier subsystem, and / or transmitting the respective payload of each respective packet of the plurality of packets to the at least one first queue together with the protocol data associated with the respective protocol of the respective packet by the packet capture system.

[0011] In some non-limiting embodiments or aspects, each respective protocol parser node of the parsing system may be associated with a respective protocol of the plurality of protocols. Additionally or alternatively, each respective second queue of the plurality of second queues may be associated with a respective protocol parser node of the parsing system. Additionally or alternatively, routing each respective packet of the plurality of packets to a respective second queue of the plurality of second queues may include routing the respective packet to a selected one of the plurality of second queues based on protocol data of each respective packet of the plurality of packets corresponding to a respective protocol of a respective protocol parser node associated with a selected one of the plurality of second queues.

[0012] In some non-limiting embodiments or aspects, the aggregation system may aggregate at least one of the data link layer data, network layer data, or transport layer data of each packet with at least one of the session layer data, presentation layer data, or application layer data of each packet. Additionally or alternatively, at least one of the data link layer data, network layer data, or transport layer data of each packet aggregated with at least one of the session layer data, presentation layer data, or application layer data of each packet may be stored in a database.

[0013] In some non-limiting embodiments or aspects, the packet capture system may generate a corresponding packet identifier for each corresponding packet in the plurality of packets. Additionally or alternatively, transmitting the corresponding payload of each corresponding packet in the plurality of packets to the at least one first queue may include transmitting the corresponding payload of each corresponding packet in the plurality of packets together with the corresponding packet identifier of the corresponding packet. Additionally or alternatively, transmitting at least one of data link layer data, network layer data, or transport layer data of each packet in the plurality of packets to the third queue may include transmitting at least one of data link layer data, network layer data, or transport layer data of each packet in the plurality of packets together with the corresponding packet identifier of the corresponding packet to the third queue. Additionally or alternatively, transmitting at least one of session layer data, presentation layer data, or application layer data of each packet in the plurality of packets to the third queue may include transmitting at least one of session layer data, presentation layer data, or application layer data of each packet in the plurality of packets together with the corresponding packet identifier of the corresponding packet to the third queue. Additionally or alternatively, aggregating at least one of the data link layer data, network layer data, or transport layer data of each packet with at least one of the session layer data, presentation layer data, or application layer data of each packet may include aggregating at least one of the data link layer data, network layer data, or transport layer data of each packet with at least one of the session layer data, presentation layer data, or application layer data of each packet based on matching the corresponding packet identifier of the corresponding packet transmitted together with at least one of the data link layer data, the network layer data, or the transport layer data with the corresponding packet identifier of the corresponding packet transmitted together with at least one of the session layer data, the presentation layer data, or the application layer data.

[0014] In some non-limiting embodiments or aspects, a mapping database may receive custom field data associated with a plurality of custom fields. Additionally or alternatively, the mapping database may generate a corresponding mapping between each corresponding custom field in the plurality of custom fields and a corresponding common field of the plurality of common fields. Additionally or alternatively, the aggregation may include: based on a corresponding mapping of a corresponding custom field associated with at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data, or the application layer data to the corresponding common field, converting at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data, or the application layer data of a first packet of the plurality of packets.

[0015] In some non-limiting embodiments or aspects, the policy database may receive aggregation policy data associated with at least one aggregation policy. Additionally or alternatively, the aggregation may include filtering at least one field of the first packet of the plurality of packets based on the at least one aggregation policy.

[0016] In some non-limiting embodiments or aspects, the routing system can subscribe to a topic of the first queue, and each respective topic in the topics can be associated with a respective protocol in a plurality of protocols. Additionally or alternatively, transmitting a respective payload of each respective packet in the plurality of packets to at least one first queue can include publishing the respective payload of each respective packet in the plurality of packets with a respective topic in the topics in the first queue based on the respective protocol of the respective packet.

[0017] In some non-limiting embodiments or aspects, routing the respective payload of each respective packet in the plurality of packets may include publishing the respective payload of the respective packet in a respective second queue in the plurality of second queues based on the respective topic of each respective packet in the plurality of packets. Additionally or alternatively, parsing at least one of session layer data, presentation layer data, or application layer data from the respective payload of each respective packet may include consuming, by a respective protocol parser node of the parsing system, the respective payload of each respective packet from each respective second queue in the plurality of second queues.

[0018] In some non-limiting embodiments or aspects, a resource monitoring system may monitor the packet capture system, the routing system, the parsing system, the first queue, the plurality of second queues, and the third queue to provide a corresponding metric for each of the packet capture system, the routing system, the parsing system, the first queue, the second queue, and the third queue. Additionally or alternatively, a node provisioning system may determine whether to add or remove a node to or from at least one of the packet capture system, the routing system, or the parsing system based on at least one of the corresponding metric of the packet capture system, the corresponding metric of the routing system, the corresponding metric of the parsing system, the corresponding metric of the first queue, the corresponding metric of the second queue, or the corresponding metric of the third queue.

[0019] In some non-limiting embodiments or aspects, the resource monitoring system may include a packet capture system monitoring agent, and the node provisioning system may include a packet capture system node provisioning system. Additionally or alternatively, the resource monitoring system may include a first queue monitoring agent and a routing system monitoring agent, and the node provisioning system may include a routing system node provisioning system. Additionally or alternatively, the resource monitoring system may include a second queue monitoring agent and a resolution system monitoring agent, and the node provisioning system may include a resolution system node provisioning system.

[0020] According to non-limiting embodiments or aspects, a system for protocol parsing for network security is provided. In some non-limiting embodiments or aspects, a system for protocol parsing for network security may include a packet capture system, the packet capture system being configured to: receive a plurality of packets; parse at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets; transmit the corresponding payload of each corresponding packet in the plurality of packets to at least one first queue; and transmit at least one of the data link layer data, network layer data, or transport layer data of each packet in the plurality of packets to a third queue. The routing system may be configured to route the corresponding payload of each corresponding packet in the plurality of packets to a corresponding second queue in a plurality of second queues based on the corresponding protocol of the corresponding packet. The parsing system may include a plurality of protocol parser nodes. Each corresponding protocol parser node may be configured to: parse at least one of session layer data, presentation layer data, or application layer data from the corresponding payload of each corresponding packet in each of the plurality of second queues; and transmit at least one of the session layer data, presentation layer data, or application layer data of each packet in the plurality of packets to the third queue.

[0021] In some non-limiting embodiments or aspects, the packet capture system may include at least one first server, the routing system may include at least one second server, the parsing system may include multiple third servers, and / or each corresponding protocol parser node may include a corresponding third server among the multiple third servers.

[0022] In some non-limiting embodiments or aspects, each of the at least one first server may include a packet capture subsystem, a lower layer parser subsystem, and a protocol identifier subsystem. Additionally or alternatively, receiving the plurality of packets may include receiving the plurality of packets by the packet capture subsystem. Additionally or alternatively, wherein parsing at least one of data link layer data, network layer data, or transport layer data from each packet may include parsing at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets by the lower layer parser subsystem. Additionally or alternatively, transmitting the respective payload of each respective packet of the plurality of packets to the at least one first queue may include determining the respective protocol of each respective packet of the plurality of packets by the protocol identifier subsystem, and transmitting the respective payload of each respective packet of the plurality of packets to the at least one first queue together with the protocol data associated with the respective protocol of the respective packet by the packet capture system.

[0023] In some non-limiting embodiments or aspects, each respective protocol parser node of the parsing system may be associated with a respective protocol of a plurality of protocols. Additionally or alternatively, wherein each respective second queue of the plurality of second queues may be associated with a respective protocol parser node of the parsing system. Additionally or alternatively, routing each respective packet of the plurality of packets to a respective second queue of the plurality of second queues may include routing the respective packet to a selected one of the plurality of second queues based on protocol data of each respective packet of the plurality of packets corresponding to a respective protocol of a respective protocol parser associated with a selected one of the plurality of second queues.

[0024] In some non-limiting embodiments or aspects, the aggregation system can be configured to aggregate at least one of the data link layer data, network layer data, or transport layer data of each packet with at least one of the session layer data, presentation layer data, or application layer data of each packet, and store at least one of the data link layer data, network layer data, or transport layer data of each packet aggregated with at least one of the session layer data, presentation layer data, or application layer data of each packet in a database.

[0025] In some non-limiting embodiments or aspects, a mapping database may be configured to receive custom field data associated with a plurality of custom fields and generate a corresponding mapping between each corresponding custom field in the plurality of custom fields and a corresponding common field in the plurality of common fields. Additionally or alternatively, a policy database may be configured to receive aggregation policy data associated with at least one aggregation policy. Additionally or alternatively, aggregation may include: based on a corresponding mapping of a corresponding custom field associated with at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data, or the application layer data to the corresponding common field, converting at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data, or the application layer data of a first packet of the plurality of packets. Additionally or alternatively, aggregation may include filtering at least one field of the first packet of the plurality of packets based on the at least one aggregation policy.

[0026] In some non-limiting embodiments or aspects, the resource monitoring system may be configured to monitor the packet capture system, the routing system, the parsing system, the first queue, the plurality of second queues, and the third queue to provide a corresponding metric for each of the packet capture system, the routing system, the parsing system, the first queue, the second queue, and the third queue. Additionally or alternatively, the node provisioning system may be configured to determine whether to add or remove a node to or from at least one of the packet capture system, the routing system, or the parsing system based on at least one of the corresponding metric of the packet capture system, the corresponding metric of the routing system, the corresponding metric of the parsing system, the corresponding metric of the first queue, the corresponding metric of the second queue, or the corresponding metric of the third queue.

[0027] According to non-limiting embodiments or aspects, a computer program product for protocol parsing for network security is provided. In some non-limiting embodiments or aspects, a computer program product for protocol parsing for network security may include at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium including one or more instructions, the one or more instructions, when executed by at least one processor, cause the at least one processor to: receive a plurality of packets; parse at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets; transmit the corresponding payload of each corresponding packet in the plurality of packets to at least one first queue; route the corresponding payload of each corresponding packet in the plurality of packets to a corresponding second queue of a plurality of second queues based on the corresponding protocol of the corresponding packet; parse at least one of session layer data, presentation layer data, or application layer data from the corresponding payload of each corresponding packet from each of the plurality of second queues; transmit at least one of data link layer data, network layer data, or transport layer data of each of the plurality of packets to a third queue; and transmit at least one of session layer data, presentation layer data, or application layer data of each of the plurality of packets to the third queue.

[0028] Other non-limiting embodiments or aspects are set forth in the following numbered clauses:

[0029] Clause 1: A computer-implemented method comprising: receiving a plurality of packets by a packet capture system; parsing at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets by the packet capture system; transmitting a corresponding payload of each corresponding packet in the plurality of packets to at least one first queue by the packet capture system; routing a corresponding payload of each corresponding packet in the plurality of packets to a corresponding second queue in a plurality of second queues by a routing system based on a corresponding protocol of each corresponding packet in the plurality of packets; parsing at least one of session layer data, presentation layer data, or application layer data from a corresponding payload of each corresponding packet from each corresponding second queue in the plurality of second queues by a corresponding protocol parser node of the parsing system; transmitting at least one of data link layer data, network layer data, or transport layer data of each of the plurality of packets to a third queue by the packet capture system; and transmitting at least one of session layer data, presentation layer data, or application layer data of each of the plurality of packets to the third queue by the parsing system.

[0030] Clause 2: A method according to clause 1, wherein the packet capture system includes at least one first server, the routing system includes at least one second server, the parsing system includes multiple third servers, and each corresponding protocol parser node includes a corresponding third server among the multiple third servers.

[0031] Clause 3: A method according to clause 1 or 2, wherein each of the at least one first server includes a packet capture subsystem, a lower layer parser subsystem and a protocol identifier subsystem, wherein receiving the multiple packets includes receiving the multiple packets by the packet capture subsystem, wherein parsing at least one of data link layer data, network layer data or transport layer data from each packet includes parsing at least one of data link layer data, network layer data or transport layer data from each packet of the multiple packets by the lower layer parser subsystem, and wherein transmitting the corresponding payload of each corresponding packet of the multiple packets to the at least one first queue includes: determining the corresponding protocol of each corresponding packet of the multiple packets by the protocol identifier subsystem; and transmitting the corresponding payload of each corresponding packet of the multiple packets together with the protocol data associated with the corresponding protocol of the corresponding packet by the packet capture system to the at least one first queue.

[0032] Clause 4: A method according to any one of clauses 1-3, wherein each corresponding protocol parser node of the parsing system is associated with a corresponding protocol among a plurality of protocols, wherein each corresponding second queue of the plurality of second queues is associated with a corresponding protocol parser node of the parsing system, and wherein routing each corresponding packet of the plurality of packets to a corresponding second queue of the plurality of second queues comprises: routing the corresponding packet to a selected one of the plurality of second queues based on protocol data of each corresponding packet of the plurality of packets corresponding to a corresponding protocol of a corresponding protocol parser node associated with a selected one of the plurality of second queues.

[0033] Clause 5: The method according to any one of clauses 1-4 further includes: aggregating at least one of the data link layer data, network layer data or transport layer data of each packet with at least one of the session layer data, presentation layer data or application layer data of each packet by an aggregation system; and storing at least one of the data link layer data, network layer data or transport layer data of each packet aggregated with at least one of the session layer data, presentation layer data or application layer data of each packet in a database.

[0034] Clause 6: The method according to any one of clauses 1-5 further includes: generating, by the packet capture system, a corresponding packet identifier for each corresponding packet in the multiple packets, wherein transmitting the corresponding payload of each corresponding packet in the multiple packets to at least one first queue includes transmitting the corresponding payload of each corresponding packet in the multiple packets together with the corresponding packet identifier of the corresponding packet, wherein transmitting at least one of data link layer data, network layer data or transport layer data of each packet in the multiple packets to the third queue includes transmitting at least one of data link layer data, network layer data or transport layer data of each packet in the multiple packets together with the corresponding packet identifier of the corresponding packet to the third queue, wherein transmitting at least one of session layer data, presentation layer data or application layer data of each packet in the multiple packets to the third queue It includes transmitting at least one of the session layer data, presentation layer data or application layer data of each packet of the multiple packets together with the corresponding packet identifier of the corresponding packet to the third queue, and wherein, aggregating at least one of the data link layer data, network layer data or transport layer data of each packet with at least one of the session layer data, presentation layer data or application layer data of each packet includes: based on matching the corresponding packet identifier of the corresponding packet transmitted together with at least one of the session layer data, the presentation layer data or the application layer data with the corresponding packet identifier of the corresponding packet transmitted together with at least one of the data link layer data, the network layer data or the transport layer data, at least one of the data link layer data, the network layer data or the transport layer data of each packet is aggregating at least one of the data link layer data, the network layer data or the transport layer data with the at least one of the session layer data, the presentation layer data or the application layer data.

[0035] Clause 7: The method according to any one of clauses 1-6 further includes: receiving custom field data associated with multiple custom fields by a mapping database; and generating a corresponding mapping between each corresponding custom field in the multiple custom fields and a corresponding common field of a plurality of common fields by the mapping database, wherein the aggregation includes converting at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data or the application layer data of the first packet of the multiple packets based on the corresponding mapping of the corresponding custom field associated with at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data or the application layer data to the corresponding common field.

[0036] Clause 8: The method of any one of clauses 1-7, further comprising: receiving, by a policy database, aggregation policy data associated with at least one aggregation policy, wherein the aggregation comprises filtering at least one field of a first packet of the plurality of packets based on the at least one aggregation policy.

[0037] Clause 9: The method according to any one of clauses 1-8 further includes: the routing system subscribing to a topic of the first queue, each corresponding topic in the topics being associated with a corresponding protocol in a plurality of protocols, wherein transmitting the corresponding payload of each corresponding packet in the plurality of packets to at least one first queue includes publishing each corresponding payload of each corresponding packet in the plurality of packets in the first queue together with the corresponding topic in the topics based on the corresponding protocol of the corresponding packet.

[0038] Clause 10: A method according to any one of clauses 1-9, wherein routing the corresponding payload of each corresponding packet in the multiple packets includes publishing the corresponding payload of the corresponding packet in a corresponding second queue in the multiple second queues based on the corresponding topic of each corresponding packet in the multiple packets, and wherein parsing at least one of session layer data, presentation layer data, or application layer data from the corresponding payload of each corresponding packet includes: consuming the corresponding payload of each corresponding packet from each corresponding second queue in the multiple second queues by a corresponding protocol parser node of the parsing system.

[0039] Clause 11: The method according to any one of clauses 1-10 further includes: a resource monitoring system monitoring the packet capture system, the routing system, the parsing system, the first queue, the multiple second queues and the third queue to provide corresponding metrics for each of the packet capture system, the routing system, the parsing system, the first queue, the second queue and the third queue; and a node provisioning system determining whether to add or remove a node to or from at least one of the packet capture system, the routing system or the parsing system based on at least one of the corresponding metrics of the packet capture system, the corresponding metrics of the routing system, the corresponding metrics of the parsing system, the corresponding metrics of the first queue, the corresponding metrics of the second queue or the corresponding metrics of the third queue.

[0040] Clause 12: A method according to any one of clauses 1-11, wherein at least one of the following is true: the resource monitoring system includes a packet capture system monitoring agent, and the node provisioning system includes a packet capture system node provisioning system; the resource monitoring system includes a first queue monitoring agent and a routing system monitoring agent, and the node provisioning system includes a routing system node provisioning system; or the resource monitoring system includes a second queue monitoring agent and a resolution system monitoring agent, and the node provisioning system includes a resolution system node provisioning system.

[0041] Clause 13: A system comprising: a packet capture system, the packet capture system being configured to: receive a plurality of packets; parse at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets; transmit a corresponding payload of each corresponding packet in the plurality of packets to at least one first queue; and transmit at least one of the data link layer data, network layer data, or transport layer data of each of the plurality of packets to a third queue; a routing system, the routing system being configured to route the corresponding payload of each corresponding packet in the plurality of packets to a corresponding second queue in a plurality of second queues based on a corresponding protocol of the corresponding packet; and a parsing system comprising a plurality of protocol parser nodes, each corresponding protocol parser node being configured to: parse at least one of session layer data, presentation layer data, or application layer data from a corresponding payload of each corresponding packet from each corresponding second queue in the plurality of second queues; and transmit at least one of the session layer data, presentation layer data, or application layer data of each of the plurality of packets to the third queue.

[0042] Clause 14: A system according to clause 13, wherein the packet capture system includes at least one first server, the routing system includes at least one second server, the parsing system includes multiple third servers, and each corresponding protocol parser node includes a corresponding third server among the multiple third servers.

[0043] Clause 15: A system according to one of clauses 13 or 14, wherein each of the at least one first server includes a packet capture subsystem, a lower layer parser subsystem and a protocol identifier subsystem, wherein receiving the plurality of packets includes receiving the plurality of packets by the packet capture subsystem, wherein parsing at least one of data link layer data, network layer data or transport layer data from each packet includes parsing at least one of data link layer data, network layer data or transport layer data from each of the plurality of packets by the lower layer parser subsystem, and wherein transmitting a corresponding payload of each corresponding packet of the plurality of packets to the at least one first queue includes: determining a corresponding protocol of each corresponding packet of the plurality of packets by the protocol identifier subsystem; and transmitting a corresponding payload of each corresponding packet of the plurality of packets together with protocol data associated with the corresponding protocol of the corresponding packet by the packet capture system to the at least one first queue.

[0044] Clause 16: A system according to any one of clauses 13-15, wherein each corresponding protocol parser node of the parsing system is associated with a corresponding protocol among a plurality of protocols, wherein each corresponding second queue of the plurality of second queues is associated with a corresponding protocol parser node of the parsing system, and wherein routing each corresponding packet of the plurality of packets to a corresponding second queue of the plurality of second queues comprises: routing the corresponding packet to a selected one of the plurality of second queues based on protocol data of each corresponding packet of the plurality of packets corresponding to a corresponding protocol of a corresponding protocol parser associated with a selected one of the plurality of second queues.

[0045] Clause 17: The system according to any one of Clauses 13-16 further includes an aggregation system, which is configured to: aggregate at least one of the data link layer data, network layer data or transport layer data of each packet with at least one of the session layer data, presentation layer data or application layer data of each packet; and store at least one of the data link layer data, network layer data or transport layer data of each packet aggregated with at least one of the session layer data, presentation layer data or application layer data of each packet in a database.

[0046] Clause 18: The system according to any one of clauses 13-17 further includes: a mapping database, which is configured to receive custom field data associated with multiple custom fields and generate a corresponding mapping between each corresponding custom field in the multiple custom fields and a corresponding common field in the multiple common fields; and a policy database, which is configured to receive aggregation policy data associated with at least one aggregation policy, wherein the aggregation includes converting at least one of the data link layer data, network layer data, transport layer data, session layer data, presentation layer data or application layer data of a first packet of the multiple packets based on a corresponding mapping of a corresponding custom field associated with at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data or the application layer data to the corresponding common field, and wherein the aggregation includes filtering at least one field of the first packet of the multiple packets based on the at least one aggregation policy.

[0047] Clause 19: The system according to any one of clauses 13-18 further includes: a resource monitoring system, which is configured to monitor the packet capture system, the routing system, the parsing system, the first queue, the multiple second queues and the third queue to provide corresponding metrics for each of the packet capture system, the routing system, the parsing system, the first queue, the second queue and the third queue; and a node provisioning system, which is configured to determine whether to add a node to or remove a node from at least one of the packet capture system, the routing system or the parsing system based on at least one of the corresponding metrics of the packet capture system, the corresponding metrics of the routing system, the corresponding metrics of the parsing system, the corresponding metrics of the first queue, the corresponding metrics of the second queue or the corresponding metrics of the third queue.

[0048] Item 20: A computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to: receive a plurality of packets; parse at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets; transmit a corresponding payload of each corresponding packet in the plurality of packets to at least one first queue; route the corresponding payload of each corresponding packet in the plurality of packets to a corresponding second queue in a plurality of second queues based on a corresponding protocol of each corresponding packet in the plurality of packets; parse at least one of session layer data, presentation layer data, or application layer data from the corresponding payload of each corresponding packet from each of the plurality of second queues; transmit at least one of data link layer data, network layer data, or transport layer data for each of the plurality of packets to a third queue; and transmit at least one of session layer data, presentation layer data, or application layer data for each of the plurality of packets to the third queue.

[0049] These and other features and characteristics of the presently disclosed subject matter, as well as methods of operation and function of the related structural elements and combinations of parts and economies of manufacturing will become more apparent after considering the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals indicate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for illustration and description purposes only and are not intended as definitions of the limitations of the disclosed subject matter. As used in this specification and claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Additional advantages and details of the disclosed subject matter are explained in more detail below with reference to exemplary embodiments or aspects shown in the accompanying drawings, in which:

[0051] Figure 1 are diagrams of non-limiting embodiments or aspects of environments in which methods, systems, and / or computer program products described herein may be implemented according to the principles of the presently disclosed subject matter;

[0052] Figure 2 yes Figure 1 drawings of non-limiting embodiments or aspects of components of one or more devices;

[0053] Figure 3 is a flow chart of a non-limiting embodiment or aspect of a process for protocol parsing according to the principles of the presently disclosed subject matter;

[0054] Figures 4A-4CAccording to the principles of the presently disclosed subject matter Figure 3 drawings of non-limiting examples or aspects of implementation of non-limiting examples or aspects of the processes shown; and

[0055] Figure 5 According to the principles of the presently disclosed subject matter Figure 3 Swim lane diagram of a non-limiting example or aspect of implementation of a non-limiting example or aspect of a process is shown. DETAILED DESCRIPTION

[0056] For the purpose of description below, the terms "end", "upper", "lower", "right", "left", "vertical", "horizontal", "top", "bottom", "lateral", "longitudinal" and their derivatives shall refer to the disclosed subject matter as it is oriented in the accompanying drawings. However, it should be understood that the disclosed subject matter can adopt various alternative variations and step sequences, unless expressly specified to the contrary. It should also be understood that the specific devices and processes shown in the drawings and described in the following specification are merely exemplary embodiments or aspects of the disclosed subject matter. Therefore, unless otherwise indicated, specific dimensions and other physical characteristics associated with the embodiments or aspects disclosed herein should not be considered as limiting.

[0057] The aspects, parts, elements, structures, actions, steps, functions, instructions, etc. used herein should not be understood as critical or necessary unless explicitly described as such. Moreover, as used herein, the article "one" is intended to include one or more items, and can be used interchangeably with "one or more" and "at least one". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related items and unrelated items, etc.), and can be used interchangeably with "one or more" or "at least one". In the case of wishing only one item, the term "one" or similar language is used. Moreover, as used herein, the term "having" and the like are intended to be open terms. In addition, unless otherwise explicitly stated, the phrase "based on" is intended to mean "based at least in part on".

[0058] As used herein, the terms "communication" and "transmission" may refer to the reception, acceptance, transmission, migration, provision, etc. of information (e.g., data, signals, messages, instructions, commands, etc.). A unit (e.g., a device, a system, a component of a device or system, a combination thereof, etc.) communicating with another unit means that the unit is able to directly or indirectly receive information from the other unit and / or send information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, etc.) that is wired and / or wireless in nature. In addition, although the information sent may be modified, processed, relayed, and / or routed between the first unit and the second unit, the two units may also communicate with each other. For example, the first unit may communicate with the second unit even if the first unit passively receives information and does not actively send information to the second unit. As another example, the first unit may communicate with the second unit if at least one intermediate unit (e.g., a third unit located between the first unit and the second unit) processes the information received from the first unit and transmits the processed information to the second unit. In some non-limiting embodiments or aspects, a message may refer to a network packet (eg, a data packet, etc.) that includes data. It should be appreciated that many other arrangements are possible.

[0059] As used herein, the terms "issuer institution," "portable financial device issuer," "issuer," or "issuer bank" may refer to one or more entities that provide accounts to customers for conducting transactions (e.g., payment transactions) (e.g., initiating credit and / or debit payments). For example, an issuer institution may provide customers with account identifiers, such as a primary account number (PAN), that uniquely identify one or more accounts associated with the customer. The account identifier may be implemented on a portable financial device, such as a physical financial instrument (e.g., a payment card), and / or may be electronic and used for electronic payments. The terms "issuer institution" and "issuer institution system" may also refer to one or more computer systems operated by or on behalf of the issuer institution, such as a server computer executing one or more software applications. For example, an issuer institution system may include one or more authorization servers for authorizing transactions.

[0060] As used herein, the term "account identifier" may include one or more types of identifiers associated with a user account (e.g., PAN, card number, payment card number, payment token, etc.). In some non-limiting embodiments or aspects, an issuing institution may provide an account identifier (e.g., PAN, payment token, etc.) to a user, which uniquely identifies one or more accounts associated with the user. The account identifier may be embodied on a physical financial instrument (e.g., a portable financial instrument, a payment card, a credit card, a debit card, etc.), and / or may be electronic information transmitted to a user so that the user can use it for electronic payments. In some non-limiting embodiments or aspects, the account identifier may be an original account identifier, wherein the original account identifier is provided to the user when the account associated with the account identifier is created. In some non-limiting embodiments or aspects, the account identifier may be an account identifier provided to the user after the original account identifier is provided to the user (e.g., a supplemental account identifier). For example, if the original account identifier is forgotten, stolen, etc., the supplemental account identifier may be provided to the user. In some non-limiting embodiments or aspects, the account identifier may be directly or indirectly associated with an issuing institution such that the account identifier may be a payment token mapped to a PAN or other type of identifier. The account identifier may be any combination of alphanumeric characters, characters, and / or symbols, etc. The issuing institution may be associated with a bank identification number (BIN) that uniquely identifies the issuing institution.

[0061] As used herein, the term "payment token" or "token" may refer to an identifier used as a substitute or replacement identifier for an account identifier such as a PAN. A token may be associated with a PAN or other account identifier in one or more data structures (e.g., one or more databases, etc.) such that the token can be used to conduct transactions (e.g., payment transactions) without directly using an account identifier such as a PAN. In some examples, an account identifier such as a PAN may be associated with multiple tokens for different individuals, different uses, and / or different purposes. For example, a payment token may include a series of numbers and / or alphanumeric characters that may be used as a substitute for the original account identifier. For example, a payment token "4900 0000 00000001" may be used in place of a PAN "4147 0900 0000 1234". In some non-limiting embodiments or aspects, a payment token may be "format-reserved" and may have a digital format consistent with account identifiers used in existing payment processing networks (e.g., an ISO 8583 financial transaction message format). In some non-limiting embodiments or aspects, a payment token may be used in place of a PAN to initiate, authorize, settle, or settle a payment transaction, or to represent an original credential in other systems where the original credential would normally be provided. In some non-limiting embodiments or aspects, a token value may be generated such that recovery of the original PAN or other account identifier from the token value may not be computationally possible (e.g., using a one-way hash or other cryptographic function). In addition, in some non-limiting embodiments or aspects, a token format may be configured to allow an entity receiving a payment token to identify it as a payment token and to recognize the entity that issued the token.

[0062] As used herein, the term "provisioning" may refer to the process of enabling a device to use a resource or service. For example, provisioning may involve enabling a device to use an account to perform a transaction. Additionally or alternatively, provisioning may include adding provisioning data associated with account data (e.g., a payment token representing an account number) to the device.

[0063] As used herein, the term "token requester" may refer to an entity that seeks to implement tokenization in accordance with non-limiting embodiments or aspects of the presently disclosed subject matter. For example, a token requester may initiate a request to tokenize a PAN by submitting a token request message to a token service provider. Additionally or alternatively, once the requester has received a payment token in response to the token request message, the token requester may no longer need to store the PAN associated with the token. In some non-limiting embodiments or aspects, the requester may be an application, device, process, or system configured to perform actions associated with the token. For example, the requester may request to register with a network token system, request token generation, token activation, token deactivation, token exchange, other token lifecycle management related processes, and / or any other token related processes. In some non-limiting embodiments or aspects, the requester may connect to the network token system via any suitable communication network and / or protocol (e.g., using HTTPS, SOAP, and / or XML interfaces, etc.). For example, token requesters may include card-on-file merchants, acquirers, acquirer processors, payment gateways operating on behalf of merchants, payment enablers (e.g., original equipment manufacturers, mobile network operators, etc.), digital wallet providers, issuers, third-party wallet providers, payment processing networks, etc. In some non-limiting embodiments or aspects, token requesters may request tokens for multiple domains and / or channels. Additionally or alternatively, a token service provider within a tokenization ecosystem may uniquely register and identify token requesters. For example, during token requester registration, a token service provider may formally process the token requester's application to participate in the token service system. In some non-limiting embodiments or aspects, a token service provider may collect information about the nature of the requester and the relevant use of the token to verify and formally approve the token requester and establish appropriate domain restriction controls. Additionally or alternatively, a token requester identifier may be assigned to a successfully registered token requester, which may also be entered and maintained within the token library. In some non-limiting embodiments or aspects, the token requester identifier may be revoked, and / or a new token requester identifier may be assigned to the token requester. In some non-limiting embodiments or aspects, this information may be reported and audited by the token service provider.

[0064] As used herein, the term "token service provider" may refer to an entity including one or more server computers that generate, process, and maintain payment tokens in a token service system. For example, a token service provider may include a token library that stores generated tokens or communicates with the token library. Additionally or alternatively, the token library may maintain a one-to-one mapping between a token and a PAN represented by the token. In some non-limiting embodiments or aspects, the token service provider is able to reserve an authorized BIN as a token BIN to issue a token that can be submitted to the PAN of the token service provider. In some non-limiting embodiments or aspects, various entities of the tokenization ecosystem may assume the role of a token service provider. For example, a payment network and an issuer or its agent may become a token service provider by implementing a token service according to a non-limiting embodiment or aspect of the subject matter currently disclosed. Additionally or alternatively, the token service provider may provide reports or data outputs to reporting tools regarding approved, pending, or rejected token requests, including any assigned token requester IDs. The token service provider may provide data outputs associated with token-based transactions to reporting tools and applications, and present tokens and / or PANs in report outputs as needed. In some non-limiting embodiments or aspects, the EMVCo standards organization may publish specifications that define how a tokenization system may operate. For example, such specifications may be informational, but they are not intended to limit any currently disclosed subject matter.

[0065] As used herein, the term "token repository" may refer to a repository that maintains established token-to-PAN mappings. For example, the token repository may also maintain other attributes of the token requester that may be determined at registration, and / or may be used by the token service provider to apply domain restrictions or other controls during transaction processing. In some non-limiting embodiments or aspects, the token repository may be part of a token service system. For example, the token repository may be provided as part of a token service provider. Additionally or alternatively, the token repository may be a remote repository accessible to the token service provider. In some non-limiting embodiments or aspects, the token repository may be protected by strong underlying physical and logical security due to the sensitive nature of the data mappings stored and managed therein. Additionally or alternatively, the token repository may be operated by any suitable entity, including a payment network, an issuer, a clearing house, other financial institutions, a transaction service provider, and the like.

[0066] As used herein, the term "merchant" may refer to one or more entities (e.g., operators of retail businesses that provide goods and / or services and / or access to goods and / or services to users (e.g., customers, consumers, customers of merchants, etc.) based on transactions (e.g., payment transactions). As used herein, the term "merchant system" may refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer that executes one or more software applications. As used herein, the term "product" may refer to one or more goods and / or services provided by a merchant.

[0067] As used herein, a "point of sale (POS) device" may refer to one or more devices that may be used by a merchant to initiate a transaction (e.g., a payment transaction), participate in a transaction, and / or process a transaction. For example, a POS device may include one or more computers, peripheral devices, card readers, near field communication (NFC) receivers, radio frequency identification (RFID) receivers and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, computers, servers, input devices, etc.

[0068] As used herein, a "point of sale (POS) system" may refer to one or more computers and / or peripheral devices that a merchant uses to conduct transactions. For example, a POS system may include one or more POS devices, and / or other similar devices that can be used to conduct payment transactions. A POS system (e.g., a merchant POS system) may also include one or more server computers that are programmed or configured to process online payment transactions via a web page, mobile application, or the like.

[0069] As used herein, the term "transaction service provider" may refer to an entity that receives transaction authorization requests from merchants or other entities and, in some cases, provides payment assurance through an agreement between the transaction service provider and an issuer organization. In some non-limiting embodiments or aspects, the transaction service provider may include a credit card company, a debit card company, etc. As used herein, the term "transaction service provider system" may also refer to one or more computer systems operated by or on behalf of the transaction service provider, such as a transaction processing server that executes one or more software applications. The transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of the transaction service provider.

[0070] As used herein, the term "acquirer" may refer to an entity that is licensed by a transaction service provider and approved by the transaction service provider to initiate a transaction (e.g., a payment transaction) using a portable financial device associated with the transaction service provider. As used herein, the term "acquirer system" may also refer to one or more computer systems, computer devices, etc. operated by or on behalf of an acquirer. Transactions may include payment transactions (e.g., purchases, original credit transactions (OCTs), account funding transactions (AFTs), etc.). In some non-limiting embodiments or aspects, an acquirer may be authorized by a transaction service provider to sign a contract with a merchant or service provider to initiate a transaction using a portable financial device of a transaction service provider. An acquirer may sign a contract with a payment service provider to enable the payment service provider to provide initiation to a merchant. An acquirer may monitor the compliance of a payment service provider according to transaction service provider regulations. An acquirer may perform due diligence on a payment service provider and ensure that appropriate due diligence is performed before signing a contract with an initiated merchant. An acquirer may be responsible for all transaction service provider plans operated or initiated by an acquirer. The acquirer may be responsible for the actions of the acquirer payment service provider, merchants initiated by the acquirer payment service provider, etc. In some non-limiting embodiments or aspects, the acquirer may be a financial institution, such as a bank.

[0071] As used herein, the terms "electronic wallet," "electronic wallet mobile application," and "digital wallet" may refer to one or more electronic devices and / or one or more software applications configured to initiate and / or conduct transactions (e.g., payment transactions, electronic payment transactions, etc.). For example, an electronic wallet may include a user device (e.g., a mobile device) that executes an application and server-side software and / or a database for maintaining transaction data and providing the transaction data to the user device. As used herein, the term "electronic wallet provider" may include an entity that provides and / or maintains an electronic wallet and / or electronic wallet mobile application for a user (e.g., a customer). Examples of electronic wallet providers include, but are not limited to and In some non-limiting examples, a financial institution (e.g., an issuer institution) may be an e-wallet provider. As used herein, the term "e-wallet provider system" may refer to one or more computer systems, computer devices, servers, server groups, etc. operated by or on behalf of an e-wallet provider.

[0072] As used herein, the term "portable financial device" may refer to a payment device, an electronic payment device, a payment card (e.g., a credit or debit card), a gift card, a smart card, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a key chain device or fob, an RFID transponder, a retailer discount or membership card, a cellular telephone, an electronic wallet mobile application, a personal digital assistant (PDA), a pager, a security card, a computer, an access card, a wireless terminal, a transponder, etc. In some non-limiting embodiments or aspects, a portable financial device may include volatile or non-volatile memory to store information (e.g., an account identifier, the name of an account holder, etc.).

[0073] As used herein, the term "payment gateway" may refer to an entity and / or a payment processing system operated by or on behalf of such an entity, which provides payment services (e.g., transaction service provider payment services, payment processing services, etc.) to one or more merchants (e.g., transaction service provider payment services, etc.). The payment service may be associated with the use of a portable financial device managed by a transaction service provider. As used herein, the term "payment gateway system" may refer to one or more computer systems, computer devices, servers, server groups, etc. operated by or on behalf of a payment gateway, and / or the payment gateway itself. As used herein, the term "payment gateway mobile application" may refer to one or more electronic devices and / or one or more software applications configured to provide payment services for a transaction (e.g., a payment transaction, an electronic payment transaction, etc.).

[0074] As used herein, the terms "client" and "client device" may refer to one or more client-side devices or systems (e.g., at a remote location of a transaction service provider) used to initiate or facilitate a transaction (e.g., a payment transaction). As an example, a "client device" may refer to one or more POS devices used by a merchant, one or more acquirer host computers used by an acquirer, one or more mobile devices used by a user, and the like. In some non-limiting embodiments or aspects, a client device may be an electronic device configured to communicate with one or more networks and initiate or facilitate a transaction. For example, a client device may include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular phones, wearable devices (e.g., watches, glasses, lenses, clothing, etc.), PDAs, and the like. In addition, a "client" may also refer to an entity (e.g., a merchant, an acquirer, etc.) that owns, utilizes, and / or operates a client device for initiating a transaction (e.g., for initiating a transaction with a transaction service provider).

[0075] As used herein, the term "computing device" may refer to one or more electronic devices configured to communicate directly or indirectly with or on one or more networks. The computing device may be a mobile device, a desktop computer, and / or any other similar device. In addition, the term "computer" may refer to any computing device including necessary components for receiving, processing, and outputting data and typically including a display, a processor, a memory, an input device, and a network interface. As used herein, the term "server" may refer to or include one or more processors or computers, storage devices, or similar computer arrangements that operate or facilitate the communication and processing in a network environment such as the Internet, but it should be understood that communication may be facilitated through one or more public or private network environments, and various other arrangements are also possible. In addition, multiple computers (e.g., servers) or other computerized devices (e.g., POS devices) that communicate directly or indirectly in a network environment may constitute a "system" such as a merchant's POS system.

[0076] As used herein, the term "processor" may refer to any type of processing unit, such as a single processor having one or more cores, one or more cores of one or more processors, multiple processors each having one or more cores, and / or other arrangements and combinations of processing units.

[0077] As used herein, the term "system" may refer to one or more computing devices or combinations of computing devices (e.g., processors, servers, client devices, software applications, components of these devices, etc.). As used herein, references to "devices," "servers," "processors," and the like may refer to a previously stated device, server, or processor stated as performing a previous step or function, a different server or processor, and / or a combination of servers and / or processors. For example, as used in the specification and claims, a first server or first processor stated as performing a first step or first function may refer to the same or different server or the same or different processor stated as performing a second step or second function.

[0078] Non-limiting embodiments or aspects of the disclosed subject matter relate to systems, methods, and computer program products for protocol parsing, including but not limited to network security protocol (e.g., application layer protocol) parsing. For example, non-limiting embodiments or aspects of the disclosed subject matter provide a packet capture system to parse lower layer data (e.g., data link layer data, network layer data, and / or transport layer data) from packets and transmit at least a portion of each packet (e.g., payload, etc.) to a first queue (e.g., payload queue), provide a routing system to route portions of packets (e.g., payload) to corresponding second queues (e.g., protocol-based queues), and provide a parsing system including a protocol parser node to parse higher layer data (e.g., session layer data, presentation layer data, or application layer data) from portions of packets (e.g., payload) based on the corresponding protocol of each packet and transmit the higher layer data to a third queue (e.g., metadata queue) to be aggregated with the lower layer data. Such non-limiting embodiments or aspects provide techniques and systems that enable packet capture / lower layer analyzers to be decoupled from protocol parsers. In this way, relatively low overhead (e.g., computing resources, network resources, etc.) may be involved in building, expanding, updating, and / or deploying analyzers and / or parsers. This can allow for faster and / or continuous building, expansion, updating, and / or deployment of analyzers and / or parsers. Additionally or alternatively, such non-limiting embodiments or aspects provide techniques and systems that provide for deploying packet capture and / or lower layer analyzers on one server and providing protocol parsers on other (separate) servers. Additionally or alternatively, such non-limiting embodiments or aspects provide techniques and systems that enable aggregation of higher layer data from multiple different protocol parsers (e.g., for different application layer protocols) and lower layer data from separate lower layer analyzers via a third message queue. Additionally or alternatively, such non-limiting embodiments or aspects provide techniques and systems that can achieve scaling (e.g., deploying additional packet capture system nodes, routing system nodes, parsing system nodes, etc.; removing underutilized nodes; any combination thereof, etc.) relatively easily because the nodes of each system are independent (e.g., decoupled). For example, utilization of resources (e.g., computing resources, network resources, data rates (e.g., number of incoming packets per second, number of outgoing packets per second, etc.), packet drops, etc.) of each system and / or its nodes may be monitored (e.g., by at least one resource monitoring system and / or its agent) and new nodes may be provisioned accordingly (e.g., by a node provisioning system, etc.). In this way, packet drops may be reduced and / or parsing / aggregation of data may be accelerated.Additionally or alternatively, such non-limiting embodiments or aspects provide techniques and systems that enable custom fields (e.g., protocol specific fields, vendor specific fields, etc.) from different protocol parsers to be converted into common fields (e.g., when aggregating data from a third queue). Additionally or alternatively, such non-limiting embodiments or aspects provide techniques and systems that enable filtering of specific fields (e.g., protocol specific fields, vendor specific fields, etc.) from different protocol parsers (e.g., when aggregating data from a third queue), which can allow more efficient use of memory / storage devices because unnecessary fields are eliminated. In this way, enrichment of data from the third queue can be achieved. Additionally or alternatively, such non-limiting embodiments or aspects provide techniques and systems that enable faster threat (e.g., intrusion) detection because more data from different protocol parsers and lower-layer analyzers can be aggregated faster. In this way, the time to detect, investigate, and remediate threats can be reduced.

[0079] For purposes of illustration, in the following description, although the presently disclosed subject matter is described with respect to methods, systems, and computer program products for protocol parsing for, for example, network security, one skilled in the art will recognize that the disclosed subject matter is not limited to the illustrative embodiments or aspects. For example, the methods, systems, and computer program products described herein can be used with various settings, such as being suitable for protocol parsing in any setting using a variety of different (application layer) protocols, such as cloud computing, software as a service (SaaS), etc.

[0080] Reference now Figure 1 , Figure 1 1 is a diagram of a non-limiting embodiment or aspect of an environment 100 in which systems, products, and / or methods as described herein may be implemented. Figure 1 As shown, environment 100 includes transaction service provider system 102 , issuer system 104 , client device 106 , merchant system 108 , acquirer system 110 , and communication network 112 .

[0081] The transaction service provider system 102 may include one or more devices capable of receiving information from and / or transmitting information to the issuer system 104, the client device 106, the merchant system 108, and / or the acquirer system 110 via the communication network 112. For example, the transaction service provider system 102 may include a computing device, such as a server (e.g., a transaction processing server), a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the transaction service provider system 102 may be associated with a transaction service provider described herein. In some non-limiting embodiments or aspects, the transaction service provider system 102 may communicate with a data storage device, which may be local or remote to the transaction service provider system 102. In some non-limiting embodiments or aspects, the transaction service provider system 102 is capable of receiving information from the data storage device, storing information in the data storage device, transmitting information to the data storage device, or searching for information stored in the data storage device.

[0082] The issuer system 104 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 102, the client device 106, the merchant system 108, and / or the acquirer system 110 via the communication network 112. For example, the issuer system 104 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the issuer system 104 may be associated with an issuer organization described herein. For example, the issuer system 104 may be associated with an issuer organization that issues credit accounts, debit accounts, credit cards, debit cards, and / or the like to users associated with the client device 106.

[0083] The client devices 106 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 102, the issuer system 104, the merchant system 108, and / or the acquirer system 110 via the communication network 112. Additionally or alternatively, each client device 106 may include a device capable of receiving information from and / or transmitting information to other client devices 106 via the communication network 112, another network (e.g., a temporary network, a local network, a private network, a virtual private network, etc.), and / or any other suitable communication technology. For example, the client devices 106 may include client devices, etc. In some non-limiting embodiments or aspects, the client devices 106 may or may not be capable of communicating via a short-range wireless communication connection (e.g., an NFC communication connection, an RFID communication connection, Communication connection, communication connection, etc.) (e.g., from merchant system 108 or from another client device 106), and / or transmit information (e.g., to merchant system 108) via a short-range wireless communication connection.

[0084] The merchant system 108 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 102, the issuer system 104, the client device 106, and / or the acquirer system 110 via the communication network 112. The merchant system 108 may also include a communication connection (e.g., an NFC communication connection, an RFID communication connection, a NFC communication connection, a ... Communication connection, In some non-limiting embodiments or aspects, the merchant system 108 may include a computing device, such as a server, a server group, a client device, a client device group, and / or other similar devices. In some non-limiting embodiments or aspects, the merchant system 108 may be associated with the merchant described herein. In some non-limiting embodiments or aspects, the merchant system 108 may include one or more client devices. For example, the merchant system 108 may include a client device that allows the merchant to transmit information to the transaction service provider system 102. In some non-limiting embodiments or aspects, the merchant system 108 may include one or more devices that can be used by the merchant to conduct payment transactions with users, such as computers, computer systems, and / or peripheral devices. For example, the merchant system 108 may include a POS device and / or a POS system.

[0085] The acquirer system 110 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 102, the issuer system 104, the client device 106, and / or the merchant system 108 via the communication network 112. For example, the acquirer system 110 may include a computing device, a server, a server group, etc. In some non-limiting embodiments or aspects, the acquirer system 110 may be associated with an acquirer as described herein.

[0086] The communication network 112 may include one or more wired and / or wireless networks. For example, the communication network 112 may include a cellular network (e.g., a long term evolution (LTE) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network (e.g., a private network associated with a transaction service provider), an ad hoc network, an intranet, the Internet, a fiber-based network, a cloud computing network, etc., and / or a combination of these or other types of networks.

[0087] In some non-limiting embodiments or aspects, processing a transaction may include generating and / or transmitting at least one transaction message (e.g., an authorization request, an authorization response, any combination thereof, etc.). For example, a client device (e.g., a client device 106, a POS device of a merchant system 108, etc.) may initiate a transaction, for example, by generating an authorization request. Additionally or alternatively, a client device (e.g., a client device 106, at least one device of a merchant system 108, etc.) may transmit an authorization request. For example, the client device 106 may transmit the authorization request to a merchant system 108 and / or a payment gateway (e.g., a payment gateway of a transaction service provider system 102, a third-party payment gateway separated from the transaction service provider system 102, etc.). Additionally or alternatively, the merchant system 108 (e.g., its POS device) may transmit the authorization request to an acquirer system 110 and / or a payment gateway. In some non-limiting embodiments or aspects, the acquirer system 110 and / or the payment gateway may transmit the authorization request to the transaction service provider system 102 and / or the issuer system 104. Additionally or alternatively, the transaction service provider system 102 may transmit the authorization request to the issuer system 104. In some non-limiting embodiments or aspects, the issuer system 104 may determine an authorization decision (e.g., authorization, rejection, etc.) based on the authorization request. For example, the authorization request may cause the issuer system 104 to determine the authorization decision based on the authorization request. In some non-limiting embodiments or aspects, the issuer system 104 may generate an authorization response based on the authorization decision. Additionally or alternatively, the issuer system 104 may transmit the authorization response. For example, the issuer system 104 may transmit the authorization response to the transaction service provider system 102 and / or the payment gateway. Additionally or alternatively, the transaction service provider system 102 and / or the payment gateway may transmit the authorization response to the acquirer system 110, the merchant system 108, and / or the client device 106. Additionally or alternatively, the acquirer system 110 may transmit the authorization response to the merchant system 108 and / or the payment gateway. Additionally or alternatively, the payment gateway may transmit the authorization response to the merchant system 108 and / or the client device 106. Additionally or alternatively, the merchant system 108 may transmit the authorization response to the client device 106. In some non-limiting embodiments or aspects, the merchant system 108 may receive the authorization response (e.g., from the acquirer system 110 and / or the payment gateway). Additionally or alternatively, the merchant system 108 may complete the transaction based on the authorization response (e.g., provide, ship and / or deliver goods and / or services associated with the transaction; fulfill an order associated with the transaction; any combination thereof; etc.).

[0088] For purposes of illustration, processing a transaction may include generating a transaction message (e.g., an authorization request, etc.) based on an account identifier of a customer (e.g., associated with a customer device 106, etc.) and / or transaction data associated with the transaction. For example, a merchant system 108 (e.g., a client device of the merchant system 108, a POS device of the merchant system 108, etc.) may initiate a transaction, for example, by generating an authorization request (e.g., in response to receiving an account identifier from a portable financial device, etc. of the customer). Additionally or alternatively, the merchant system 108 may transmit the authorization request to an acquirer system 110. Additionally or alternatively, the acquirer system 110 may transmit the authorization request to the transaction service provider system 102. Additionally or alternatively, the transaction service provider system 102 may transmit the authorization request to the issuer system 104. The issuer system 104 may determine an authorization decision (e.g., authorization, rejection, etc.) based on the authorization request, and / or the issuer system 104 may generate an authorization response based on the authorization decision and / or the authorization request. Additionally or alternatively, the issuer system 104 may transmit the authorization response to the transaction service provider system 102. Additionally or alternatively, the transaction service provider system 102 may communicate the authorization response to the acquirer system 110 , which may communicate the authorization response to the merchant system 108 .

[0089] For purposes of illustration, clearing and / or settlement of a transaction may include generating a message (e.g., a clearing message, a settlement message, etc.) based on an account identifier of a customer (e.g., associated with a customer device 106, etc.) and / or transaction data associated with the transaction. For example, the merchant system 108 may generate at least one clearing message (e.g., a plurality of clearing messages, a batch of clearing messages, etc.). Additionally or alternatively, the merchant system 108 may transmit the clearing message to the acquirer system 110. Additionally or alternatively, the acquirer system 110 may transmit the clearing message to the transaction service provider system 102. Additionally or alternatively, the transaction service provider system 102 may transmit the clearing message to the issuer system 104. Additionally or alternatively, the issuer system 104 may generate at least one settlement message based on the clearing message. Additionally or alternatively, the issuer system 104 may transmit the settlement message and / or funds to the transaction service provider system 102 (and / or a settlement bank system associated with the transaction service provider system 102). Additionally or alternatively, the transaction service provider system 102 (and / or the settlement bank system) may transmit the settlement message and / or funds to the acquirer system 110, which may transmit the settlement message and / or funds to the merchant system 108 (and / or an account associated with the merchant system 108).

[0090] Figure 1The number and arrangement of systems, devices, and / or networks shown are provided as examples. Additional systems, devices, and / or networks, fewer systems, devices, and / or networks, different systems, devices, and / or networks, and / or in a similar manner may exist. Figure 1 The systems, devices and / or networks shown in the figure may be arranged in different ways. In addition, the system and / or device may be implemented in a single system and / or device. Figure 1 Two or more systems or devices shown in Figure 1 A single system or device shown in the environment 100 may be implemented as multiple distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) and / or a set of devices (e.g., one or more devices) of the environment 100 may perform one or more functions described as being performed by another set of systems or another set of devices of the environment 100.

[0091] Reference now Figure 2 , Figure 2 2 is a diagram of example components of an apparatus 200. The apparatus 200 may correspond to one or more apparatuses of the transaction service provider system 102, one or more apparatuses of the issuer system 104, the client device 106, one or more apparatuses of the merchant system 108, and / or one or more apparatuses of the acquirer system 110. In some non-limiting embodiments or aspects, the transaction service provider system 102, the issuer system 104, the client device 106, the merchant system 108, and / or the acquirer system 110 may include at least one apparatus 200 and / or at least one component of the apparatus 200. Figure 2 As shown, apparatus 200 may include a bus 202 , a processor 204 , a memory 206 , a storage component 208 , an input component 210 , an output component 212 , and a communication interface 214 .

[0092] The bus 202 may include components that permit communication between components of the device 200. In some non-limiting embodiments or aspects, the processor 204 may be implemented in hardware, software, firmware, and / or any combination thereof. For example, the processor 204 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component that can be programmed to perform a function (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), etc. The memory 206 may include a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by the processor 204.

[0093] The storage component 208 may store information and / or software associated with the operation and use of the device 200. For example, the storage component 208 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, a solid-state disk, etc.), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of computer-readable medium, and a corresponding drive.

[0094] Input components 210 may include components that permit device 200 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, a microphone, a camera, etc.). Additionally or alternatively, input components 210 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output components 212 may include components that provide output information from device 200 (e.g., a display, a speaker, one or more light emitting diodes (LEDs), etc.).

[0095] The communication interface 214 may include a transceiver-type component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables the device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 214 may permit the device 200 to receive information from another device and / or provide information to another device. For example, the communication interface 214 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, interface, interface, Interface, cellular network interface, etc.

[0096] The device 200 may perform one or more processes described herein. The device 200 may perform these processes based on the processor 204 executing software instructions stored by a computer-readable medium such as the memory 206 and / or the storage component 208. A computer-readable medium (e.g., a non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A non-transitory memory device includes a memory space located within a single physical storage device or a memory space spread across multiple physical storage devices.

[0097] The software instructions may be read into the memory 206 and / or storage component 208 from another computer-readable medium or from another device via the communication interface 214. When executed, the software instructions stored in the memory 206 and / or storage component 208 may cause the processor 204 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, the embodiments or aspects described herein are not limited to any specific combination of hardware circuitry and software.

[0098] Figure 2 The number and arrangement of components shown are provided as examples. In some non-limiting embodiments or aspects, the device 200 may include additional components, fewer components, different components, or components in a manner similar to that of the embodiment of the present invention. Figure 2 Additionally or alternatively, one set of components (eg, one or more components) of the apparatus 200 may perform one or more functions described as being performed by another set of components of the apparatus 200.

[0099] Reference now Figure 3 , Figure 3 is a flow chart of a non-limiting embodiment or aspect of a process 300 of protocol parsing according to the principles of the present disclosure. In some non-limiting embodiments or aspects, one or more steps of the process 300 may be performed (e.g., completely, partially, etc.) by the transaction service provider system 102 (e.g., one or more devices of the transaction service provider system 102). In some non-limiting embodiments or aspects, one or more steps of the process 300 may be performed (e.g., completely, partially, etc.) by another system, another device, another group of systems or another group of devices (e.g., issuer system 104, customer device 106, merchant system 108, acquirer system 110, etc.) that is separate from or includes the transaction service provider system 102. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the packet capture system may include and / or be a portion of at least one computing device, the at least one computing device being, for example, at least one server (e.g., the first server) or a partition thereof. Additionally or alternatively, the packet capture system may be the same or similar to the packet capture system 402. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the first queue may include and / or be a part of at least one computing device, the at least one computing device being, for example, a message queue (e.g., Kafka, etc.), a first-in-first-out (FIFO) queue, a priority queue, at least one server or partition thereof, a database, any combination thereof, etc. Additionally or alternatively, the first queue may be the same or similar to the first queue 404. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the routing system may include and / or be a portion of at least one computing device, the at least one computing device being, for example, at least one server (e.g., the second server) or a partition thereof. Additionally or alternatively, the routing system may be the same or similar to routing system 406. In some non-limiting embodiments or aspects, reference is made to Figure 3, the second queue may include and / or be a part of at least one computing device, such as a message queue, a FIFO queue, a priority queue, at least one server or partition thereof, a database, any combination thereof, etc. Additionally or alternatively, the second queue may be the same or similar to the second queue 408. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the resolution system may include and / or be a portion of at least one computing device, the at least one computing device being, for example, at least one server (e.g., a third server) or a partition thereof. Additionally or alternatively, the resolution system may be the same or similar to the resolution system 410. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the third queue may include and / or be a part of at least one computing device, the at least one computing device being, for example, a message queue, a FIFO queue, a priority queue, at least one server or partition thereof, a database, any combination thereof, etc. Additionally or alternatively, the third queue may be the same or similar to the third queue 412. In some non-limiting embodiments or aspects, reference Figure 3 , the aggregation system may include and / or be a portion of at least one computing device, such as at least one server (e.g., the fourth server) or a partition thereof. Additionally or alternatively, the aggregation system may be the same or similar to aggregation system 414. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the database may include and / or be a portion of at least one computing device, the at least one computing device being, for example, at least one server or a partition thereof, a database (e.g., a structured query language (SQL) database, a non-SQL database, a relational database, a non-relational database, a distributed database, etc.), any combination thereof, etc. Additionally or alternatively, the database may be the same or similar to database 416. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the resource monitoring system may include and / or be a portion of at least one computing device, the at least one computing device being, for example, at least one server (e.g., the fifth server) or a partition thereof. Additionally or alternatively, the resource monitoring system may be the same or similar to resource monitoring system 418. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the node provisioning system may include and / or be a part of at least one computing device, the at least one computing device being, for example, at least one server (e.g., the sixth server) or a partition thereof. Additionally or alternatively, the node provisioning system may be the same or similar to the node provisioning system 420. In some non-limiting embodiments or aspects, reference Figure 3, the mapping database may include and / or be a portion of at least one computing device, the at least one computing device being, for example, at least one server or a partition thereof, a database (e.g., a SQL database, a non-SQL database, a relational database, a non-relational database, a distributed database, etc.), any combination thereof, etc. Additionally or alternatively, the mapping database may be the same or similar to the mapping database 422. In some non-limiting embodiments or aspects, reference is made to Figure 3 , the policy database may include and / or be a portion of at least one computing device, such as at least one server or a partition thereof, a database (e.g., a SQL database, a non-SQL database, a relational database, a non-relational database, a distributed database, etc.), any combination thereof, etc. Additionally or alternatively, the policy database may be the same or similar to policy database 424.

[0100] like Figure 3 As shown, process 300 may include receiving packets at step 302. For example, a packet capture system may receive a plurality of packets (eg, from a communication network, such as communication network 112, etc., and / or a network tap connected thereto).

[0101] In some non-limiting embodiments or aspects, the packet capture system may include at least one first server. Additionally or alternatively, each of the plurality of first servers may include at least one of a packet capture subsystem, a lower layer parser subsystem, a protocol identifier subsystem, any combination thereof, and the like. For example, each first server may include a packet capture subsystem. In some non-limiting embodiments or aspects, receiving the plurality of packets may include receiving the plurality of packets by the packet capture subsystem (e.g., from a communication network, such as communication network 112, etc., and / or a network tap connected thereto).

[0102] like Figure 3 As shown in , at step 304, process 300 may include parsing lower layer data (e.g., data link layer data, network layer data, and / or transport layer data) from each packet. For example, the packet capture system may parse at least one of data link layer data, network layer data, transport layer data, any combination thereof, etc. from each of the plurality of packets. In some non-limiting embodiments or aspects, parsing the lower layer data may include parsing a header of each packet to identify at least one field (e.g., at least one link layer data field, at least one network layer data field, and / or at least one transport layer data field) and / or extracting corresponding data from each field (e.g., the identified field).

[0103] In some non-limiting embodiments or aspects, the packet capture system may include at least one first server. Additionally or alternatively, each of the plurality of first servers may include at least one of a packet capture subsystem, a lower layer parser subsystem, a protocol identifier subsystem, any combination thereof, and the like. For example, each first server may include a lower layer parser subsystem. In some non-limiting embodiments or aspects, parsing lower layer data from each packet may include parsing lower layer data (e.g., at least one of data link layer data, network layer data, transport layer data, any combination thereof, and the like) from each of the plurality of packets by the lower layer parser subsystem. In some non-limiting embodiments, the packet capture subsystem may transmit the packet to the lower layer parser subsystem (e.g., before the lower layer parser subsystem parses each packet).

[0104] like Figure 3 As shown, at step 306, process 300 may include forwarding at least a portion of each packet to a first queue. For example, the packet capture system may transmit at least a portion of each respective packet (e.g., a respective payload, at least a portion of a respective header, any combination thereof, etc.) to at least one first queue. In some non-limiting embodiments or aspects, the packet capture system may determine a respective protocol (e.g., an application layer protocol) for each respective packet and / or transmit the respective packet and / or a portion thereof (e.g., a payload) to the first queue along with protocol data associated with the respective protocol of the respective packet. For example, determining the respective protocol of each packet may include (e.g., by the packet capture system and / or its protocol identifier subsystem) analyzing each respective packet (e.g., at least a portion of a respective payload, at least a portion of a respective header, any combination thereof, etc.) to identify the protocol (e.g., an application layer protocol, etc.) of the respective packet. For purposes of illustration, exemplary application layer protocols may include Hypertext Transfer Protocol (HTTP), Secure Hypertext Transfer Protocol (HTTPS), Simple Mail Transfer Protocol (SMTP), Domain Name System (DNS), Server Message Block (SMB), Internet Message Access Protocol (IMAP), Post Office Protocol (POP), Real-time Transport Protocol (RTP), etc.

[0105] In some non-limiting embodiments or aspects, the packet capture system may include at least one first server. Additionally or alternatively, each of the plurality of first servers may include at least one of a packet capture subsystem, a lower layer parser subsystem, a protocol identifier subsystem, any combination thereof, and the like. For example, each first server may include a protocol identifier subsystem. In some non-limiting embodiments or aspects, transmitting each corresponding packet and / or a portion thereof (e.g., a payload) to the first queue may include determining, by the protocol identifier subsystem, a corresponding protocol (e.g., an application layer protocol) of each corresponding packet, and / or transmitting, by the packet capture system, the corresponding packet and / or a portion thereof (e.g., a payload) to the first queue together with protocol data associated with the corresponding protocol of the corresponding packet. In some non-limiting embodiments, the packet capture subsystem and / or the lower layer parser subsystem may transmit the packet to the protocol identifier subsystem (e.g., before the protocol identifier subsystem transmits each packet and / or a portion thereof).

[0106] In some non-limiting embodiments or aspects, a packet capture system (e.g., a protocol identifier subsystem thereof) may generate a corresponding packet identifier for each respective packet in the plurality of packets. Additionally or alternatively, transmitting the respective packet and / or a portion thereof (e.g., a payload) to the first queue may include transmitting the respective packet and / or a portion thereof (e.g., a payload) together with the respective packet identifier of the respective packet. In some non-limiting embodiments or aspects, each respective packet identifier may be generated (e.g., by a packet capture system and / or a protocol identifier subsystem thereof) based on a timestamp, a source IP address (e.g., src_ip), a destination IP address (e.g., dst_ip), a source port (e.g., src_port), a destination port (e.g., dst_port), any combination thereof, etc. of the respective packet. For example, a flow hash may be generated (e.g., by a packet capture system and / or a protocol identifier subsystem thereof) based on a hash of the source IP address (e.g., src_ip), the destination IP address (e.g., dst_ip), the source port (e.g., src_port), and the destination port (e.g., dst_port) of each packet. In this way, multiple packets in the same packet flow may include the same flow hash. Additionally or alternatively, the timestamp of each packet may uniquely identify the corresponding packet relative to other packets in the same flow. In some non-limiting embodiments, the packet identifier may include at least one (e.g., both) of the flow hash and / or the timestamp.

[0107] In some non-limiting embodiments, the protocol identifier subsystem may store the payload of each packet in the first queue as part of a tuple (e.g., a four-tuple of protocol, flow_hash, payload_size, and payload, etc.). In some non-limiting embodiments or aspects, the payload may include binary data. For example, unlike strings / text, such binary data may not be NULL-terminated. Additionally or alternatively, in the first queue, each payload may be stored in a continuous memory. In this way, the routing system may use the payload size of the corresponding payload (e.g., payload_size) to copy the corresponding payload from the memory until the end of the payload is reached (as indicated by the payload size) to route to the corresponding second queue based on the protocol field of the tuple.

[0108] In some non-limiting embodiments, the protocol identifier subsystem may use the first packet of a flow (e.g., the first packet of a session) to identify the protocol type of an application payload. For example, communications between two parties may be fragmented and sent in multiple packets over a communications network, and / or the first packet of a session (e.g., an established connection between two peers) may have data associated with an application layer protocol. Thus, to determine the protocol of the packets of the session, the protocol identifier subsystem may generate a flow hash, as described herein. Additionally or alternatively, the protocol identifier subsystem may maintain a mapping of flow hashes to corresponding protocols associated therewith. Thus, the flow hashes of the packets of the session should remain the same, and / or the protocol identifier subsystem may determine the application layer protocol of the fragmented payload of the session based on the flow hash.

[0109] In some non-limiting embodiments or aspects, the routing system may subscribe to a topic of the first queue. For example, each respective topic may be associated with a respective protocol in a plurality of protocols (e.g., possible protocols, supported protocols, etc.). In some non-limiting embodiments or aspects, the plurality of protocols may include all protocols (e.g., application layer protocols) associated with at least one of the protocol parser nodes of the parsing system, as described herein. In some non-limiting embodiments or aspects, transmitting each respective packet and / or a portion thereof (e.g., a payload) to the first queue may include (e.g., by a packet capture system and / or its protocol identifier subsystem) publishing the respective packet and / or a portion thereof (e.g., a payload) in the first queue together with the respective topic based on the respective protocol of each respective packet.

[0110] like Figure 3As shown, at step 308, process 300 may include routing each packet to a corresponding second queue based on the corresponding protocol of the packet. For example, the routing system may route each corresponding packet and / or a portion thereof (e.g., a payload) to a corresponding second queue in the plurality of second queues based on the corresponding protocol of the packet (e.g., an application layer protocol).

[0111] In some non-limiting embodiments or aspects, the routing system may include at least one second server.

[0112] In some non-limiting embodiments or aspects, each respective protocol parser node of the parsing system may be associated with a respective protocol (e.g., a respective application layer protocol) of the plurality of protocols. Additionally or alternatively, each respective second queue of the plurality of second queues may be associated with a respective protocol parser node of the parsing system.

[0113] In some non-limiting embodiments or aspects, routing each respective packet and / or portion thereof (e.g., payload) to a respective second queue may include routing the respective packet and / or portion thereof (e.g., payload) to a selected second queue based on protocol data (e.g., subject) of each respective packet corresponding to a respective protocol of a respective protocol parser node associated with a selected second queue among a plurality of second queues.

[0114] In some non-limiting embodiments or aspects, the routing system can subscribe to the topic of the first queue, as described herein. Additionally or alternatively, routing each corresponding packet and / or portion thereof (e.g., payload) can include the routing system consuming each corresponding packet and / or portion thereof (e.g., payload) from the first queue.

[0115] In some non-limiting embodiments or aspects, routing each respective packet and / or portion thereof (e.g., payload) may include (e.g., by a routing system) publishing the respective packet and / or portion thereof (e.g., payload) in a respective second queue based on a respective subject (e.g., protocol) of each respective packet. Additionally or alternatively, each protocol parser node of the parsing system may subscribe to at least one (e.g., a respective one) of the second queues.

[0116] In some non-limiting embodiments or aspects, the routing system may subscribe to multiple topics (e.g., protocols) in the first queue. For example, the routing system may retrieve a payload tuple (e.g., a four-tuple of protocol, flow_hash, payload_size, and payload) from the first queue and / or route the payload to the corresponding second queue based on the protocol field of the tuple. Additionally or alternatively, the routing system may store the flow in the corresponding second queue as a tuple (e.g., a three-tuple of flow_hash, payload_size, and payload). In some non-limiting embodiments or aspects, the payload may include binary data. For example, unlike a string / text, such binary data may not be terminated with NULL. Additionally or alternatively, in the corresponding second queue, each payload may be stored in a continuous memory. In this way, the parsing system (e.g., a corresponding protocol parser node) may use the payload size (e.g., payload_size) of the corresponding payload to copy the corresponding payload from the memory until the end of the payload is reached (as indicated by the payload size) for processing by the corresponding protocol parser node.

[0117] like Figure 3 As shown, at step 310, process 300 may include parsing higher layer data (e.g., session layer data, presentation layer data, and / or application layer data). For example, a respective protocol parser node of the parsing system may parse at least one of session layer data, presentation layer data, application layer data, any combination thereof, etc. from each respective packet and / or portion thereof (e.g., payload) from the respective second queue.

[0118] In some non-limiting embodiments or aspects, the parsing system may include multiple third servers. Additionally or alternatively, each respective protocol parser node may include a respective third server and / or a partition thereof.

[0119] In some non-limiting embodiments or aspects, the routing system may publish each respective packet and / or portion thereof (e.g., payload) in a respective second queue, as described herein. Additionally or alternatively, parsing higher layer data (e.g., session layer data, presentation layer data, application layer data, etc.) from each respective packet and / or portion thereof (e.g., payload) may include consuming, by a respective protocol parser node of the parsing system, each respective packet and / or portion thereof (e.g., payload) from a respective second queue of the plurality of second queues.

[0120] In some non-limiting embodiments or aspects, all packets of a corresponding packet flow (e.g., identified based on a flow hash, etc.) may be routed to the same second queue and / or the same protocol parser node. This may ensure that all payloads of packets in the corresponding flow are parsed by the same protocol parser node and / or prevent multiple different protocol parser nodes from parsing packets from the same flow (e.g., preventing redundancy, enabling one protocol parser node to identify patterns based on all payloads of packets in the corresponding flow rather than based on incomplete information of only a subset of packets in the corresponding flow, etc.).

[0121] In some non-limiting embodiments, all packets of a corresponding packet flow (e.g., identified based on a flow hash, etc.) may be routed to each of a plurality of protocol parser nodes (e.g., via a corresponding second queue associated with the corresponding protocol parser node). For example, the protocol parser nodes may include at least one protocol parser node for a first protocol from each of a plurality of vendors. Additionally or alternatively, all packets of a first packet flow of a first protocol may be routed to at least one protocol parser node for the first protocol from each vendor. For purposes of illustration, if there is a first protocol parser node for a first protocol from a first vendor and a second protocol parser node for a first protocol from a second vendor, all packets of the corresponding packet flow may be routed to each of the first protocol parser node and the second protocol parser node.

[0122] like Figure 3 As shown, at step 312, process 300 may include transmitting the parsed data to a third queue. For example, the packet capture system may transmit lower layer data (e.g., data link layer data, network layer data, transport layer data, etc.) of each packet to the third queue. Additionally or alternatively, the parsing system (e.g., its protocol parser node) may transmit higher layer data (e.g., session layer data, presentation layer data, application layer data, etc.) of each packet to the third queue.

[0123] In some non-limiting embodiments or aspects, transmitting the lower layer data of each packet to the third queue may include transmitting the lower layer data of each packet to the third queue together with its corresponding packet identifier. Additionally or alternatively, transmitting the higher layer data of each packet to the third queue may include transmitting the higher layer data of each packet to the third queue together with its corresponding packet identifier.

[0124] like Figure 3As shown, at step 314, process 300 may include aggregating and / or storing the parsed data. For example, the aggregation system may aggregate lower layer data (e.g., data link layer data, network layer data, transport layer data, etc.) for each packet with higher layer data (e.g., session layer data, presentation layer data, application layer data, etc.) for each packet. Additionally or alternatively, the aggregation system may store the aggregated data (e.g., lower layer data aggregated with higher layer data) for each packet in a database.

[0125] In some non-limiting embodiments or aspects, the aggregation system may include at least one fourth server.

[0126] In some non-limiting embodiments or aspects, aggregating lower layer data of each packet with higher layer data may include aggregating the lower layer data with the higher layer data based on a match between corresponding packet identifiers of corresponding packets transmitted with the lower layer data and corresponding packet identifiers of corresponding packets transmitted with the higher layer data.

[0127] In some non-limiting embodiments or aspects, a mapping database may receive (e.g., from a user and / or a computing device of the user) and / or store custom field data associated with a plurality of custom fields (e.g., fields of parsed data from at least one parser node of a parsing system, which may have a custom (e.g., unique, vendor-specific, etc.) format). Additionally or alternatively, the mapping database may generate a respective mapping between each respective custom field in the plurality of custom fields and a respective public field in a plurality of common fields (e.g., fields selected by at least one entity associated with an aggregation system, which may have a common (e.g., selected, preferred, standardized, common, etc.) format). In some non-limiting embodiments or aspects, the aggregation system may convert at least some of the lower layer data and / or higher layer data (e.g., at least one of data link layer data, network layer data, transport layer data, session layer data, presentation layer data, application layer data, any combination thereof, etc.) (e.g., at least one custom field) of at least one packet (e.g., a first packet) based on the respective mapping of the respective custom fields to the respective public fields.

[0128] In some non-limiting embodiments or aspects, a policy database may receive and / or store (e.g., from a user and / or a computing device of the user) aggregation policy data associated with at least one aggregation policy. For example, the aggregation policy may include a policy (e.g., a rule, a script, etc.) to filter (e.g., remove) at least one field from data (e.g., lower layer data, higher layer data, etc.), add at least one field to data (e.g., generate at least one field and / or append at least one field to data), combine at least two fields of data, any combination thereof, etc. In some non-limiting embodiments or aspects, the aggregation system may filter at least one field of at least one packet (e.g., a first packet) and / or data associated therewith (e.g., higher layer data, lower layer data, etc.) based on the aggregation policy (or multiple policies). Additionally or alternatively, the aggregation system may add (e.g., generate and / or append) at least one field to data associated with at least one packet (e.g., a first packet, a second packet, etc.) based on the aggregation policy (or multiple policies). Additionally or alternatively, the aggregation system may combine at least two fields of at least one packet (eg, first packet, second packet, third packet, etc.) and / or data associated therewith based on an aggregation policy (or policies).

[0129] In some non-limiting embodiments or aspects, after aggregating the data, the data may be stored in a database based on an aggregation policy defined in a policy database. Additionally or alternatively, the aggregated data may be used (e.g., accessed, read, analyzed, etc.) by an intrusion detection and / or prevention system. For example, the intrusion detection and / or prevention system determines whether there is at least one anomaly based on the aggregated data stored in the database. In response to determining that there is such anomaly, the intrusion detection and / or prevention system may generate at least one alarm based on at least one security policy in the policy database. In some non-limiting embodiments or aspects, the intrusion detection and / or prevention system may retrieve the aggregated data from the database, and / or receive the data directly from the aggregation system (e.g., the aggregation system may transmit a copy of the aggregated data to the intrusion detection and / or prevention system, and transmit another copy of the aggregated data to the database). In some non-limiting embodiments or aspects, the aggregated data may be retrieved from the database (e.g., in response to input from a user) to investigate a network attack (e.g., by a user).

[0130] In some non-limiting embodiments or aspects, the resource monitoring system may monitor at least one of the packet capture system, the routing system, the resolution system, the aggregation system, the first queue, the second queue, the third queue, any combination thereof, etc. to provide corresponding metrics of each of the aforementioned systems and / or queues (e.g., resource utilization (e.g., processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.), data rate, packet drop rate, enqueue rate (e.g., enqueue operation), dequeue rate (e.g., dequeue operation), queue utilization, any combination thereof, etc.). Additionally or alternatively, the node provisioning system may determine whether to add a node (e.g., a server and / or a partition thereof) to or remove a node from at least one of the packet capture system, the routing system, the resolution system, and / or the aggregation system based on at least one of the corresponding metrics.

[0131] In some non-limiting embodiments or aspects, the resource monitoring system may include a packet capture system monitoring agent. Additionally or alternatively, the node provisioning system may include a packet capture system node provisioning system. In some non-limiting embodiments or aspects, the packet capture system monitoring agent may determine whether to add a node (e.g., an additional first server or an additional partition thereof) to the packet capture system based on a metric of the packet capture system. For example, if resource utilization (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) is too high (e.g., meets (e.g., exceeds) at least one first threshold), the data rate (e.g., packet rate) is too high (e.g., meets (e.g., exceeds) a second threshold), and / or the packet drop rate is too high (e.g., meets (e.g., exceeds) a third threshold), the packet capture system monitoring agent may determine to add a node to the packet capture system. Additionally or alternatively, if resource utilization (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) is too low (e.g., satisfies (e.g., is lower than) at least one fourth threshold), the data rate (e.g., packet rate) is too low (e.g., satisfies (e.g., is lower than) a fifth threshold), and / or the packet drop rate is too low (e.g., satisfies (e.g., is lower than) a sixth threshold and / or is zero), the packet capture system monitoring agent may determine to remove the node from the packet capture system. In some non-limiting embodiments or aspects, the packet capture system node provisioning system may add (e.g., provision, upgrade, etc.) nodes to the capture system, or remove (e.g., downgrade) nodes from the packet capture system, respectively, based on the packet capture system monitoring agent determining whether to add or remove nodes.

[0132] In some non-limiting embodiments or aspects, the resource monitoring system may include a first queue monitoring agent and / or a routing system monitoring agent. Additionally or alternatively, the node provisioning system may include a routing system node provisioning system. In some non-limiting embodiments or aspects, the first queue monitoring agent and / or the routing system monitoring agent may determine whether to add a node (e.g., an additional second server or an additional partition thereof) to the routing system based on a metric of the first queue and / or the routing system. For example, if the resource utilization of the routing system (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) is too high (e.g., meets (e.g., exceeds) at least one first threshold), the data rate (e.g., packet rate) of the routing system is too high (e.g., meets (e.g., exceeds) a second threshold), and / or the packet drop rate of the routing system is too high (e.g., meets (e.g., exceeds) a third threshold), the routing system monitoring agent may determine to add a node to the routing system. Additionally or alternatively, if the enqueue rate of the first queue is too high (e.g., satisfies (e.g., exceeds) a fourth threshold), the dequeue rate of the first queue is too low (e.g., lower than the enqueue rate of the first queue, satisfies (e.g., is lower than) at least one fifth threshold, etc.), and / or the queue utilization is too high (e.g., satisfies (e.g., exceeds) a sixth threshold), the first queue monitoring agent may determine to add a node to the routing system. Additionally or alternatively, if the resource utilization (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) of the routing system is too low (e.g., satisfies (e.g., is lower than) at least one seventh threshold), the data rate (e.g., packet rate) of the routing system is too low (e.g., satisfies (e.g., is lower than) an eighth threshold), and / or the packet drop rate of the routing system is too low (e.g., satisfies (e.g., is lower than) a ninth threshold or is zero), the routing system monitoring agent may determine to remove a node from the routing system. Additionally or alternatively, if the enqueue rate of the first queue is too low (e.g., meets (e.g., is lower than) a tenth threshold), the dequeue rate of the first queue is too high (e.g., is greater than the enqueue rate of the first queue, meets (e.g., exceeds) at least one eleventh threshold, etc.), and / or the queue utilization is too low (e.g., meets (e.g., is lower than) a twelfth threshold), the first queue monitoring agent may determine to remove the node from the routing system. In some non-limiting embodiments or aspects, the routing system node provisioning system may add (e.g., provision, upgrade, etc.) nodes to the routing system, or remove (e.g., downgrade) nodes from the routing system based on the routing system monitoring agent and / or the first queue monitoring agent determining whether to add or remove nodes, respectively.

[0133] In some non-limiting embodiments or aspects, the resource monitoring system may include a second queue monitoring agent and / or a resolution system monitoring agent. Additionally or alternatively, the node provisioning system may include a resolution system node provisioning system. In some non-limiting embodiments or aspects, the second queue monitoring agent and / or the resolution system monitoring agent may determine whether to add a node (e.g., an additional second server or an additional partition thereof) to the resolution system based on a metric of the second queue and / or the resolution system. For example, if the resource utilization (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) of the resolution system and / or any of its protocol parser nodes is too high (e.g., meets (e.g., exceeds) at least one first threshold), the data rate (e.g., packet rate) of the resolution system and / or any of its protocol parser nodes is too high (e.g., meets (e.g., exceeds) a second threshold), and / or the packet drop rate of the resolution system and / or any of its protocol parser nodes is too high (e.g., meets (e.g., exceeds) a third threshold), then the resolution system monitoring agent may determine to add a node to the resolution system. Additionally or alternatively, the second queue monitoring agent may determine to add a node to the resolution system if the enqueue rate of at least one (e.g., at least some) of the second queues is too high (e.g., satisfies (e.g., exceeds) a fourth threshold), the dequeue rate of at least one (e.g., at least some) of the second queues is too low (e.g., lower than the enqueue rate of the corresponding second queue, satisfies (e.g., lower than) at least one fifth threshold, etc.), and / or the queue utilization of at least one (e.g., at least some) of the second queues is too high (e.g., satisfies (e.g., exceeds) a sixth threshold). Additionally or alternatively, the resolution system monitoring agent may determine to remove a node from the resolution system if resource utilization (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) of the resolution system and / or any of its protocol parser nodes is too low (e.g., satisfies (e.g., is lower than) at least one seventh threshold), the data rate (e.g., packet rate) of the resolution system and / or any of its protocol parser nodes is too low (e.g., satisfies (e.g., is lower than) an eighth threshold), and / or a packet drop rate of the resolution system and / or any of its protocol parser nodes is too low (e.g., satisfies (e.g., is lower than) a ninth threshold or is zero).Additionally or alternatively, if the enqueue rate of at least one (e.g., at least some) of the second queues is too low (e.g., satisfies (e.g., is lower than) a tenth threshold), the dequeue rate of at least one (e.g., at least some) of the second queues is too high (e.g., greater than the enqueue rate of the corresponding second queue, satisfies (e.g., exceeds) at least one eleventh threshold, etc.), and / or the queue utilization of at least one (e.g., at least some) of the second queues is too low (e.g., satisfies (e.g., is lower than) a twelfth threshold), the second queue monitoring agent may determine to remove the node from the resolution system. In some non-limiting embodiments or aspects, the resolution system node provisioning system may add (e.g., provision, upgrade, etc.) nodes to the resolution system, or remove (e.g., downgrade) nodes from the resolution system based on the resolution system monitoring agent and / or the second queue monitoring agent determining whether to add or remove nodes, respectively.

[0134] In some non-limiting embodiments or aspects, the resource monitoring system may include a third queue monitoring agent and / or an aggregate system monitoring agent. Additionally or alternatively, the node provisioning system may include an aggregate system node provisioning system. In some non-limiting embodiments or aspects, the third queue monitoring agent and / or the aggregate system monitoring agent may determine whether to add a node (e.g., an additional fourth server or an additional partition thereof) to the aggregate system based on a metric of the third queue and / or the aggregate system. For example, if the resource utilization of the aggregate system (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) is too high (e.g., meets (e.g., exceeds) at least one first threshold), the data rate (e.g., packet rate) of the aggregate system is too high (e.g., meets (e.g., exceeds) a second threshold), and / or the packet drop rate of the aggregate system is too high (e.g., meets (e.g., exceeds) a third threshold), the aggregate system monitoring agent may determine to add a node to the aggregate system. Additionally or alternatively, if the enqueue rate of the third queue is too high (e.g., satisfies (e.g., exceeds) a fourth threshold), the dequeue rate of the third queue is too low (e.g., lower than the enqueue rate of the first queue, satisfies (e.g., is lower than) at least one fifth threshold, etc.), and / or the queue utilization is too high (e.g., satisfies (e.g., exceeds) a sixth threshold), the third queue monitoring agent may determine to add a node to the aggregation system. Additionally or alternatively, if the resource utilization (e.g., at least one of processor utilization, memory utilization, input / output (I / O) utilization, any combination thereof, etc.) of the aggregation system is too low (e.g., satisfies (e.g., is lower than) at least one seventh threshold), the data rate (e.g., packet rate) of the aggregation system is too low (e.g., satisfies (e.g., is lower than) an eighth threshold), and / or the packet drop rate of the aggregation system is too low (e.g., satisfies (e.g., is lower than) a ninth threshold or is zero), the routing system monitoring agent may determine to remove a node from the aggregation system. Additionally or alternatively, if the enqueue rate of the third queue is too low (e.g., meets (e.g., is lower than) a tenth threshold), the dequeue rate of the third queue is too high (e.g., is greater than the enqueue rate of the third queue, meets (e.g., exceeds) at least one eleventh threshold, etc.), and / or the queue utilization is too low (e.g., meets (e.g., is lower than) a twelfth threshold), the third queue monitoring agent may determine to remove the node from the aggregation system. In some non-limiting embodiments or aspects, the aggregation system node provisioning system may add (e.g., provision, upgrade, etc.) nodes to the aggregation system, or remove (e.g., downgrade) nodes from the aggregation system based on the aggregation system monitoring agent and / or the third queue monitoring agent determining whether to add or remove nodes, respectively.

[0135] Reference Figures 4A-4C , Figures 4A-4C is with Figure 3FIG. 4 is a diagram of an exemplary implementation 400 of a non-limiting embodiment or aspect of the process 300 shown in FIG. Figures 4A-4C As shown, implementation 400 may include at least one packet 403, a packet capture system 402, a packet capture subsystem 402a, a lower layer parser subsystem 402b, a protocol identifier subsystem 402c, a first queue 404, a routing system 406, at least one second queue (e.g., at least one first protocol queue 408-1, at least one second queue 408-2...at least one nth protocol queue 408-n, collectively referred to as second queues 408 and individually referred to as second queues 408), a parsing system 410, at least one protocol parser node (e.g., at least one first protocol parser node 410-1, at least one second protocol parser node 410-2...at least one nth protocol parser node 410-n), a third queue 412, an aggregation system 414, a database 416, a resource monitoring system 418, a node provisioning system 420, a mapping database 422, a policy database 424, any combination thereof, and the like. In some non-limiting embodiments or aspects, various components of implementation 400 may be the same as or similar to and / or a part of transaction service provider system 102 (e.g., one or more devices of transaction service provider system 102). In some non-limiting embodiments or aspects, various components of implementation 400 may be the same as or similar to and / or a part of another system, another device, another group of systems or another group of devices (e.g., issuer system 104, client device 106, merchant system 108, acquirer system 110, etc.) that is independent of or includes transaction service provider system 102.

[0136] In some non-limiting embodiments or aspects, the packet capture system 402 may include and / or be a portion of at least one computing device. For example, the packet capture system 402 may include at least one server (e.g., a first server) or at least one partition thereof and / or be a portion of at least one server or at least one partition thereof. Additionally or alternatively, the packet capture system 402 may be configured as described above with reference to Figure 3 The packet capture systems described are the same or similar.

[0137] In some non-limiting embodiments or aspects, the packet capture system 402 may include at least one subsystem. For example, the packet capture system 402 may include a packet capture subsystem 402a, a lower-layer parser subsystem 402b, a protocol identifier subsystem 402c, any combination thereof, and the like. In some non-limiting embodiments or aspects, each subsystem (e.g., the packet capture subsystem 402a, the lower-layer parser subsystem 402b, the protocol identifier subsystem 402c, and the like) may be implemented (e.g., completely, partially, and the like) by at least one server and / or a partition thereof. For example, the packet capture system 402 may include a plurality of first servers, and each first server may include an instance of each of the packet capture subsystem 402a, the lower-layer parser subsystem 402b, and the protocol identifier subsystem 402c. Additionally or alternatively, the plurality of first servers may operate in parallel.

[0138] In some non-limiting embodiments or aspects, the first queue 404 may include at least one computing device and / or be a part thereof. For example, the first queue 404 may include a message queue (e.g., Kafka, etc.), a FIFO queue, a priority queue, at least one server or partition thereof, a database, any combination thereof, etc. Additionally or alternatively, the first queue 404 may be associated with the above referenced Figure 3 The same or similar to the first queue described.

[0139] In some non-limiting embodiments or aspects, routing system 406 may include and / or be a portion of at least one computing device. For example, routing system 406 may include at least one server (e.g., a second server) or a partition thereof. Additionally or alternatively, routing system 406 may be configured as described above with reference to Figure 3 The routing systems described herein may be the same or similar. In some non-limiting embodiments or aspects, routing system 406 may include multiple second servers, and each second server may include an instance of routing system 404. Additionally or alternatively, multiple second servers may operate in parallel.

[0140] In some non-limiting embodiments or aspects, each second queue 408 may include at least one computing device and / or be a part thereof. For example, each second queue 408 may include a message queue (e.g., Kafka, etc.), a FIFO queue, a priority queue, at least one server or partition thereof, a database, any combination thereof, etc. Additionally or alternatively, each second queue 408 may be connected to the above referenced Figure 3 The second queue described is the same or similar.

[0141] In some non-limiting embodiments or aspects, the parsing system 410 may include and / or be a portion of at least one computing device. For example, the parsing system 410 may include at least one server (e.g., a third server) or a partition thereof. Additionally or alternatively, the parsing system 410 may be configured as described above with reference to Figure 3 The parsing systems described are the same or similar.

[0142] In some non-limiting embodiments or aspects, the parsing system 410 may include at least one protocol parser node (e.g., at least one first protocol parser node 410-1, at least one second protocol parser node 410-2...at least one nth protocol parser node 410-n, any combination thereof, etc.). For example, each protocol parser node may be implemented (e.g., completely, partially, etc.) by at least one server and / or its partition. In some non-limiting embodiments or aspects, the parsing system 410 may include multiple third servers, and each third server may include at least one protocol parser node. Additionally or alternatively, multiple third servers and / or multiple protocol parser nodes may work in parallel. In some non-limiting embodiments or aspects, each protocol parser node (e.g., at least one first protocol parser node 410-1, at least one second protocol parser node 410-2...at least one nth protocol parser node 410-n, any combination thereof, etc.) may be implemented with reference to the above. Figure 3 The protocol parser nodes described are the same or similar.

[0143] In some non-limiting embodiments or aspects, the third queue 412 may include at least one computing device and / or be a part thereof. For example, the third queue 412 may include a message queue (e.g., Kafka, etc.), a FIFO queue, a priority queue, at least one server or partition thereof, a database, any combination thereof, etc. Additionally or alternatively, the third queue 412 may be associated with the above referenced Figure 3 The third queue described is the same or similar.

[0144] In some non-limiting embodiments or aspects, the aggregation system 414 may include and / or be a portion of at least one computing device. For example, the aggregation system 414 may include at least one server (e.g., the fourth server) or a partition thereof. Additionally or alternatively, the aggregation system 414 may be the same as the one described above with reference to Figure 3 The polymerisation systems described are the same or similar.

[0145] In some non-limiting embodiments or aspects, database 416 may include and / or be a portion of at least one computing device. For example, database 416 may include a database (e.g., a SQL database, a non-SQL database, a relational database, a non-relational database, a distributed database, etc.), at least one server or partition thereof, any combination thereof, and the like. Additionally or alternatively, database 416 may be associated with the above-referenced Figure 3 The databases described are the same or similar.

[0146] In some non-limiting embodiments or aspects, the resource monitoring system 418 may include and / or be a portion of at least one computing device. For example, the resource monitoring system 418 may include at least one server (e.g., the fifth server) or a partition thereof. Additionally or alternatively, the resource monitoring system 418 may be configured as described above with reference to Figure 3 The resource monitoring systems described are the same or similar.

[0147] In some non-limiting embodiments or aspects, the node provisioning system 420 may include and / or be a portion of at least one computing device. For example, the node provisioning system 420 may include at least one server (e.g., the sixth server) or a partition thereof. Additionally or alternatively, the node provisioning system 420 may be configured as described above with reference to Figure 3 The node provisioning system described is the same or similar.

[0148] In some non-limiting embodiments or aspects, mapping database 422 may include and / or be a portion of at least one computing device. For example, mapping database 422 may include a database (e.g., a SQL database, a non-SQL database, a relational database, a non-relational database, a distributed database, etc.), at least one server or partition thereof, any combination thereof, and the like. Additionally or alternatively, mapping database 422 may be associated with the above referenced Figure 3 The mapping databases described are the same or similar.

[0149] In some non-limiting embodiments or aspects, the policy database 424 may include and / or be a portion of at least one computing device. For example, the policy database 424 may include a database (e.g., a SQL database, a non-SQL database, a relational database, a non-relational database, a distributed database, etc.), at least one server or partition thereof, any combination thereof, etc. Additionally or alternatively, the policy database 424 may be configured as described above with reference to Figure 3 The policy databases described are the same or similar.

[0150] refer to Figure 4AIn some non-limiting embodiments or aspects, packet capture system 402 can receive a plurality of packets (e.g., from a communication network, such as communication network 112, etc., and / or a network tap connected thereto), as described herein. For example, packet capture subsystem 402a can receive packets, as described herein.

[0151] In some non-limiting embodiments or aspects, the packet capture system 402 can parse lower layer data (e.g., at least one of data link layer data, network layer data, transport layer data, any combination thereof, etc.) from each packet, as described herein. For example, the lower layer parser subsystem 402b can parse lower layer data (e.g., at least one of data link layer data, network layer data, transport layer data, any combination thereof, etc.) from each packet, as described herein.

[0152] In some non-limiting embodiments or aspects, the packet capture system 402 can transmit at least a portion of each respective packet (e.g., a respective payload, at least a portion of a respective header, any combination thereof, etc.) to at least one first queue 404, as described herein. Additionally or alternatively, the packet capture system 402 can determine a respective protocol (e.g., an application layer protocol) for each respective packet and / or transmit the respective packet and / or a portion thereof (e.g., a payload) together with protocol data associated with the respective protocol of the respective packet to the first queue 404. For example, the protocol identifier subsystem 402c can determine a respective protocol (e.g., an application layer protocol) for each respective packet and / or transmit the respective packet and / or a portion thereof (e.g., a payload) together with protocol data associated with the respective protocol of the respective packet to the first queue 404, as described herein.

[0153] In some non-limiting embodiments or aspects, the packet capture system 402 (e.g., its protocol identifier subsystem 402c, etc.) can generate a corresponding packet identifier for each corresponding packet as described herein. Additionally or alternatively, the packet capture system 402 (e.g., its protocol identifier subsystem 402c, etc.) can transmit the corresponding packet and / or a portion thereof (e.g., a payload) together with the corresponding packet identifier of the corresponding packet.

[0154] In some non-limiting embodiments or aspects, the routing system 406 can subscribe to the topics of the first queue 404, as described herein. For example, each respective topic can be associated with a respective protocol of a plurality of protocols, as described herein. In some non-limiting embodiments or aspects, the packet capture system 402 (e.g., its protocol identifier subsystem 402c, etc.) can publish the respective packets and / or portions thereof (e.g., payload) in the first queue with the respective topics based on the respective protocols of each respective packet.

[0155] In some non-limiting embodiments or aspects, the routing system 406 may route the respective packets and / or portions thereof (e.g., payload) to respective second queues of a plurality of second queues based on the respective protocol (e.g., application layer protocol) of each respective packet, as described herein.

[0156] In some non-limiting embodiments or aspects, each respective protocol parser node (e.g., at least one first protocol parser node 410-1, at least one second protocol parser node 410-2, ... at least one nth protocol parser node 410-n) of the parsing system 410 may be associated with a respective protocol (e.g., a respective application layer protocol) of the plurality of protocols, as described herein. Additionally or alternatively, each respective second queue 408 may be associated with a respective protocol parser node of the parsing system 410, as described herein.

[0157] In some non-limiting embodiments or aspects, the routing system 406 may route the corresponding packets and / or portions thereof (e.g., payload) to the selected second queue 408 based on the protocol data (e.g., subject) of each corresponding packet corresponding to the corresponding protocol of a corresponding protocol parser node (e.g., a corresponding one of the first protocol parser node 410-1, the second protocol parser node 410-2, ... the nth protocol parser node 410-n) associated with the selected second queue in the second queue 408.

[0158] refer to Figure 4BIn some non-limiting embodiments or aspects, there may be multiple protocol parser nodes associated with each corresponding protocol in multiple (application layer) protocols. For example, each first protocol parser node 410-1 in a plurality of first protocol parser nodes 410-1 (e.g., first-first protocol parser node 410-1-1, second-first protocol parser node 410-1-2, third-first protocol parser node 410-1-3, etc.) may be associated with a first application layer protocol P1. Additionally or alternatively, each second protocol parser node 410-2 in a plurality of second protocol parser nodes 410-2 (e.g., first-second protocol parser node 410-2-1, second-first protocol parser node 410-1-2, etc.) may be associated with a second application layer protocol P2. In some non-limiting embodiments or aspects, each protocol parser node may have a corresponding second queue 408 associated therewith (e.g., each protocol parser node may subscribe to and / or consume packets (and / or portions thereof) from a corresponding one of the second queues 408, etc.). For example, the first-first protocol parser node 410-1-1 may have a first-first protocol second queue 408-1-1 associated therewith; the second-first protocol parser node 410-1-2 may have a second-first protocol second queue 408-1-2 associated therewith; the third-first protocol parser node 410-1-3 may have a third-first protocol second queue 408-1-3 associated therewith; the first-second protocol parser node 410-2-1 may have a first-second protocol second queue 408-2-1 associated therewith; the second-first protocol parser node 410-1-2 may have a second-second protocol second queue 408-2-2 associated therewith; any combination thereof And so on. In some non-limiting embodiments or aspects, at least some of the protocol parser nodes may come from different vendors. For example, the first-first protocol parser node 410-1-1, the second-first protocol parser node 410-1-2 and / or the first-second protocol parser node 410-2-1 may be from the first vendor V1; the third-first protocol parser node 410-1-3 and / or the second-second protocol parser node 410-2-2 may be from the second vendor V2; any combination thereof, and so on. In this way, the first-first protocol parser node 410-1-1 may include a first node N1 associated with a first vendor V1 of a first protocol P1, and / or the second-first protocol parser node 410-1-2 may include a second node N2 associated with a first vendor V1 of a first protocol P1. Additionally or alternatively, the third-first protocol parser node 410-1-3 may include a third node N3 associated with a second vendor V2 of a first protocol P1.Additionally or alternatively, the first-second protocol parser node 410-2-1 may include a fourth node N4 associated with the first vendor V1 of the second protocol P2, and / or the second-second protocol parser node 410-2-2 may include a fifth node N5 associated with the second vendor V2 of the second protocol P2. In some non-limiting embodiments or aspects, the second queue 408 may include an identifier associated with a protocol, a vendor, and / or a node number of a corresponding protocol parser node associated therewith. For example, the first-first protocol second queue 408-1-1 may have an identifier based on the first protocol P1 and the first vendor V1 (e.g., P1_V1_1); the second-first protocol second queue 408-1-2 may have an identifier based on the first protocol P1 and the first vendor V1 (e.g., P1_V1_2); the third-first protocol second queue 408-1-3 may have an identifier based on the first protocol P1 and the second vendor V2 (e.g., P1_V2_1); the first-second protocol second queue 408-2-1 may have an identifier based on the second protocol P2 and the first vendor V1 (e.g., P2_V1_1); and / or the second-second protocol second queue 408-2-2 may have an identifier based on the second protocol P2 and the second vendor V2 (e.g., P2_V2_1).

[0159] In some non-limiting embodiments, each protocol parser node of the parsing system 410 can have its own second queue 408. Additionally or alternatively, the routing system 406 can assign the corresponding second queue 408 to the corresponding protocol parser node as a protocol_vendor_identifier, as described herein. In some non-limiting embodiments, the routing system 406 can maintain (e.g., store) a list of the number of second queues 408 for each protocol for each vendor. In some non-limiting embodiments or aspects, each protocol parser node can transmit a vendor (e.g., V1, V2, etc.) and / or a protocol (e.g., P1, P2, etc.) to the routing system 406. Additionally or alternatively, the routing system 406 can determine the topic of the corresponding protocol parser node as follows. The routing system 406 can check whether the corresponding vendor is in the vendor list, and if not, the routing system 406 can add the corresponding vendor to the vendor list. Additionally or alternatively, the routing system 406 can retrieve from the list the total number of second queues 408 for the corresponding protocol of the corresponding vendor, and / or the routing system 406 can increase the number by one (e.g., indicating that the corresponding protocol parser node is added to the list). Additionally or alternatively, the routing system 406 can determine (e.g., calculate) an identifier of the corresponding second queue 408 based on the list. Additionally or alternatively, the routing system 406 can determine (e.g., generate) a topic as protocol_vendor_identifier (e.g., P1_V1_1, P1_V1_2, P1_V2_1, P2_V1_1, P2_V2_1, etc.).

[0160] In some non-limiting embodiments, packet flows (e.g., packets from a session, etc.) can be assigned to corresponding second queues 408 and / or corresponding protocol parser nodes based on flow hashes. For example, routing system 406 can determine the modulus of the flow hash to the number of second queues 408 for a given protocol (e.g., flow_hash mod(# of queues)).

[0161] Continue to refer Figure 4A In some non-limiting embodiments or aspects, routing system 406 can subscribe to the topic of first queue 404, as described herein. Additionally or alternatively, routing system 406 can consume each corresponding packet and / or portion thereof (e.g., payload) from first queue 404, as described herein.

[0162] In some non-limiting embodiments or aspects, routing system 406 can publish the respective packets and / or portions thereof (e.g., payload) in respective second queues 408 based on respective topics (e.g., protocols) of each respective packet, as described herein. Additionally or alternatively, each protocol parser node of parsing system 410 can subscribe to at least one (e.g., a respective one) of second queues 408, as described herein.

[0163] In some non-limiting embodiments or aspects, each respective protocol parser node of the parsing system 410 can parse higher layer data (e.g., at least one of session layer data, presentation layer data, application layer data, any combination thereof, etc.) from each respective packet and / or portion thereof (e.g., payload) from a respective second queue as described herein.

[0164] In some non-limiting embodiments or aspects, routing system 406 can publish each respective packet and / or portion thereof (e.g., payload) in respective second queue 408, as described herein. Additionally or alternatively, each respective protocol parser node of parsing system 410 can subscribe to and / or consume respective packets and / or portions thereof (e.g., payload) from respective second queue 408, as described herein.

[0165] In some non-limiting embodiments or aspects, the packet capture system 402 can transmit the lower layer data (e.g., data link layer data, network layer data, transport layer data, etc.) of each packet to the third queue 412, as described herein. Additionally or alternatively, the parsing system 410 (e.g., its protocol parser node) can transmit the higher layer data (e.g., session layer data, presentation layer data, application layer data, etc.) of each packet to the third queue 412, as described herein.

[0166] In some non-limiting embodiments or aspects, aggregation system 414 can aggregate lower layer data (e.g., data link layer data, network layer data, transport layer data, etc.) for each packet with higher layer data (e.g., session layer data, presentation layer data, application layer data, etc.) for each packet, as described herein. Additionally or alternatively, aggregation system 414 can store aggregated data (e.g., lower layer data aggregated with higher layer data) for each packet in database 416, as described herein.

[0167] In some non-limiting embodiments or aspects, mapping database 422 may receive (e.g., from a user and / or a computing device of the user) and / or store custom field data associated with a plurality of custom fields (e.g., fields of parsed data from at least one parser node of a parsing system, which may have a custom (e.g., unique, vendor-specific, etc.) format), as described herein. Additionally or alternatively, mapping database 422 may generate a respective mapping between each respective custom field in the plurality of custom fields and a respective respective public field of a plurality of public fields (e.g., fields selected by at least one entity associated with an aggregation system, which may have a common (e.g., selected, preferred, standardized, common, etc.) format), as described herein. In some non-limiting embodiments or aspects, the aggregation system 414 can convert at least some of the lower layer data and / or higher layer data (e.g., at least one of data link layer data, network layer data, transport layer data, session layer data, presentation layer data, application layer data, any combination thereof, etc.) (e.g., at least one custom field) of at least one packet (e.g., a first packet) based on a corresponding mapping of corresponding custom fields to corresponding common fields, as described herein.

[0168] In some non-limiting embodiments or aspects, the policy database 424 may receive and / or store (e.g., from a user and / or a computing device of the user) aggregation policy data associated with at least one aggregation policy, as described herein. In some non-limiting embodiments or aspects, the aggregation system 414 may filter at least one field of at least one packet (e.g., a first packet) and / or data associated therewith (e.g., higher layer data, lower layer data, etc.) based on the aggregation policy (or multiple policies), as described herein. Additionally or alternatively, the aggregation system 414 may add (e.g., generate and / or append) at least one field to data associated with at least one packet (e.g., a first packet, a second packet, etc.) based on the aggregation policy (or multiple policies), as described herein. Additionally or alternatively, the aggregation system 414 may combine at least two fields of at least one packet (e.g., a first packet, a second packet, a third packet, etc.) and / or data associated therewith based on the aggregation policy (or multiple policies), as described herein.

[0169] refer to Figure 4CIn some non-limiting embodiments or aspects, the resource monitoring system 418 can monitor at least one of the packet capture system 402, the routing system 406, the parsing system 410, the aggregation system 414, the first queue 404, the second queue 408, the third queue 412, any combination thereof, etc. to provide a corresponding metric (e.g., resource utilization (e.g., processor utilization, memory utilization, input / output (I / O) utilization, any group thereof, etc.), data rate, packet drop rate, enqueue rate (e.g., enqueue operations), dequeue rate (e.g., dequeue operations), queue utilization, any combination thereof, etc.) for each of the foregoing systems and / or queues, as described herein. Additionally or alternatively, the node provisioning system 420 can determine whether to add a node (e.g., a server and / or a partition thereof) to or remove a node from at least one of the packet capture system 402, the routing system 406, the parsing system 410, and / or the aggregation system 414 based on at least one of the corresponding metrics, as described herein.

[0170] In some non-limiting embodiments or aspects, resource monitoring system 418 may include a packet capture system monitoring agent, as described herein. Additionally or alternatively, node provisioning system 420 may include a packet capture system node provisioning system, as described herein. In some non-limiting embodiments or aspects, the packet capture system monitoring agent may determine whether to add a node (e.g., an additional first server or an additional partition thereof) to and / or remove a node from packet capture system 402 based on a metric of the packet capture system, as described herein.

[0171] In some non-limiting embodiments or aspects, the resource monitoring system 418 may include a first queue monitoring agent and / or a routing system monitoring agent, as described herein. Additionally or alternatively, the node provisioning system 420 may include a routing system node provisioning system, as described herein. In some non-limiting embodiments or aspects, the first queue monitoring agent and / or the routing system monitoring agent may determine whether to add a node (e.g., an additional second server or an additional partition thereof) to and / or remove a node from the routing system 406 based on metrics of the first queue 404 and / or the routing system 406, as described herein.

[0172] In some non-limiting embodiments or aspects, resource monitoring system 418 may include a second queue monitoring agent and / or a resolution system monitoring agent, as described herein. Additionally or alternatively, node provisioning system 420 may include a resolution system node provisioning system, as described herein. In some non-limiting embodiments or aspects, second queue monitoring agent and / or resolution system monitoring agent may determine whether to add nodes (e.g., additional second servers or additional partitions thereof) to and / or remove nodes from resolution system 410 based on metrics of second queue 408 and / or resolution system 410, as described herein.

[0173] Reference now Figure 5 , Figure 5 is with Figure 3 The process 300 shown and / or Figures 4A-4C A swim lane diagram of an exemplary implementation 500 of a non-limiting embodiment or aspect related to the illustrated implementation 400. In some non-limiting embodiments or aspects, one or more steps of implementation 500 may be performed (e.g., completely, partially, etc.) by a transaction service provider system 102 (e.g., one or more devices of the transaction service provider system 102). In some non-limiting embodiments or aspects, one or more steps of implementation 500 may be performed (e.g., completely, partially, etc.) by another system, another device, another group of systems or another group of devices (e.g., issuer system 104, client device 106, merchant system 108, acquirer system 110, etc.) separate from or including the transaction service provider system 102.

[0174] like Figure 5 As shown, at step 502, process 500 may include receiving, by mapping database 422 (e.g., from a user and / or the user's computing device), custom field data associated with a plurality of custom fields (e.g., fields of parsed data from at least one parser node of a parsing system, which may have a custom (e.g., unique, vendor-specific, etc.) format), as described herein.

[0175] like Figure 5 As shown, at step 504, policy database 424 may receive (eg, from a user and / or a user's computing device) aggregate policy data associated with at least one aggregate policy, as described herein.

[0176] like Figure 5 As shown, at step 506, the mapping database 422 can generate a corresponding mapping between each corresponding custom field in a plurality of custom fields and a corresponding common field in a plurality of common fields (e.g., fields selected by at least one entity associated with the aggregation system, which can have a common (e.g., selected, preferred, standardized, common, etc.) format), as described herein.

[0177] like Figure 5 As shown, at step 508, aggregation system 414 can subscribe to data (eg, lower layer data, higher layer data, etc.) from third queue 412, as described herein.

[0178] like Figure 5As shown, at step 510, the third queue 412 can publish (and / or have published therein from the packet capture system 402 and / or the parsing system 410) data (e.g., lower layer data, higher layer data, etc.), as described herein. Additionally or alternatively, the aggregation system 414 can consume data from the third queue 412, as described herein.

[0179] like Figure 5 As shown, at step 512, mapping database 422 can transmit the mapping between custom field(s) and common field(s) to aggregation system 414. Additionally or alternatively, aggregation system 414 can retrieve (e.g., read, request, etc.) the mapping from mapping database 422.

[0180] like Figure 5 As shown, at step 514, the policy database 424 can transmit the aggregated policy data to the aggregation system 414. Additionally or alternatively, the aggregation system 414 can retrieve (eg, read, request, etc.) the aggregated policy data from the policy database 424.

[0181] like Figure 5 As shown, at step 516, the aggregation system 414 may filter at least one field of at least one packet (e.g., the first packet) and / or data associated therewith (e.g., higher layer data, lower layer data, etc.) based on the aggregation policy (or policies), as described herein. Additionally or alternatively, the aggregation system 414 may add (e.g., generate and / or append) at least one field to the data associated with at least one packet (e.g., the first packet, the second packet, etc.) based on the aggregation policy (or policies), as described herein. Additionally or alternatively, the aggregation system 414 may combine at least two fields of at least one packet (e.g., the first packet, the second packet, the third packet, etc.) and / or data associated therewith based on the aggregation policy (or policies), as described herein.

[0182] like Figure 5 As shown, at step 518, the aggregation system 414 can convert at least some of the lower layer data and / or higher layer data (e.g., at least one of data link layer data, network layer data, transport layer data, session layer data, presentation layer data, application layer data, any combination thereof, etc.) (e.g., at least one custom field) of at least one packet (e.g., a first packet) based on a corresponding mapping of corresponding custom fields to corresponding common fields, as described herein.

[0183] like Figure 5 As shown, at step 520, aggregation system 414 can aggregate the lower layer data for each packet with higher layer data (eg, filtered and / or transformed), as described herein.

[0184] like Figure 5 As shown, at step 520, aggregation system 414 may store aggregated data (eg, lower layer data aggregated with higher layer data) for each packet in database 416, as described herein.

[0185] Although the disclosed subject matter has been described in detail for purposes of illustration based on what are presently considered to be the most practical and preferred embodiments or aspects, it should be understood that such details are for that purpose only and that the disclosed subject matter is not limited to the disclosed embodiments or aspects, but rather is intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it should be understood that the presently disclosed subject matter contemplates that one or more features of any embodiment or aspect may be combined with one or more features of any other embodiment or aspect as much as possible.

Claims

1. A computer-implemented method comprising: receiving, by a packet capture system, a plurality of packets; parsing, by the packet capture system, at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets; transmitting, by the packet capture system, a respective payload of each respective packet of the plurality of packets to at least one first queue, wherein the first queue comprises a payload queue; routing, by a routing system, a respective payload of each respective packet in the plurality of packets to a respective second queue of a plurality of second queues based on a respective protocol of the respective packet, wherein the second queues include a protocol-based queue; parsing, by a respective protocol parser node of the parsing system, at least one of session layer data, presentation layer data, or application layer data from a respective payload of each respective packet from each respective second queue of the plurality of second queues; transmitting, by the packet capture system, at least one of data link layer data, network layer data, or transport layer data of each of the plurality of packets to a third queue, wherein the third queue comprises a metadata queue; as well as At least one of session layer data, presentation layer data, or application layer data of each packet of the plurality of packets is transmitted by the parsing system to the third queue.

2. The method according to claim 1, wherein: The packet capture system includes at least one first server, the routing system includes at least one second server, the parsing system includes a plurality of third servers, and each respective protocol parser node includes a respective third server of the plurality of third servers.

3. The method according to claim 2, wherein: Each of the at least one first server comprises a packet capture subsystem, a lower layer parser subsystem, and a protocol identifier subsystem, wherein receiving the plurality of packets comprises receiving the plurality of packets by the packet capture subsystem, wherein parsing at least one of data link layer data, network layer data, or transport layer data from each packet comprises parsing, by the lower layer parser subsystem, at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets, and Wherein, transmitting the corresponding payload of each corresponding packet in the plurality of packets to the at least one first queue comprises: determining, by the protocol identifier subsystem, a respective protocol for each respective packet in the plurality of packets; and A respective payload of each respective packet of the plurality of packets is transmitted by the packet capture system to the at least one first queue along with protocol data associated with a respective protocol of the respective packet.

4. The method according to claim 3, wherein: Each corresponding protocol parser node of the parsing system is associated with a corresponding protocol of the plurality of protocols, wherein each corresponding second queue of the plurality of second queues is associated with a corresponding protocol parser node of the parsing system, and Wherein, routing each corresponding packet among the multiple packets to a corresponding second queue among the multiple second queues includes: routing the corresponding packet to a selected one of the multiple second queues based on that the protocol data of each corresponding packet among the multiple packets corresponds to a corresponding protocol of a corresponding protocol parser node associated with a selected one of the multiple second queues.

5. The method according to claim 1, further comprising: aggregating, by the aggregation system, at least one of the data link layer data, the network layer data, or the transport layer data of each packet with at least one of the session layer data, the presentation layer data, or the application layer data of each packet; as well as At least one of data link layer data, network layer data, or transport layer data for each packet is stored in a database aggregated with at least one of session layer data, presentation layer data, or application layer data for each packet.

6. The method according to claim 5, further comprising: generating, by the packet capture system, a respective packet identifier for each respective packet in the plurality of packets, wherein transmitting the corresponding payload of each corresponding packet in the plurality of packets to at least one first queue comprises: transmitting the corresponding payload of each corresponding packet in the plurality of packets together with the corresponding packet identifier of the corresponding packet, wherein transmitting at least one of data link layer data, network layer data, or transport layer data of each of the plurality of packets to the third queue comprises: transmitting at least one of data link layer data, network layer data, or transport layer data of each of the plurality of packets together with a corresponding packet identifier of the corresponding packet to the third queue, wherein transmitting at least one of session layer data, presentation layer data, or application layer data of each of the plurality of packets to the third queue comprises: transmitting at least one of session layer data, presentation layer data, or application layer data of each of the plurality of packets together with a corresponding packet identifier of the corresponding packet to the third queue, and Wherein, aggregating at least one of the data link layer data, network layer data or transport layer data of each packet with at least one of the session layer data, presentation layer data or application layer data of each packet includes: aggregating at least one of the data link layer data, network layer data or transport layer data of each packet with at least one of the session layer data, presentation layer data or application layer data of each packet based on matching the corresponding packet identifier of the corresponding packet transmitted together with at least one of the data link layer data, the network layer data or the transport layer data with the corresponding packet identifier of the corresponding packet transmitted together with at least one of the session layer data, the presentation layer data or the application layer data.

7. The method according to claim 5, further comprising: Receiving, by a mapping database, custom field data associated with a plurality of custom fields; as well as generating, by the mapping database, a corresponding mapping between each corresponding custom field in the plurality of custom fields and a corresponding public field in the plurality of public fields, Wherein, the aggregation includes converting at least one of the data link layer data, network layer data, transport layer data, session layer data, presentation layer data or application layer data of the first packet of the multiple packets based on a corresponding mapping of the corresponding custom fields associated with at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data or the application layer data to the corresponding common fields.

8. The method according to claim 5, further comprising: receiving, by a policy database, aggregate policy data associated with at least one aggregate policy, The aggregation includes filtering at least one field of a first group among the multiple groups based on the at least one aggregation strategy.

9. The method according to claim 1, further comprising: subscribing, by the routing system, to a plurality of topics of the first queue, each respective topic of the plurality of topics being associated with a respective protocol of a plurality of protocols, Wherein, transmitting the corresponding payload of each corresponding packet in the multiple packets to at least one first queue includes: publishing each corresponding payload of each corresponding packet in the multiple packets together with the corresponding topic in the multiple topics in the first queue based on the corresponding protocol of the corresponding packet.

10. The method according to claim 9, wherein: Routing the respective payload of each respective packet in the plurality of packets comprises: publishing the respective payload of the respective packet in a respective second queue in the plurality of second queues based on the respective topic of each respective packet in the plurality of packets, and Wherein, parsing at least one of session layer data, presentation layer data or application layer data from the corresponding payload of each corresponding packet includes: consuming the corresponding payload of each corresponding packet from each corresponding second queue of the multiple second queues by the corresponding protocol parser node of the parsing system.

11. The method according to claim 1, further comprising: monitoring, by a resource monitoring system, the packet capture system, the routing system, the parsing system, the first queue, the plurality of second queues, and the third queue to provide respective metrics for each of the packet capture system, the routing system, the parsing system, the first queue, the second queue, and the third queue; as well as A node provisioning system determines whether to add or remove a node to or from at least one of the packet capture system, the routing system, or the parsing system based on at least one of the corresponding metrics of the packet capture system, the corresponding metrics of the routing system, the corresponding metrics of the parsing system, the corresponding metrics of the first queue, the corresponding metrics of the second queue, or the corresponding metrics of the third queue.

12. The method according to claim 11, wherein: At least one of the following is true: The resource monitoring system includes a packet capture system monitoring agent, and the node provisioning system includes a packet capture system node provisioning system; The resource monitoring system includes a first queue monitoring agent and a routing system monitoring agent, and the node provisioning system includes a routing system node provisioning system; or The resource monitoring system includes a second queue monitoring agent and a resolution system monitoring agent, and the node provisioning system includes a resolution system node provisioning system.

13. A system comprising: A packet capture system, the packet capture system comprising at least one hardware processor, the at least one hardware processor being configured to: Receiving multiple packets; parsing at least one of data link layer data, network layer data, or transport layer data from each packet of the plurality of packets; transferring a respective payload of each respective packet of the plurality of packets to at least one first queue, wherein the first queue comprises a payload queue; and transmitting at least one of data link layer data, network layer data, or transport layer data of each packet of the plurality of packets to a third queue, wherein the third queue comprises a metadata queue; a routing system comprising at least one hardware processor configured to route a respective payload of each respective packet of the plurality of packets to a respective second queue of a plurality of second queues based on a respective protocol of the respective packet, wherein the second queues comprise a protocol-based queue; and A parsing system comprising a plurality of protocol parser nodes, each respective protocol parser node comprising at least one hardware processor, wherein the at least one hardware processor is configured to: parsing at least one of session layer data, presentation layer data, or application layer data from a respective payload of each respective packet from each respective second queue of the plurality of second queues; and At least one of session layer data, presentation layer data, or application layer data of each packet of the plurality of packets is transferred to the third queue.

14. The system according to claim 13, wherein: The packet capture system includes at least one first server, the routing system includes at least one second server, the parsing system includes a plurality of third servers, and each respective protocol parser node includes a respective third server of the plurality of third servers.

15. The system of claim 14, wherein: Each of the at least one first server comprises a packet capture subsystem, a lower layer parser subsystem, and a protocol identifier subsystem, wherein receiving the plurality of packets comprises receiving the plurality of packets by the packet capture subsystem, wherein parsing at least one of data link layer data, network layer data, or transport layer data from each packet comprises parsing, by the lower layer parser subsystem, at least one of data link layer data, network layer data, or transport layer data from each of the plurality of packets, and Wherein, transmitting the corresponding payload of each corresponding packet in the plurality of packets to the at least one first queue comprises: determining, by the protocol identifier subsystem, a respective protocol for each respective packet in the plurality of packets; and A respective payload of each respective packet of the plurality of packets is transmitted by the packet capture system to the at least one first queue along with protocol data associated with a respective protocol of the respective packet.

16. The system according to claim 15, wherein: Each corresponding protocol parser node of the parsing system is associated with a corresponding protocol of the plurality of protocols, wherein each corresponding second queue of the plurality of second queues is associated with a corresponding protocol parser node of the parsing system, and Wherein, routing each corresponding packet among the multiple packets to a corresponding second queue among the multiple second queues includes: routing the corresponding packet to a selected one of the multiple second queues based on that the protocol data of each corresponding packet among the multiple packets corresponds to a corresponding protocol of a corresponding protocol parser associated with a selected one of the multiple second queues.

17. The system of claim 13, further comprising an aggregation system configured to: aggregating at least one of data link layer data, network layer data, or transport layer data for each packet with at least one of session layer data, presentation layer data, or application layer data for each packet; and At least one of data link layer data, network layer data, or transport layer data for each packet is stored in a database aggregated with at least one of session layer data, presentation layer data, or application layer data for each packet.

18. The system of claim 17, further comprising: A mapping database, the mapping database being configured to: receiving custom field data associated with a plurality of custom fields; as well as generating a corresponding mapping between each corresponding custom field in the plurality of custom fields and a corresponding public field in the plurality of public fields; as well as a policy database configured to receive aggregate policy data associated with at least one aggregate policy, wherein the aggregation comprises converting at least one of data link layer data, network layer data, transport layer data, session layer data, presentation layer data, or application layer data of a first packet of the plurality of packets based on a corresponding mapping of a corresponding custom field associated with at least one of the data link layer data, the network layer data, the transport layer data, the session layer data, the presentation layer data, or the application layer data to the corresponding common field, and The aggregation includes filtering at least one field of a first group among the multiple groups based on the at least one aggregation strategy.

19. The system of claim 13, further comprising: a resource monitoring system configured to monitor the packet capture system, the routing system, the parsing system, the first queue, the plurality of second queues, and the third queue to provide respective metrics for each of the packet capture system, the routing system, the parsing system, the first queue, the second queue, and the third queue; as well as A node provisioning system configured to determine whether to add or remove a node to or from at least one of the packet capture system, the routing system, or the resolution system based on at least one of a corresponding metric of the packet capture system, a corresponding metric of the routing system, a corresponding metric of the resolution system, a corresponding metric of the first queue, a corresponding metric of the second queue, or a corresponding metric of the third queue.

20. A computer program product, the computer program product comprising at least one non-transitory computer readable medium, the at least one non-transitory computer readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to: Receiving multiple packets; parsing at least one of data link layer data, network layer data, or transport layer data from each packet of the plurality of packets; communicating a respective payload of each respective packet of the plurality of packets to at least one first queue, wherein the first queue comprises a payload queue; routing a respective payload of each respective packet in the plurality of packets to a respective second queue in a plurality of second queues based on a respective protocol of the respective packet, wherein the second queues include a protocol-based queue; parsing at least one of session layer data, presentation layer data, or application layer data from a respective payload of each respective packet from each respective second queue of the plurality of second queues; transmitting at least one of data link layer data, network layer data, or transport layer data of each packet of the plurality of packets to a third queue, wherein the third queue comprises a metadata queue; as well as At least one of session layer data, presentation layer data, or application layer data of each packet of the plurality of packets is transferred to the third queue.

Images