Passive network asset discovery method and apparatus

Abstract

This application discloses a passive network asset discovery method and apparatus. The method includes: connecting raw network traffic to the Suricata network threat detection engine, generating a target data stream through the Suricata network threat detection engine, processing the network layer data stream based on a PoF fingerprinting tool to extract network layer fingerprint information, simultaneously processing the application layer data stream based on the Suricata network threat detection engine to extract application layer fingerprint information, and generating an asset discovery result based on the network layer fingerprint information, application layer fingerprint information, other fingerprint information, and fingerprint matching confidence. This solution embeds PoF's passive asset identification capability into the Suricata processing pipeline as a dynamic library, fully utilizing Suricata's high-performance traffic processing and application parsing and identification capabilities to improve the performance and accuracy of asset discovery; and it uses a completely passive approach to discover assets, without generating any probe traffic, thus not affecting the network or business systems.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to methods and apparatus for passive network asset discovery. Background Technology

[0002] With the deepening of digital transformation, the network environments of various organizations are becoming increasingly complex. The widespread application of new technologies such as cloud computing, containerization, and the Internet of Things has led to network assets exhibiting dynamic, diversified, and heterogeneous characteristics. Against this backdrop, comprehensively and accurately understanding network asset information has become a fundamental task for cybersecurity operations. However, existing network asset discovery technologies still face many challenges in dealing with today's complex network environment.

[0003] Existing asset discovery methods mainly include active scanning technology, agent-based asset discovery technology, and log analysis-based asset discovery. However, among these methods, active scanning technology suffers from interference problems, and passive discovery has insufficient identification capabilities. Therefore, how to achieve efficient, accurate, and interference-free network asset discovery is an urgent problem to be solved. Summary of the Invention

[0004] This application aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of this application proposes a passive network asset discovery method, which includes: The raw network traffic is fed into the Suricata network threat detection engine, and the target data stream is generated by the Suricata network threat detection engine; the target data stream includes application layer data stream and network layer data stream. The network layer data stream is processed using a P0F fingerprinting tool to extract network layer fingerprint information; simultaneously, the application layer data stream is processed using the Suricata network threat detection engine to extract application layer fingerprint information. The network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection results, link type, and cache validity period; the application layer fingerprint information includes service type, version information, domain name, and certificate characteristics. Obtain other fingerprint information and fingerprint matching confidence scores corresponding to the current asset, and generate asset discovery results based on network layer fingerprint information, application layer fingerprint information, other fingerprint information and fingerprint matching confidence scores; among them, other fingerprint information includes discovery time, event type and quintuple.

[0005] In one possible implementation, the network layer data stream is processed using a P0F fingerprinting tool to extract network layer fingerprint information, including: When it is determined that the current packet is a TCP handshake packet based on the network layer data flow, feature extraction is performed on the TCP handshake packet to obtain a feature vector; The feature vector is compared with the preset p0f fingerprint database to obtain the network layer fingerprint information.

[0006] In one possible implementation, the method further includes: Search the preset fingerprint cache for historical fingerprint information corresponding to the TCP handshake packet; If historical fingerprint information exists in the preset fingerprint cache and the historical fingerprint information has not expired, then the historical fingerprint information will be used as the network layer fingerprint information.

[0007] In one possible implementation, the method further includes: The network layer fingerprint information is stored in a preset fingerprint cache in a first preset format; wherein, the first preset format includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period; The network layer fingerprint information is output to the preset fingerprint association module in a second preset format; wherein, the second preset format includes a 5-tuple, operating system type, operating system version, network hop distance, whether it is behind NAT, link type, and fingerprint matching confidence.

[0008] In one possible implementation, the application layer data stream is processed based on the Suricata network threat detection engine to extract application layer fingerprint information, including: After identifying the corresponding application layer protocol type based on the port number and packet characteristics in the application layer data stream, if the HTTP protocol is identified, the Suricata network threat detection engine is used to parse the HTTP request and response messages and extract the application layer fingerprint information. If the TLS / SSL protocol is identified, the Suricata network threat detection engine extracts the plaintext metadata of the handshake phase without decrypting the data, and extracts the application layer fingerprint information. If the DNS protocol is identified, the query and response messages are parsed using the Suricata network threat detection engine to extract application layer fingerprint information; If SSH is identified, the Suricata network threat detection engine parses the information from the protocol handshake phase to extract the application layer fingerprint information.

[0009] In one possible implementation, the method further includes: The application layer fingerprint information is output to the preset fingerprint association module in a third preset format; wherein, the third preset format includes a five-tuple, service type, version information, domain name and certificate characteristics.

[0010] In one possible implementation, the raw network traffic is connected to the Suricata network threat detection engine, and the target data stream is generated through the Suricata network threat detection engine, including: After the raw network traffic is fed into the Suricata network threat detection engine via traffic mirroring, packet objects are obtained. The data packet object is parsed to obtain structured data packet information, which includes multiple IP fragment data packets. Multiple IP fragmented data packets are cached and reassembled to restore a complete IP datagram. After associating and sorting multiple complete IP datagrams, a complete network layer data stream is restored. Based on the five-tuple, bidirectional IP datagrams belonging to the same TCP connection in the network layer data stream are extracted and associated. The TCP segmented packets are sorted and reassembled according to the sequence number to restore the complete application layer data stream.

[0011] A second aspect of this application provides a passive network asset discovery device, the device comprising: The first generation module is used to connect the raw network traffic to the Suricata network threat detection engine and generate the target data stream through the Suricata network threat detection engine; wherein, the target data stream includes application layer data stream and network layer data stream; The extraction module is used to process network layer data streams based on P0F fingerprinting tools to extract network layer fingerprint information; at the same time, it processes application layer data streams based on the Suricata network threat detection engine to extract application layer fingerprint information. Among them, the network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection results, link type and cache validity period, and the application layer fingerprint information includes service type, version information, domain name and certificate characteristics. The second generation module is used to obtain other fingerprint information and fingerprint matching confidence scores corresponding to the current asset, and generate asset discovery results based on network layer fingerprint information, application layer fingerprint information, other fingerprint information and fingerprint matching confidence scores; among them, other fingerprint information includes discovery time, event type and quintuple.

[0012] In one possible implementation, the extraction module described above is specifically used for: When it is determined that the current packet is a TCP handshake packet based on the network layer data flow, feature extraction is performed on the TCP handshake packet to obtain a feature vector; The feature vector is compared with the preset p0f fingerprint database to obtain the network layer fingerprint information.

[0013] In one possible implementation, the above-described device is further used for: Search the preset fingerprint cache for historical fingerprint information corresponding to the TCP handshake packet; If historical fingerprint information exists in the preset fingerprint cache and the historical fingerprint information has not expired, then the historical fingerprint information will be used as the network layer fingerprint information.

[0014] In one possible implementation, the above-described device is further used for: The network layer fingerprint information is stored in a preset fingerprint cache in a first preset format; wherein, the first preset format includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period; The network layer fingerprint information is output to the preset fingerprint association module in a second preset format; wherein, the second preset format includes a 5-tuple, operating system type, operating system version, network hop distance, whether it is behind NAT, link type, and fingerprint matching confidence.

[0015] In one possible implementation, the extraction module is further configured to: After identifying the corresponding application layer protocol type based on the port number and packet characteristics in the application layer data stream, if the HTTP protocol is identified, the Suricata network threat detection engine is used to parse the HTTP request and response messages and extract the application layer fingerprint information. If the TLS / SSL protocol is identified, the Suricata network threat detection engine extracts the plaintext metadata of the handshake phase without decrypting the data, and extracts the application layer fingerprint information. If the DNS protocol is identified, the query and response messages are parsed using the Suricata network threat detection engine to extract application layer fingerprint information; If SSH is identified, the Suricata network threat detection engine parses the information from the protocol handshake phase to extract the application layer fingerprint information.

[0016] In one possible implementation, the above-described device is further used for: The application layer fingerprint information is output to the preset fingerprint association module in a third preset format; wherein, the third preset format includes a five-tuple, service type, version information, domain name and certificate characteristics.

[0017] In one possible implementation, the first generation module is specifically used for: After the raw network traffic is fed into the Suricata network threat detection engine via traffic mirroring, packet objects are obtained. The data packet object is parsed to obtain structured data packet information, which includes multiple IP fragment data packets. Multiple IP fragmented data packets are cached and reassembled to restore a complete IP datagram. After associating and sorting multiple complete IP datagrams, a complete network layer data stream is restored. Based on the five-tuple, bidirectional IP datagrams belonging to the same TCP connection in the network layer data stream are extracted and associated. The TCP segmented packets are sorted and reassembled according to the sequence number to restore the complete application layer data stream.

[0018] A third aspect of this application provides an electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the passive network asset discovery method as described in the first aspect.

[0019] The fourth aspect of this application provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the passive network asset discovery method as described in the first aspect.

[0020] The embodiments of this application have the following beneficial effects: The passive network asset discovery method provided in this application includes: connecting raw network traffic to the Suricata network threat detection engine and generating a target data stream through the Suricata network threat detection engine, wherein the target data stream includes an application layer data stream and a network layer data stream; processing the network layer data stream based on a P0F fingerprinting tool to extract network layer fingerprint information; and simultaneously processing the application layer data stream based on the Suricata network threat detection engine to extract application layer fingerprint information, wherein the network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection result, link type, and cache validity period; and the application layer fingerprint information includes service type, version information, domain name, and certificate characteristics; obtaining other fingerprint information and fingerprint matching confidence levels corresponding to the current asset; and generating an asset discovery result based on the network layer fingerprint information, application layer fingerprint information, other fingerprint information, and fingerprint matching confidence levels, wherein the other fingerprint information includes discovery time, event type, and a 5-tuple. This solution embeds P0F's passive asset identification capabilities into the Suricata processing pipeline as a dynamic library, fully leveraging Suricata's high-performance traffic processing and application parsing capabilities to improve the performance and accuracy of asset discovery. Furthermore, by simultaneously extracting network layer and application layer fingerprint information based on Suricata flow reassembly, it achieves multi-dimensional asset information collection in a single packet processing step, enriching asset attributes. Based on real-time traffic analysis, it can promptly detect new and changed assets, and discover assets in a completely passive manner without generating any probe traffic, thus avoiding any impact on the network or business systems. Additionally, it supports plug-in extensions, allowing for easy integration with other passive identification tools, and is implemented using open-source tools, eliminating the need to purchase commercial software and resulting in low deployment and maintenance costs. Attached Figure Description

[0021] Figure 1 A block diagram of a computer device provided in an embodiment of this application; Figure 2 A flowchart illustrating the steps of a passive network asset discovery method provided in this application embodiment; Figure 3 A flowchart illustrating the steps for generating a target data stream is provided in an embodiment of this application. Figure 4 A flowchart illustrating the steps for extracting network layer fingerprint information is provided in this embodiment of the application. Figure 5 This is a structural block diagram of a passive network asset discovery device provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] In the field of cybersecurity, asset discovery is a fundamental task of security operations and maintenance. Existing asset discovery methods mainly include: 1. Proactive scanning technology: Actively probing the target network by sending probe packets to obtain asset information. While comprehensive, this method generates a large amount of network traffic, is easily detected by security devices, and may impact the target system. 2. Agent-based asset discovery: Installing an agent program on each host to manually record and periodically report asset information. This method requires deploying and maintaining a large number of agents, resulting in high management costs. 3. Log analysis-based asset discovery: Discovering assets by analyzing the logs of network devices and security devices. This method relies on the completeness and accuracy of the logs.

[0024] The main problems with existing technologies include: 1. Limitations of active scanning: It generates a large amount of network traffic, which may trigger security alerts and affect business systems. It also cannot detect assets that are in a listening state but do not respond to probes.

[0025] 2. Shortcomings of passive discovery: Existing passive discovery methods mainly rely on traffic mirroring and analysis, but lack the ability to deeply identify operating systems and application services.

[0026] 3. Low tool integration: Most existing tools run independently and lack deep integration with intrusion detection systems, failing to fully utilize the existing traffic analysis capabilities of intrusion detection systems (IDS).

[0027] Based on this, this application proposes a passive network asset discovery method. This solution embeds P0F's passive asset identification capabilities into the Suricata processing pipeline as a dynamic library, fully utilizing Suricata's high-performance traffic processing and application parsing and identification capabilities to improve the performance and accuracy of asset discovery. Furthermore, by simultaneously extracting network layer fingerprint information and application layer fingerprint information based on Suricata flow reassembly, it achieves multi-dimensional asset information collection in a single packet processing, enriching asset attributes. Moreover, it can promptly discover new assets and asset changes based on real-time traffic analysis, and uses a completely passive approach to discover assets without generating any probe traffic, thus avoiding any impact on the network and business systems. Additionally, it supports plug-in extensions, allowing for easy integration with other passive identification tools, and is implemented based on open-source tools, eliminating the need to purchase commercial software and resulting in low deployment and maintenance costs.

[0028] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0029] The passive network asset discovery method provided in this application can be applied to computer devices (electronic devices). The computer device can be a server or a terminal. The server can be a single server or a server cluster composed of multiple servers. This application does not specifically limit this. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices.

[0030] Taking a computer device as an example, Figure 1 A block diagram of a server is shown, such as Figure 1 As shown, the server may include a processor and memory connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. When the computer program is executed by the processor, it implements a passive network asset discovery method.

[0031] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the server to which the present application is applied. Optionally, the server may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0032] It should be noted that the execution subject of the embodiments of this application can be a computer device or a passive network asset discovery device. The following method embodiments will be described with a computer device as the execution subject.

[0033] Figure 2 This is a flowchart illustrating the steps of a passive network asset discovery method provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps: Step 202: Connect the raw network traffic to the Suricata network threat detection engine and generate the target data stream through the Suricata network threat detection engine.

[0034] The target data stream can include application layer data streams and network layer data streams. Suricata is a high-performance, open-source network threat detection engine that can operate in multiple modes, such as intrusion detection systems, intrusion prevention systems, or network security monitoring, and has become one of the mainstream network detection and response solutions in the industry. Suricata's core design philosophy is multi-threaded parallel processing and deep protocol parsing. Its architecture breaks through the performance bottleneck of traditional single-threaded detection engines, possessing powerful application layer protocol parsing capabilities. Furthermore, it can perform deep parsing of various application layer protocols such as HTTP and DNS.

[0035] In some alternative embodiments, such as Figure 3 As shown, Figure 3 A flowchart illustrating the steps for generating a target data stream, as provided in this application embodiment, includes: Step 302: After the original network traffic is connected to the Suricata network threat detection engine via traffic mirroring, the data packet object is obtained.

[0036] Step 304: Parse the data packet object to obtain structured data packet information.

[0037] Step 306: Cache and reassemble multiple IP fragmented data packets to restore complete IP datagrams. After associating and sorting multiple complete IP datagrams, restore the complete network layer data stream.

[0038] Step 308: Extract bidirectional IP datagrams belonging to the same TCP connection from the network layer data stream based on the 5-tuple and associate them. Sort and reassemble the TCP segmented packets according to the sequence number to restore the complete application layer data stream.

[0039] The process involves acquiring raw network traffic via the switch's SPAN port or network TAP, and then mirroring this traffic to the Suricata network threat detection engine. Suricata captures network traffic through its listening interface, converting the raw packets received by the network card into data packet objects in memory, providing the data foundation for subsequent processing.

[0040] Next, the data packet object is parsed, which involves sequentially parsing the Ethernet frame header, VLAN tag (if any), MPLS tag (if any), IPv4 / IPv6 header, TCP / UDP / ICMP and other transport layer headers, extracting the protocol fields of each layer to form structured data packet information. This data packet information includes multiple IP fragmented data packets, each carrying the same IP identifier field.

[0041] For multiple IP fragmented data packets that have undergone IP fragmentation, Suricata can cache and reassemble multiple IP fragmented data packets belonging to the same original data packet based on information such as identifiers, fragment offsets, and fragmentation flags in the IP header, to restore the complete IP datagram and ensure the integrity of subsequent protocol parsing.

[0042] Next, using network layer flow characteristics such as source IP address, destination IP address, and IP protocol number as unique indexes, complete IP datagrams with identical source IP, destination IP, and protocol number can be identified as belonging to the same network layer data stream. Then, according to the time sequence of packet capture, the associated IP datagrams are arranged in order to form an ordered and complete sequence of IP datagrams, thus restoring the network layer data stream.

[0043] Based on the aforementioned network layer data stream, IP datagrams carrying the TCP protocol are extracted. Using the 5-tuple as a unique index, the TCP session state is identified and maintained. Bidirectional IP datagrams belonging to the same TCP connection are associated and bound to establish a TCP session context. For TCP segments associated with the same TCP session, the TCP segment messages are sorted and reassembled according to the sequence number in the TCP header to obtain a complete and continuous application layer business data sequence, thus restoring the complete application layer data stream.

[0044] Bidirectional IP datagrams refer to a set of IP datagrams belonging to the same network communication session and transmitted separately in the request and response directions. They include IP packets in both the client-to-server and server-to-client directions, serving as the fundamental carrier for TCP full-duplex communication and application-layer request-response interactions. During stream reassembly, bidirectional IP datagrams must be uniformly associated, sorted, and processed to reconstruct the complete application-layer data stream. The aforementioned five-tuple includes source IP, destination IP, source port, destination port, and protocol, after which the parallel fingerprint extraction process can proceed.

[0045] Step 204: Process the network layer data stream using the P0F fingerprinting tool to extract the network layer fingerprint information; simultaneously process the application layer data stream using the Suricata network threat detection engine to extract the application layer fingerprint information.

[0046] The network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection results, link type, and cache validity period. The application layer fingerprint information includes service type, version information, domain name, and certificate characteristics. The NAT detection results, also known as network address translation detection results, are used to characterize network address translation rules and external network penetration capabilities, and can include NAT type, external network mapped address, mapped port, and penetration feasibility.

[0047] Furthermore, p0f is a classic passive operating system fingerprinting tool that identifies the operating system type of a remote host by analyzing the structural characteristics of TCP / IP packets. Its identification process is entirely passive, sending no probe packets to the target, thus possessing extremely high stealth capabilities. In this application, p0f is deeply integrated with Suricata, undertaking the core function of network layer operating system fingerprinting.

[0048] In some alternative embodiments, such as Figure 4 As shown, Figure 4 A flowchart illustrating the steps for extracting network layer fingerprint information, as provided in this application embodiment, includes: Step 402: If the current packet is determined to be a TCP handshake packet based on the network layer data flow, perform feature extraction on the TCP handshake packet to obtain a feature vector.

[0049] Step 404: Compare the feature vector with the preset p0f fingerprint database to obtain the network layer fingerprint information.

[0050] If the current packet is a TCP handshake packet (i.e., TCP Flags are SYN or SYN+ACK), the network layer fingerprint extraction process is triggered; if it is not a TCP handshake packet, the p0f network layer fingerprint extraction process is skipped.

[0051] Therefore, when it is determined that the current packet is a TCP handshake packet based on the network layer data flow, feature extraction can be performed on the TCP handshake packet to obtain a feature vector. This feature vector may include, but is not limited to, the initial time to live (TTL) value of the IP header, the window size of the TCP header, the list and order of TCP options, the Don't Fragment (DF) flag of the IP header, the value of the IP identifier, the total length of the data packet, and the maximum segment size (MSS).

[0052] The initial time to live value can be estimated by rounding up the actual received value. The TCP option list and its order include the maximum segment size (MSS), selective acknowledgment (SACK), timestamps (TS), and no-operation (NOP).

[0053] In some optional embodiments, before feature extraction, historical fingerprint information corresponding to TCP handshake packets can be queried in a preset fingerprint cache. If historical fingerprint information exists in the preset fingerprint cache and has not expired, the historical fingerprint information is used as network layer fingerprint information, and subsequent fingerprint matching steps are skipped. If there is no record in the cache or it has expired, fingerprint matching continues.

[0054] In the fingerprint matching process, the feature vector can be compared with a preset P0F fingerprint database to obtain the network layer fingerprint information. Optionally, the matching process can adopt a hierarchical strategy, that is, first perform exact matching to find fingerprint records that completely match the features; if exact matching fails, perform fuzzy matching, allowing reasonable deviations in some fields; if all matches fail, the operating system is marked as unknown and the feature vector is recorded.

[0055] In some optional embodiments, network layer fingerprint information can be stored in a preset fingerprint cache in a first preset format. The first preset format includes operating system type, operating system version, network hop count distance, NAT detection result, link type, and cache validity period. Subsequent traffic from the same IP address can directly reuse this result, avoiding duplicate calculations. For example, the cache validity period can be determined by setting the TTL to 3600 seconds.

[0056] In some alternative embodiments, the network layer fingerprint information is output to a preset fingerprint association module in a second preset format. The second preset format includes a 5-tuple, operating system type, operating system version, network hop count distance, whether it is behind a NAT, link type, and fingerprint matching confidence. For example, the link type can be Ethernet, Digital Subscriber Line (DSL), etc. Whether it is behind a NAT is used to determine whether the network device is in a local area network (LAN) environment and whether address translation is performed through a NAT gateway, thus distinguishing between LAN devices and devices directly connected to the public network.

[0057] Application layer fingerprint extraction can be performed in parallel with network layer fingerprint extraction. In some optional embodiments, the application layer data stream is processed based on the Suricata network threat detection engine to extract application layer fingerprint information, including: After identifying the corresponding application layer protocol type based on the port number and packet characteristics in the application layer data stream, if the HTTP protocol is identified, the Suricata network threat detection engine parses the HTTP request and response messages to extract the application layer fingerprint information.

[0058] Based on port number and packet characteristics, Suricata identifies the application layer protocol type of the current traffic. Suricata supports automatic identification of multiple protocols including HTTP, TLS / SSL, DNS, SMB, FTP, SSH, and SMTP, providing a basis for subsequent targeted parsing. For traffic identified as HTTP, Suricata parses HTTP request and response messages, extracting the following application layer fingerprint information: the Server field in the response header (identifying server type and version), the User-Agent field in the request header (identifying client type and version), framework identifier fields such as X-Powered-By in the response header, HTTP status codes, URIs, and other information (to assist in determining the service type).

[0059] If the TLS / SSL protocol is identified, the Suricata network threat detection engine extracts plaintext metadata from the handshake phase without decrypting the data, thus obtaining application-layer fingerprint information. Specifically, for encrypted traffic identified as TLS / SSL, Suricata extracts the following plaintext metadata from the handshake phase without decryption: server name indicator field (identifying the server domain name), server certificate information (including issuer, subject, validity period, public key algorithm, etc.), a list of supported cipher suites, JA3 / JA3S fingerprint (calculated based on TLS handshake features), and the Transport Layer Security (TLS) version number. The TLS handshake features are the set of key characteristics carried in the client-server exchange messages during the TLS encrypted communication establishment phase, including protocol configuration, encryption algorithm, and extended capabilities.

[0060] If the DNS protocol is identified, the Suricata network threat detection engine parses the query and response messages to extract application-layer fingerprint information. Specifically, for DNS traffic, Suricata parses the query and response messages to extract: the DNS query domain name, the A / AAAA record (domain name resolution result) in the DNS response, the query type (A, MX, TXT, etc.), and authoritative name server information. The authoritative name server information is the set of parameters related to the server that has the final resolution management authority for the specified domain name and stores the official, genuine resolution records for that domain name within the DNS domain name resolution system.

[0061] If SSH is identified, the Suricata network threat detection engine analyzes the protocol handshake phase information to extract application-layer fingerprint information. Specifically, for SSH traffic, Suricata analyzes the following handshake phase information: SSH server version string, SSH client version string, key exchange algorithm, and HASSH fingerprint (calculated based on SSH handshake features). The HASSH fingerprint uniquely identifies the SSH terminal's program implementation and version information, serving as a crucial feature for asset identification and behavior detection in encrypted traffic scenarios. The SSH handshake features are the algorithm negotiation characteristics of the plaintext interaction during the SSH protocol handshake phase, specifically including the key exchange algorithm, encryption algorithm, MAC algorithm, and compression algorithm.

[0062] Optionally, for other application layer protocol types, key information from the traffic of other protocols can be extracted as the application layer fingerprint information of that protocol.

[0063] In some optional embodiments, the application layer fingerprint information can be output to the preset fingerprint association module in a third preset format; wherein, the third preset format includes a 5-tuple, service type, version information, domain name and certificate characteristics.

[0064] Step 206: Obtain other fingerprint information and fingerprint matching confidence scores corresponding to the current asset, and generate asset discovery results based on network layer fingerprint information, application layer fingerprint information, other fingerprint information and fingerprint matching confidence scores.

[0065] Other fingerprint information includes discovery time, event type, and 5-tuple. After extracting the network layer fingerprint information and application layer fingerprint information, the operating system fingerprint and application layer fingerprint from the same network session (source IP, destination IP, source port, destination port, and protocol 5-tuple are completely identical) can be associated and merged to ensure that the bidirectional fingerprint information of a single session is completely consistent.

[0066] For fingerprint information of the same IP address appearing in different sessions, aggregate analysis is performed. For example, the same server IP may provide HTTP and HTTPS services in different sessions, and the application layer fingerprint information of multiple sessions needs to be merged into a complete service profile of that IP.

[0067] Next, based on the role in the TCP connection establishment process (initiating SYN is the client, responding with SYN+ACK is the server) and the port number range (well-known ports and small ports are the server; otherwise, it is the client), the role of the asset in the current session can be determined. The role can be any one of the following: client (using a temporary port to access the service), server (using a fixed port to provide the service), or peer (two-way communication in a P2P scenario).

[0068] The fingerprint matching confidence level is calculated based on the following factors: the accuracy of the fingerprint matching (i.e., the accuracy of an exact match is higher than that of a fuzzy match), the reliability of the fingerprint information source, the consistency of multiple recognition results, and the freshness of the fingerprint, which is the time interval since the last recognition.

[0069] Ultimately, the system can output complete asset object data after association as the asset discovery result, which includes the complete fingerprint information and confidence score of the asset, including discovery time, event type, five-tuple, operating system fingerprint, application layer fingerprint, asset role, and confidence score.

[0070] This application provides a passive network asset discovery method, which includes: connecting raw network traffic to the Suricata network threat detection engine and generating a target data stream through the Suricata network threat detection engine, wherein the target data stream includes an application layer data stream and a network layer data stream; processing the network layer data stream based on a P0F fingerprinting tool to extract network layer fingerprint information; and simultaneously processing the application layer data stream based on the Suricata network threat detection engine to extract application layer fingerprint information, wherein the network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection result, link type, and cache validity period; and the application layer fingerprint information includes service type, version information, domain name, and certificate characteristics; obtaining other fingerprint information and fingerprint matching confidence levels corresponding to the current asset; and generating an asset discovery result based on the network layer fingerprint information, application layer fingerprint information, other fingerprint information, and fingerprint matching confidence levels, wherein the other fingerprint information includes discovery time, event type, and a 5-tuple. This solution embeds P0F's passive asset identification capabilities into the Suricata processing pipeline as a dynamic library, fully leveraging Suricata's high-performance traffic processing and application parsing capabilities to improve the performance and accuracy of asset discovery. Furthermore, by simultaneously extracting network layer and application layer fingerprint information based on Suricata flow reassembly, it achieves multi-dimensional asset information collection in a single packet processing step, enriching asset attributes. Based on real-time traffic analysis, it can promptly detect new and changed assets, and discover assets in a completely passive manner without generating any probe traffic, thus avoiding any impact on the network or business systems. Additionally, it supports plug-in extensions, allowing for easy integration with other passive identification tools, and is implemented using open-source tools, eliminating the need to purchase commercial software and resulting in low deployment and maintenance costs.

[0071] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0072] Figure 5 This is a structural block diagram of a passive network asset discovery device provided in an embodiment of this application.

[0073] like Figure 5 As shown, the passive network asset discovery device 500 includes: The first generation module 502 is used to connect the raw network traffic to the Suricata network threat detection engine and generate a target data stream through the Suricata network threat detection engine; wherein, the target data stream includes an application layer data stream and a network layer data stream.

[0074] The extraction module 504 is used to process the network layer data stream based on the P0F fingerprinting tool to extract network layer fingerprint information; at the same time, it processes the application layer data stream based on the Suricata network threat detection engine to extract application layer fingerprint information. The network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period, and the application layer fingerprint information includes service type, version information, domain name and certificate characteristics.

[0075] The second generation module 506 is used to obtain other fingerprint information and fingerprint matching confidence level corresponding to the current asset, and generate asset discovery results based on network layer fingerprint information, application layer fingerprint information, other fingerprint information and fingerprint matching confidence level; wherein, other fingerprint information includes discovery time, event type and quintuple.

[0076] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here. Each module in the above passive network asset discovery apparatus can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations of each module.

[0077] In one embodiment of this application, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps: The raw network traffic is fed into the Suricata network threat detection engine, and the target data stream is generated by the Suricata network threat detection engine; the target data stream includes application layer data stream and network layer data stream. The network layer data stream is processed using a P0F fingerprinting tool to extract network layer fingerprint information; simultaneously, the application layer data stream is processed using the Suricata network threat detection engine to extract application layer fingerprint information. The network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection results, link type, and cache validity period; the application layer fingerprint information includes service type, version information, domain name, and certificate characteristics. Obtain other fingerprint information and fingerprint matching confidence scores corresponding to the current asset, and generate asset discovery results based on network layer fingerprint information, application layer fingerprint information, other fingerprint information and fingerprint matching confidence scores; among them, other fingerprint information includes discovery time, event type and quintuple.

[0078] In one embodiment of this application, the processor further performs the following steps when executing the computer program: When it is determined that the current packet is a TCP handshake packet based on the network layer data flow, feature extraction is performed on the TCP handshake packet to obtain a feature vector; The feature vector is compared with the preset p0f fingerprint database to obtain the network layer fingerprint information.

[0079] In one embodiment of this application, the processor further performs the following steps when executing the computer program: Search the preset fingerprint cache for historical fingerprint information corresponding to the TCP handshake packet; If historical fingerprint information exists in the preset fingerprint cache and the historical fingerprint information has not expired, then the historical fingerprint information will be used as the network layer fingerprint information.

[0080] In one embodiment of this application, the processor further performs the following steps when executing the computer program: The network layer fingerprint information is stored in a preset fingerprint cache in a first preset format; wherein, the first preset format includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period; The network layer fingerprint information is output to the preset fingerprint association module in a second preset format; wherein, the second preset format includes a 5-tuple, operating system type, operating system version, network hop distance, whether it is behind NAT, link type, and fingerprint matching confidence.

[0081] In one embodiment of this application, the processor further performs the following steps when executing the computer program: After identifying the corresponding application layer protocol type based on the port number and packet characteristics in the application layer data stream, if the HTTP protocol is identified, the Suricata network threat detection engine is used to parse the HTTP request and response messages and extract the application layer fingerprint information. If the TLS / SSL protocol is identified, the Suricata network threat detection engine extracts the plaintext metadata of the handshake phase without decrypting the data, and extracts the application layer fingerprint information. If the DNS protocol is identified, the query and response messages are parsed using the Suricata network threat detection engine to extract application layer fingerprint information; If SSH is identified, the Suricata network threat detection engine parses the information from the protocol handshake phase to extract the application layer fingerprint information.

[0082] In one embodiment of this application, the processor further performs the following steps when executing the computer program: The application layer fingerprint information is output to the preset fingerprint association module in a third preset format; wherein, the third preset format includes a five-tuple, service type, version information, domain name and certificate characteristics.

[0083] In one embodiment of this application, the processor further performs the following steps when executing the computer program: After the raw network traffic is fed into the Suricata network threat detection engine via traffic mirroring, packet objects are obtained. The data packet object is parsed to obtain structured data packet information, which includes multiple IP fragment data packets. Multiple IP fragmented data packets are cached and reassembled to restore a complete IP datagram; The target data stream is obtained by reconstructing the data based on the complete IP datagram, according to the 5-tuple and sequence number.

[0084] The computer device provided in this application embodiment has a similar implementation principle and technical effect to the above method embodiment, and will not be described again here.

[0085] In one embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it performs the following steps: The raw network traffic is fed into the Suricata network threat detection engine, and the target data stream is generated by the Suricata network threat detection engine; the target data stream includes application layer data stream and network layer data stream. The network layer data stream is processed using a P0F fingerprinting tool to extract network layer fingerprint information; simultaneously, the application layer data stream is processed using the Suricata network threat detection engine to extract application layer fingerprint information. The network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection results, link type, and cache validity period; the application layer fingerprint information includes service type, version information, domain name, and certificate characteristics. Obtain other fingerprint information and fingerprint matching confidence scores corresponding to the current asset, and generate asset discovery results based on network layer fingerprint information, application layer fingerprint information, other fingerprint information and fingerprint matching confidence scores; among them, other fingerprint information includes discovery time, event type and quintuple.

[0086] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps: When it is determined that the current packet is a TCP handshake packet based on the network layer data flow, feature extraction is performed on the TCP handshake packet to obtain a feature vector; The feature vector is compared with the preset p0f fingerprint database to obtain the network layer fingerprint information.

[0087] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps: Search the preset fingerprint cache for historical fingerprint information corresponding to the TCP handshake packet; If historical fingerprint information exists in the preset fingerprint cache and the historical fingerprint information has not expired, then the historical fingerprint information will be used as the network layer fingerprint information.

[0088] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps: The network layer fingerprint information is stored in a preset fingerprint cache in a first preset format; wherein, the first preset format includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period; The network layer fingerprint information is output to the preset fingerprint association module in a second preset format; wherein, the second preset format includes a 5-tuple, operating system type, operating system version, network hop distance, whether it is behind NAT, link type, and fingerprint matching confidence.

[0089] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps: After identifying the corresponding application layer protocol type based on the port number and packet characteristics in the application layer data stream, if the HTTP protocol is identified, the Suricata network threat detection engine is used to parse the HTTP request and response messages and extract the application layer fingerprint information. If the TLS / SSL protocol is identified, the Suricata network threat detection engine extracts the plaintext metadata of the handshake phase without decrypting the data, and extracts the application layer fingerprint information. If the DNS protocol is identified, the query and response messages are parsed using the Suricata network threat detection engine to extract application layer fingerprint information; If SSH is identified, the Suricata network threat detection engine parses the information from the protocol handshake phase to extract the application layer fingerprint information.

[0090] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps: The application layer fingerprint information is output to the preset fingerprint association module in a third preset format; wherein, the third preset format includes a five-tuple, service type, version information, domain name and certificate characteristics.

[0091] In one embodiment of this application, the computer program, when executed by a processor, further performs the following steps: After the raw network traffic is fed into the Suricata network threat detection engine via traffic mirroring, packet objects are obtained. The data packet object is parsed to obtain structured data packet information, which includes multiple IP fragment data packets. Multiple IP fragmented data packets are cached and reassembled to restore a complete IP datagram; The target data stream is obtained by reconstructing the data based on the complete IP datagram, according to the 5-tuple and sequence number.

[0092] The computer-readable storage medium provided in this embodiment is similar in principle and technical effect to the method embodiment described above, and will not be repeated here.

[0093] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0094] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0095] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A passive network asset discovery method, characterized in that, The method includes: The raw network traffic is connected to the Suricata network threat detection engine, and the target data stream is generated through the Suricata network threat detection engine; wherein, the target data stream includes application layer data stream and network layer data stream; The network layer data stream is processed using a P0F fingerprinting tool to extract network layer fingerprint information; simultaneously, the application layer data stream is processed using the Suricata network threat detection engine to extract application layer fingerprint information; wherein, the network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period, and the application layer fingerprint information includes service type, version information, domain name and certificate characteristics; Obtain other fingerprint information and fingerprint matching confidence scores corresponding to the current asset, and generate asset discovery results based on the network layer fingerprint information, the application layer fingerprint information, the other fingerprint information and the fingerprint matching confidence scores; wherein, the other fingerprint information includes discovery time, event type and quintuple.

2. The method according to claim 1, characterized in that, The process of processing the network layer data stream using the P0F fingerprint recognition tool to extract network layer fingerprint information includes: If the current packet is determined to be a TCP handshake packet based on the network layer data stream, feature extraction is performed on the TCP handshake packet to obtain a feature vector; The feature vector is compared with a preset p0f fingerprint database to obtain the fingerprint information of the network layer.

3. The method according to claim 2, characterized in that, The method further includes: Query the historical fingerprint information corresponding to the TCP handshake packet in the preset fingerprint cache; If the historical fingerprint information exists in the preset fingerprint cache and the historical fingerprint information has not expired, then the historical fingerprint information is used as the network layer fingerprint information.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: The network layer fingerprint information is stored in a preset fingerprint cache in a first preset format; wherein, the first preset format includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period; The network layer fingerprint information is output to the preset fingerprint association module in a second preset format; wherein, the second preset format includes a 5-tuple, operating system type, operating system version, network hop distance, whether it is behind NAT, link type, and fingerprint matching confidence.

5. The method according to claim 2, characterized in that, The process of processing the application layer data stream based on the Suricata network threat detection engine to extract application layer fingerprint information includes: After identifying the corresponding application layer protocol type based on the port number and data packet characteristics in the application layer data stream, if the HTTP protocol is identified, the Suricata network threat detection engine parses the HTTP request and response messages to extract the application layer fingerprint information. If the TLS / SSL protocol is identified, the plaintext metadata of the handshake phase is extracted through the Suricata network threat detection engine without decrypting the data, and the application layer fingerprint information is extracted. If the DNS protocol is identified, the application layer fingerprint information is extracted by parsing the query and response messages through the Suricata network threat detection engine. If SSH is identified, the application layer fingerprint information is extracted by parsing the protocol handshake phase information through the Suricata network threat detection engine.

6. The method according to claim 5, characterized in that, The method further includes: The application layer fingerprint information is output to the preset fingerprint association module in a third preset format; wherein, the third preset format includes a 5-tuple, service type, version information, domain name and certificate characteristics.

7. The method according to any one of claims 1-3, characterized in that, The step of connecting the raw network traffic to the Suricata network threat detection engine and generating the target data stream through the Suricata network threat detection engine includes: After the raw network traffic is fed into the Suricata network threat detection engine via traffic mirroring, packet objects are obtained. The data packet object is parsed to obtain structured data packet information; wherein, the data packet information includes multiple IP fragment data packets; The multiple IP fragmented data packets are cached and reassembled to restore the complete IP datagram. After the multiple complete IP datagrams are correlated and sorted, the complete network layer data stream is restored. Based on the five-tuple, bidirectional IP datagrams belonging to the same TCP connection in the network layer data stream are extracted and associated. The TCP segmented packets are sorted and reassembled according to the sequence number to restore the complete application layer data stream.

8. A passive network asset discovery device, characterized in that, The device includes: The first generation module is used to connect the raw network traffic to the Suricata network threat detection engine and generate a target data stream through the Suricata network threat detection engine; wherein, the target data stream includes an application layer data stream and a network layer data stream; The extraction module is used to process the network layer data stream based on the P0F fingerprinting tool to extract network layer fingerprint information; and simultaneously to process the application layer data stream based on the Suricata network threat detection engine to extract application layer fingerprint information; wherein, the network layer fingerprint information includes operating system type, operating system version, network hop count distance, NAT detection result, link type and cache validity period, and the application layer fingerprint information includes service type, version information, domain name and certificate characteristics; The second generation module is used to obtain other fingerprint information and fingerprint matching confidence level corresponding to the current asset, and generate asset discovery results based on the network layer fingerprint information, the application layer fingerprint information, the other fingerprint information and the fingerprint matching confidence level; wherein, the other fingerprint information includes discovery time, event type and 5-tuple.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the passive network asset discovery method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the passive network asset discovery method as described in any one of claims 1-7.

Images