Intranet domain asset identification method and device based on traffic statistical characteristics

Abstract

The application discloses an intranet domain asset identification method and device based on traffic statistical characteristics, and the method comprises the following steps: S1, acquiring traffic data of an intranet domain; S2, sequentially performing standardization processing, filtering processing and preprocessing on the traffic data to obtain useful data; S3, respectively performing screening filtering on the useful data based on source IP and destination IP to obtain first asset information and second asset information; and S4, merging the first asset information and the second asset information to obtain intranet domain asset information. Through the preprocessing such as pre-filtering on the traffic data, and then based on the communication protocol, the source IP and the destination IP are screened and filtered through corresponding identification conditions, the intranet domain asset is identified to obtain asset data information, and the accuracy of the identification result is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for identifying intranet domain assets based on traffic statistics characteristics. Background Technology

[0002] In the current network environment, many network security management actions are required to better ensure network security, such as network security monitoring, vulnerability reporting, and network threat awareness. However, to better manage network security, the first step is to identify network domain assets. Currently, methods for identifying network domain assets are mainly divided into active and passive detection. Active asset identification involves proactively probing network hosts, port scanning, and monitoring hardware characteristics and version information. Passive asset identification involves collecting traffic from the target network and analyzing the characteristics of application layer protocol data packets such as HTTP, FTP, and SMTP to passively detect network asset information. Networks are further divided into intranets and extranets. Compared to extranets, intranets have a smaller computer interconnection range and fewer IP addresses, thus having fewer asset IPs and are more suitable for passive detection. The more intranet asset information obtained from network information such as protocols, the more complete the intranet asset information, and the more effective the security management of the intranet. Currently, most internal network asset identification methods use matching rules in the IP header, such as total length, ID, and fragmentation (DF), to match network information data like protocols. However, with the increasing variety of internal network assets and the growing demand for precise identification, current rule-based asset identification technologies suffer from poor accuracy. Summary of the Invention

[0003] The purpose of this invention is to design an intranet domain asset identification method and device based on traffic statistics characteristics to solve the above problems.

[0004] The present invention achieves the above objectives through the following technical solutions:

[0005] Methods for identifying intranet domain assets based on traffic statistics characteristics include:

[0006] S1. Obtain traffic data for the internal network domain;

[0007] S2. Perform standardization, filtering, and preprocessing on the traffic data in sequence to obtain useful data;

[0008] S3. Based on the source IP and destination IP, filter the useful data to obtain the first asset information and the second asset information.

[0009] S4. Merge the first asset information and the second asset information to obtain the intranet domain asset information.

[0010] The intranet domain asset identification device based on traffic statistics features includes:

[0011] Storage; storage is used to store computer programs;

[0012] Processor; The processor is used to execute computer programs, and when executing computer programs stored in memory, it implements the intranet domain asset identification method based on traffic statistics characteristics as described above.

[0013] The beneficial effects of this invention are as follows: by pre-filtering traffic data and then filtering the source IP and destination IP based on the communication protocol using corresponding identification conditions, the internal network domain assets are identified and asset data information is obtained, which effectively improves the accuracy of the identification results. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the intranet domain asset identification method based on traffic statistics features according to the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0016] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0017] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0018] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0019] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0020] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] Methods for identifying intranet domain assets based on traffic statistics characteristics include:

[0023] S1. Obtain traffic data for the internal network domain;

[0024] S2. Perform standardization, filtering, and preprocessing on the traffic data in sequence to obtain useful data;

[0025] Standardization processing: unifying the fields, units, and types of traffic data;

[0026] Filtration processes include:

[0027] (1) Determine the protocol format of the traffic data; if it is TCP protocol, proceed to (2); if it is UDP protocol, proceed directly to (3);

[0028] (2) Determine whether the traffic data is normal traffic. If not, remove it; otherwise, proceed to (3).

[0029] (3) Determine whether the device has been missed during the missed scan. If so, the response packet is 0 and the device is removed; otherwise, all devices are retained.

[0030] Preprocessing includes data cleaning and data transformation.

[0031] S3. Based on the source IP and destination IP, filter the useful data to obtain the first asset information and the second asset information; specifically:

[0032] Filtering useful data based on source IP includes:

[0033] 1) Analyze whether the number of data packets containing useful data is greater than 'a'. If so, keep them; otherwise, discard them.

[0034] 2) Analyze whether the number of useful data bytes sent is greater than b. If so, keep it; otherwise, discard it.

[0035] 3) Useful data retained through steps 1) and 2) is used as the first asset information;

[0036] Filtering useful data based on destination IP includes:

[0037] ① Analyze whether the number of received data packets containing useful data is greater than c. If so, keep them; otherwise, discard them.

[0038] ② Analyze whether the number of useful data bytes received is greater than d. If so, retain it; otherwise, discard it.

[0039] ③ Useful data retained through steps ① and ② is used as secondary asset information.

[0040] S4. Merge the first asset information and the second asset information to obtain the intranet domain asset information;

[0041] In S3, a>0, b>0; when the traffic data is based on the TCP protocol, c>2, d>64; when the traffic data is based on the UDP protocol, c>0, d>0.

[0042] The intranet domain asset identification device based on traffic statistics features includes:

[0043] Storage; storage is used to store computer programs;

[0044] Processor; The processor is used to execute computer programs, and when executing computer programs stored in memory, it implements the intranet domain asset identification method based on traffic statistics characteristics as described above.

[0045] The working principle of the intranet domain asset identification method based on traffic statistics features in this invention is as follows:

[0046] Data collection

[0047] Prepare the environment for asset identification, collect relevant traffic data from the intranet domain using data collection devices, and commonly use network probes to obtain information.

[0048] Data import

[0049] Initialize the operating environment of the intranet domain asset identification device based on traffic statistics characteristics, and import the intranet domain data information collected by the collection device through Kafka, because Kafka has the characteristics of high performance, low latency, high availability, mature toolchain and mature ecosystem.

[0050] Data processing

[0051] Data standardization processing: The imported traffic data is standardized into the required data type. Data packets may be bits, bytes, etc., so the unit of measurement used here is numbers. In the TCP protocol, data packets are usually called "segments," which must be less than or equal to 1460 bytes. In the UDP protocol, data packets are usually called datagrams, and their size is not limited. The size of each packet can be specified by the method. However, for communication to be achieved, at least one data packet must be sent from the source IP to receive the data packets. Therefore, the unit of measurement of the data packets is converted accordingly. The payload may be bytes, kilobytes (KB), megabytes (MB), etc., so the unit of measurement used here is bytes, and the corresponding conversion is performed. The time type may be time with time zone (ZoneDateTime), local date (LocalDate), local time (LocalTime), etc., all of which are converted into timestamps.

[0052] Eliminate data inconsistencies and improve data quality.

[0053] Filtration process:

[0054] For TCP-based traffic data, if normal traffic data is missed by the vulnerability scanning device, the response packet is 0 and it is filtered; if the response packet is greater than 0, it is retained.

[0055] For TCP-based traffic data, if it is normal traffic data that has not been scanned by the vulnerability scanner and the response packet is greater than or equal to 0, it will be retained.

[0056] For TCP-based traffic data, any abnormal traffic will be filtered out.

[0057] For UDP-based traffic data, if the traffic data has been missed during a vulnerability scan, a response packet of 0 is filtered out; a response packet greater than 0 is retained.

[0058] Traffic data based on the UDP protocol that has not been missed by the missed scan device will be retained.

[0059] Normal traffic is legitimate network traffic data that meets expectations; abnormal traffic is network traffic data that does not meet expectations and represents invalid communication.

[0060] Missed traffic is filtered by a whitelist, allowing the missed traffic scanner or other devices to filter out the traffic data. Because some IPs do not exist, yet generate a large amount of data due to the scanning activity of the missed traffic scanner, the scanner can improve data accuracy by omitting these IPs.

[0061] Data preprocessing: Data cleaning handles outliers. For example, if a data packet contains a value of -1, it will be padded with zeros; if a load contains a value of -1, it will also be padded with zeros, etc. Data transformation converts the data to the form required for analysis, such as converting time to days (daytime) and tens of minutes (tentime), and converting the maximum time (MAX(time)) to the current processing time (recent_time).

[0062] Intranet Domain Asset Identification

[0063] First, assets are identified by their source IP addresses. Then, the pre-processed data is further processed, and filtering is performed based on the number of data packets sent, 'a'. This is based on the principles of TCP and UDP protocols. TCP's "three-way handshake" mechanism involves sending three data packets, so the number of data packets sent is greater than 0. UDP, being connectionless, only requires sending information in each communication, so it sends at least one data packet. Therefore, 'a' is greater than 0.

[0064] Then, also based on the source IP, data is filtered by the number of bytes sent, 'b'. Since data is sent with every communication using both TCP and UDP protocols, 'b' is greater than 0.

[0065] Next, assets are identified via the destination IP address, and filtering is performed using the number of received data packets (c) and the number of received bytes (d) as conditions. Based on the TCP protocol's "three-way handshake" mechanism, establishing a connection requires receiving SYN and ACK packets; communication is only possible after a session connection is established, so c is greater than 2. Because the MTU (Maximum Transmission Unit) at the data link layer affects the size of transmitted data, its minimum value is limited to 64 bytes. 64 bytes only includes the IP packet size, Ethernet header, trailer checksum, and FCS, without data information. Therefore, d is greater than 64. Based on the UDP protocol principle, UDP is connectionless; to achieve communication, it receives data packets from the source IP, and at least one data packet must be sent. Therefore, c is greater than 0. Because in intranet domain communication using the UDP protocol, data may not meet the minimum MTU requirement (64 bytes), and the MAC sublayer of the link layer will pad the data with zeros, so d is greater than 0.

[0066] Data filtered based on source IP and destination IP are merged to identify assets.

[0067] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A method for identifying intranet domain assets based on traffic statistics features, characterized in that, include: S1. Obtain traffic data for the internal network domain; S2. Perform standardization, filtering, and preprocessing on the traffic data in sequence to obtain useful data; S3. Based on the source IP and destination IP, filter the useful data to obtain the first asset information and the second asset information. Specifically, it includes: Filtering useful data based on source IP includes: 1) Analyze whether the number of data packets containing useful data is greater than 'a'. If so, keep them; otherwise, discard them. 2) Analyze whether the number of useful data bytes sent is greater than b. If so, keep it; otherwise, discard it. 3) Useful data retained through steps 1) and 2) is used as the first asset information; Filtering useful data based on destination IP includes: ① Analyze whether the number of received data packets containing useful data is greater than c. If so, keep them; otherwise, discard them. ② Analyze whether the number of useful data bytes received is greater than d. If so, retain it; otherwise, discard it. ③ Useful data retained through steps ① and ② is used as secondary asset information; Where a>0, b>0, c>2, d>64 when the traffic data is based on the TCP protocol; and c>0, d>0 when the traffic data is based on the UDP protocol. S4. Merge the first asset information and the second asset information to obtain the intranet domain asset information.

2. The intranet domain asset identification method based on traffic statistics features according to claim 1, characterized in that, In S2: Standardization processing: unifying the fields, units, and types of traffic data; Filtration processes include: (1) Determine the protocol format of the traffic data; if it is TCP protocol, proceed to (2); if it is UDP protocol, proceed directly to (3). (2) Determine whether the traffic data is normal traffic. If not, remove it; otherwise, proceed to (3). (3) Determine whether the device has been missed during the missed scan. If so, the response packet is 0 and the device is removed; otherwise, all devices are retained. Preprocessing includes data cleaning and data transformation.

3. An intranet domain asset identification device based on traffic statistics features, characterized in that: include: Storage; Storage is used to store computer programs; processor; The processor is used to execute a computer program, and when executing the computer program in the storage, it implements the intranet domain asset identification method based on traffic statistics features as described in any one of claims 1-2.

Images