A method for discovering shadow assets in intranet

Abstract

The present invention discloses a method for discovering shadow assets in an intranet, comprising the following steps: step S1, establishing an asset information database; step S2, collecting an IP address information table; step S3, screening suspicious IP addresses; step S4, refining and saving suspicious IP addresses; step S5, analyzing and judging shadow assets; and step S6, generating analysis and judgment results. The present invention can realize automatic discovery and automatic analysis of shadow assets, solving the problem of heavy workload and incomplete grasp of asset information for IT system administrators of enterprises or organizations.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a method for discovering shadow assets in an intranet. Background Art

[0002] As the degree of enterprise informatization continues to increase, the IT systems of enterprises or organizations are bound to become increasingly complex. Managing the equipment, software, and services involved in the IT system in the form of ledgers will encounter cumbersome and easy omissions. Even when using an asset management system, it is still difficult to avoid omissions due to the complexity of asset sources, the diversity of asset data, and the instability of ownership. The "shadow assets" born in this context not only contain sensitive internal information of the enterprise, but also lack control, resulting in incomplete management of the exposure of enterprise information assets, which are easily targeted by external attackers and a springboard for breakthroughs, causing great harm to the network and information security of enterprises or organizations. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an intranet shadow asset discovery method, which can realize automatic discovery and automatic analysis of shadow assets, and solve the problem of large workload of asset sorting and incomplete grasp of asset information of IT system administrators of enterprises or organizations.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A method for discovering shadow assets in an intranet comprises the following steps:

[0006] Step S1, establish an asset information database, which specifically includes: organizing and storing the existing asset tables of the current enterprise or organization, including but not limited to fields such as host name, IP address, MAC address, domain name, open port, business load, asset responsible person, etc.

[0007] Step S2, collecting IP address information table, which specifically includes: sorting out the network architecture of the enterprise or organization, sorting out network devices and security devices including but not limited to firewalls, switches, routers, AC, AD, etc. with dynamic and real-time updated IP addresses, collecting and generating IP address information table. The IP address information table includes but is not limited to IP address, MAC address, source device, collection time and other information.

[0008] Step S3, screening suspicious IP addresses, specifically includes: comparing the IP address information table with the asset information database, screening IP address information that is not in the asset information database, and these IP addresses may be shadow assets.

[0009] Step S4, refining and saving suspicious IP addresses, includes the following steps:

[0010] Step S4.1, actively detect and scan the suspicious IP address for port opening status, operating system type, and business load status to obtain the port and corresponding service information.

[0011] Step S4.2, based on the scanning results, attempt to perform WEB page detection on the IP addresses that may have WEB services, and save the obtained relevant information.

[0012] Step S4.3, based on the scanning results, attempt to perform reverse domain name resolution on the IP addresses that may have WEB services, focusing on obtaining the domain name and related WEB page information.

[0013] Step S4.4, sort out all collected information, and store the IP address information table for which no WEB service exists in the historical information database.

[0014] Step S5, shadow asset analysis, specifically: establish a multi-dimensional analysis model, and call it according to different situations, including but not limited to:

[0015] Same-segment inference: searching the asset information database for assets with the same network segment, such as the same C segment or the same specified mask network segment;

[0016] Same-source inference: searching the asset information database for asset information with the same source, such as the same switch or the same VLAN;

[0017] Service speculation, searching the asset information database for asset information with similar port ranges and similar services;

[0018] Stack speculates that asset information with associated services is searched in the asset information database, such as LAMP (Linux+Apache+MySQL+PHP), ELK (ElasticSearch+Logstash+Kibana), RAILS (Ruby onRails+Apache+Linux+IIS+SQL), STACK (Spring Boot+Tomcat+Angular.js+Cassandra+Kafka), etc., which were not intended at the beginning of the design, but are often used in combination during the process of technological development;

[0019] Domain name speculation: searching the asset information database for asset information with WEB business load, similar domain names and titles;

[0020] Trend analysis, searching past IP address information tables in the historical information database, aggregating indicators including but not limited to online history, online time, online duration, etc.

[0021] Step S6, generating the analysis result, includes the following steps:

[0022] Step S6.1, based on the results of the analysis, the following are classified: ① Occasional requests. Refers to IP addresses that appear occasionally (such as external visitors surfing the Internet). The IP addresses behind them may not necessarily belong to the assets of the enterprise or organization. ② Shadow assets. Assets that actually exist but are not included in the asset table of the enterprise or organization.

[0023] Step S6.2: Issue a report containing the results of the analysis, describing the most likely responsible person for the shadow assets, the business system to which they belong, the reasons for the speculation, the credibility of the speculation, and other information for reference and confirmation by the IT system administrator.

[0024] The beneficial effects of the present invention are:

[0025] The intranet shadow asset discovery method provided by the present invention can realize automatic discovery and automatic analysis of shadow assets through steps S1 to S6, thereby solving the problem of heavy workload of asset sorting and incomplete asset information for IT system administrators of enterprises or organizations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The present invention is a flowchart of a method for discovering shadow assets in an intranet according to an embodiment of the present invention.

[0027] Figure 2 The present invention provides a flowchart of an access switch acquiring an ARP table in a method for discovering shadow assets in an intranet according to an embodiment of the present invention.

[0028] Figure 3 This is a detailed flowchart of suspicious IP address information of the intranet shadow asset discovery method according to an embodiment of the present invention.

[0029] Figure 4 This is a shadow analysis flow chart of the intranet shadow asset discovery method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention is described below in conjunction with the accompanying drawings. The specific implementation methods described herein are only used to illustrate and explain the present invention and are not used to limit the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solution of the present invention by ordinary technicians in this field should fall within the protection scope of the present invention.

[0031] like Figures 1 to 4 As shown, the intranet shadow asset discovery method of the embodiment of the present invention includes the following steps:

[0032] Step S1, establishing an asset information database, specifically:

[0033] By organizing the existing asset tables of the current enterprise or organization, an IP address table as shown in Table 1, a port table as shown in Table 2, and an asset information table as shown in Table 3 can be established and stored in the asset information database.

[0034] Table 1

[0035] ID Asset ID IP address MAC Address domain name 123 123abc 192.168.169.1 00-50-56-C0-00-01 … … … … … 2345 234abc 172.16.0.1 00-60-76-E3-00-02 example.com

[0036] Table 2

[0037] ID Asset ID Open Ports Bearer Services 1234 123abc 3306 mysql … … … … 2345 234abc 80 http

[0038] Table 3

[0039] ID Asset ID Hostname Responsible Person System 12345 123abc Database Server Zhang San CMS … … … … … 23456 234abc Main WEB Wang Wu website

[0040] Step S2, collecting IP address information tables, specifically: sorting out the network structure of the enterprise or organization, using access switches at all levels as asset discovery sources, confirming the network management IP address of the switch, obtaining OID information related to the ARP table from the switch through the SNMP protocol, and integrating multiple ARP tables to obtain the IP address information table. The overall process is as follows: Figure 2 As shown, its table structure is shown in Table 4.

[0041] Table 4

[0042]

[0043] Step S3, screening suspicious IP addresses, specifically:

[0044] Compare the "IP Address" and "IP" columns in Table 1 and Table 4:

[0045] If the "IP" column is not a subset of the "IP Address" column, it means that an unknown IP address has been found and the address has not been included in the asset information database by the IT system administrator. It may be a shadow asset.

[0046] Furthermore, possible shadow assets are pushed into the next process.

[0047] If the "IP" column is a subset of the "IP Address" column, it means that the IP address information table obtained through ARP collection this time is all included assets, and there are no shadow assets. The current process ends and waits for the next collection result.

[0048] Step S4, refining and saving suspicious IP addresses, includes the following steps:

[0049] Use scanning tools (such as Nmap) to perform port scanning and service detection on suspicious IP addresses, and further perform web page requests and reverse domain name resolution attempts on possible web service ports. The overall flow chart is as follows: Figure 3 shown.

[0050] For example, the results of an Nmap detection are as follows:

[0051]

[0052] If a web service is found in the scan results, it will start to request https: / / xx.xx.xx.xx (URL structure consisting of "protocol + IP address + port"). For example, the obtained web page is shown as follows:

[0053]

[0054] Reverse DNS (rDNS) is a service that converts IP addresses into domain names by querying PTR and CNAME records. It is usually used for spam filtering, network fault diagnosis, and access control policy formulation.

[0055] For example, querying the IP address "96.227.5.107" on the Internet rDNS service provider site may yield the following results:

[0056]

[0057] When a DNS server has been configured in the intranet, reverse domain name resolution can be used to confirm the domain name that may correspond to an IP address provided by a web service in the intranet.

[0058] Step S5, shadow asset analysis, specifically:

[0059] In the same paragraph, the asset information database is searched for assets with the same mask. When the mask is not specified, a 24-bit mask is used by default. For example, when the suspicious IP address is 192.168.1.1, the asset information obtained by the search is shown in Table 5:

[0060] Table 5

[0061] Suspicious IP Address Mask Search Scope 192.168.1.1 24 (default) 192.168.1.1 to 192.168.1.254 192.168.1.1 28(Custom) 192.168.1.1 to 192.168.1.14

[0062] Homologous inference: Search the asset information database for asset information with the same source switch or the same source VLAN. As shown in Table 4, the "Device Name" and "VLAN" fields in the ARP table indicate the device and VLAN that collected the current IP address. When asset information with the same mask cannot be found in the asset information database, asset information with the same source is used to determine the ownership of the suspicious IP address.

[0063] Service speculation searches for asset information with similar port opening and similar service load in the asset information database. For example, when an enterprise or organization deploys a MySQL database in an IT system, 192.168.1.2 is used as the address of the database server. At this time, the database server 192.168.1.2 may open port 22 for remote management by operation and maintenance personnel, and port 3306 for the MySQL database to provide external services. If the high-availability design idea is followed, there may be two database servers with the same configuration, one primary and one backup. For change management needs, a test environment with the same configuration may be built. In this way, the database server 192.168.1.2 may be cloned 3 to 4 times. The primary and backup database servers and the test environment servers have basically the same open ports and service loads except for the difference in IP addresses. Using this feature, when the obtained suspicious IP address has similar port opening and similar service load as an entry in the asset information database, there is reason to suspect that it is an asset belonging to the same system and the same administrator.

[0064] Stack inference searches for relevant asset information in the asset information database. In an embodiment of the present invention, the preset stack includes but is not limited to LAMP (Linux+Apache+MySQL+PHP), ELK (ElasticSearch+Logstash+Kibana), RAILS (Ruby on Rails+Apache+Linux+IIS+SQL), STACK (Spring Boot+Tomcat+Angular.js+Cassandra+Kafka), etc. Exemplarily, when the service detection for a suspicious IP address finds the Kibana service, it will try to search the asset information database for the ElasticSearch service and the Logstash service based on the preset stack information. These conventional software combinations are usually deployed, maintained, and used by the same IT system administrator to form a subsystem that provides certain functions or services for the entire enterprise or organization.

[0065] Domain name speculation is to search the asset information database for asset information with WEB business load, similar domain names and titles. When a suspicious IP address has accessible WEB resources or a domain name, there is a high probability that a subsystem containing multiple components such as http services, middleware, databases, content management, etc. is hidden behind it. They are most likely maintained by the same group of people and belong to an IT system of an enterprise or organization. Domain names are also a type of important asset information that needs to be collected and maintained by enterprises or organizations. In this embodiment of the present invention, the detectable WEB pages are mainly obtained. <header>The content contained under the tag usually includes titles, navigation links, logos, welcome information, contact information, etc. These contents largely explain the role of the web page and its ownership. Collecting this information and searching and comparing it with the asset information database will greatly help locate suspicious IP addresses.

[0066] Trend analysis, search the past IP address information table in the historical information database, aggregate including but not limited to online history, online time, online duration and other indicators. When an IP address appears in the ARP table, there may be a variety of situations, as shown in Table 6:

[0067] Table 6

[0068] Online History Online time Possible Situations First time online Less than threshold Occasional Request First time online Greater than threshold There are corresponding assets Continuous online not applicable There are corresponding assets Continuously online and then silent not applicable The corresponding asset is removed

[0069] In the embodiment of the present invention, the regularly collected ARP table is stored in the historical information database. When it is necessary to analyze the suspicious IP, a statistical analysis of the trend will be performed in the historical information database to determine whether there is corresponding asset information for the address, and further provide reference information for determining the ownership of the suspicious IP address information.

[0070] Based on the above six aspects of information, namely, same-segment speculation, same-source speculation, service speculation, stack speculation, domain name speculation and trend analysis, an embodiment of the present invention adopts a K-nearest-neighbor algorithm to classify possible shadow assets. It should be noted that the following content is only an example. When actually establishing a classification model, multiple algorithms can be used to comprehensively establish a classification model to achieve better classification results.

[0071] The training table is established as shown in Table 7. The last column "responsible person" is the category, and the remaining columns are used as features.

[0072] In feature column processing, the number 1 indicates the same segment (source, domain name), and 0 indicates different segments (source, domain name).

[0073] One asset information (one row) may have multiple ports. The port values ​​are entered into the training table as they are, and the number of columns is set according to the number of ports. However, different assets have different port opening situations: for example, if host A opens 3 ports, three columns "Port No. 1", "Port No. 2" and "Port No. 3" will be written in the training table. If host B opens 2 ports, only two columns "Port No. 1" and "Port No. 2" can be filled. In this case, the missing columns are filled with 0, and the "Port No. 3" column of host B is set to 0.

[0074] Different services and stacks are numbered separately. For example, if host A runs NGINX, Tomcat, MySQL, etc., they are numbered 1, 2, and 3; stacks LAMP, ELK, and STACK are numbered 10, 20, and 30. MySQL is part of LAMP, so in the training table, the "stack number" of host A will be set to 10. If there is no corresponding stack for the service, it will be set to 0.

[0075] Table 7

[0076] Same paragraph Homologous Port Number X … Service Tag Stack Number Same domain name Responsible Person 0 1 80 … 5 25 1 Zhang San 0 1 80 … 9 20 0 Zhang San 1 0 443 … 20 25 0 Li Si … … … … … … … … 1 1 443 … 10 20 0 Wang Wu

[0077] Suspicious IP addresses are treated in the same way, as a point in the current n-dimensional space, and different k values ​​and distance algorithms are selected according to the actual situation. For example, the common Euclidean distance is used to calculate the distance from the current suspicious IP address point a (x_1, x_2, ..., x_n) to other points in the training table (represented by each row in the table) b (y_1, y_2, ..., y_n):

[0078]

[0079] To complete the classification of suspicious IP addresses.

[0080] Furthermore, cross validation is used to give the prediction accuracy.

[0081] The overall judgment process is as follows Figure 4 shown.

[0082] Step S6, generating the analysis result, includes the following steps:

[0083] Generate an analysis result report to explain whether the suspicious IP address is a shadow asset, its possible ownership and possibility for reference by the IT system administrator of the enterprise or organization, as shown in Table 8.

[0084] Table 8

[0085] IP address Is it a shadow asset? Possible responsible person System Accuracy 192.168.1.1 yes Zhang San Zhang San is under the jurisdiction of x 80.08％ 192.168.1.1 yes Li Si Li Si is under the jurisdiction of x 30.08％ … … … … … 192.168.1.2 no not applicable not applicable not applicable

[0086] To sum up, the intranet shadow asset discovery method provided by the present invention can realize automatic discovery and automatic analysis of shadow assets through steps S1 to S6, thereby solving the problem of heavy workload of asset sorting and incomplete grasp of asset information of IT system administrators of enterprises or organizations.< / header>

Claims

1. A method for discovering shadow assets in an intranet, characterized in that: The steps include: Step S1, establishing an asset information database; Step S2, collecting IP address information table; Step S3, screening suspicious IP addresses; Step S4, refining and saving suspicious IP addresses; Step S5, shadow asset analysis; Step S6, generating analysis results.

2. The method for discovering intranet shadow assets according to claim 1, characterized in that: In step S1, an asset information database is established, specifically: Organize and store the existing asset tables of the current enterprise or organization, including fields for host name, IP address, MAC address, domain name, open port, business load, and asset responsible person.

3. The method for discovering intranet shadow assets according to claim 1, characterized in that: In step S2, the IP address information table is collected, specifically: Organize the network architecture of an enterprise or organization, sort out the network devices and security devices with dynamic and real-time updated IP addresses, and collect and generate IP address information tables.

4. The method for discovering intranet shadow assets according to claim 3, characterized in that: The network equipment and security include firewalls, switches, routers, AC and AD, and the IP address information table includes IP address, MAC address, source device and collection time.

5. The method for discovering intranet shadow assets according to claim 1, characterized in that: In step S3, suspicious IP addresses are screened and retained, specifically: Compare the IP address information table with the asset information data table to filter out IP address information that is not in the asset information database.

6. The method for discovering intranet shadow assets according to claim 1, characterized in that: In step S4, the suspicious IP address is refined and saved, including the following steps: Step S4.1, actively detect and scan the suspicious IP address for port opening status, operating system type, and business load status, and obtain the port and corresponding service information; Step S4.2, based on the scanning results, attempt to perform WEB page detection on the IP addresses that may have WEB services, and save the obtained relevant information; Step S4.3, based on the scanning results, try to perform reverse domain name resolution on the IP addresses that may have WEB services, focusing on obtaining the domain name and related WEB page information; Step S4.4, sort out all collected information, and store the IP address information table for which no WEB service exists in the historical information database.

7. The method for discovering intranet shadow assets according to claim 1, characterized in that: In step S5, the shadow assets are analyzed and judged, specifically: Establish a multi-dimensional analysis model and call it according to different situations, including: Same-segment inference: Search the asset information database for asset information with the same network segment, including the same C segment and the same specified mask network segment; Same-source inference: searching the asset information database for asset information with the same source, including the same switch and the same VLAN; Service speculation, searching the asset information database for asset information with similar port ranges and similar services; Stack speculation, searching the asset information database for asset information with associated services; Domain name speculation: searching the asset information database for asset information with WEB business load, similar domain names and titles; Trend analysis, searching past IP address information tables in the historical information database, aggregating indicators including online history, online time, and online duration.

8. The method for discovering intranet shadow assets according to claim 1, characterized in that: In step S6, the analysis result is generated, including the following steps: Step S6.1, based on the results of the analysis, the following are classified: occasional requests, which refer to IP addresses that appear occasionally, including external visitors to the Internet. The IP addresses behind them may not necessarily belong to the assets of the enterprise or organization; Shadow assets are assets that actually exist but are not included in the balance sheet of a company or organization; Step S6.2: Issue a report containing the results of the analysis, describing the most likely responsible person for the shadow assets, the business system to which they belong, the reasons for the speculation, and the credibility of the speculation, for reference and confirmation by the IT system administrator.

Images