A method and apparatus for identifying an external semi-connection scan

Abstract

The application discloses a method and device for identifying external half-connection scanning, comprising: collecting relevant data of a host accessed by an external half-connection network; performing white list filtering on the relevant data; splicing an IP port based on the relevant data filtered by the white list; and identifying whether an external IP exists to scan a protected asset based on the spliced IP port. According to the embodiment of the application, the relevant data filtered by the white list is used to splice an IP port, and whether an external IP exists to scan a protected asset is identified based on the spliced IP port. The method of the application can effectively identify that an external host performs a half-connection network port open scanning on a protected asset.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a method and device for identifying external half-connection scanning. BACKGROUND

[0002] In the field of network security, network port service scanning is one of the higher frequency means in the process of intrusion, horizontal penetration, etc. Through network port service scanning, the service opening situation of the attack target can be quickly determined, and the open port service can be attacked accordingly. Therefore, how to identify that there is an IP in the internal network to perform port scanning is an important security detection method. If the internal network finds that there is an IP to perform port scanning, the internal host compromise and horizontal penetration behavior can be identified early.

[0003] The current typical network port service scanning identification method is to deploy network traffic probes to collect bypass traffic, etc. This method is generally deployed at the Internet exit, and it is difficult to cover all horizontal traffic of enterprise-level machine rooms (the cost is too high, and general enterprises cannot afford the implementation cost). This relatively concealed scanning method does not establish a complete TCP connection record on the scanned host, so that the host cannot perceive the half-connection scanning method used by the external host. SUMMARY

[0004] The embodiments of the present application provide a method and device for identifying external half-connection scanning, which can effectively identify the half-connection network port opening scanning of the external host to the protected assets.

[0005] The embodiments of the present application provide a method for identifying external half-connection scanning, comprising:

[0006] Collecting related data of the host accessed by the external half-connection network;

[0007] Performing whitelist filtering on the related data;

[0008] Based on the related data filtered by the whitelist, splicing the IP port;

[0009] Based on the spliced IP port, identifying whether there is external IP scanning to the protected assets.

[0010] Optionally, the related data of the host accessed by the external half-connection network is collected by using a probe deployed on the host, comprising:

[0011] In the case that the specified network card receives a network data packet, judging the network data packet based on the probe;

[0012] If the network packet is a SYN packet, network quintuple data of the network packet is obtained, the network quintuple data is taken as a key, and is stored into a first extended Berkeley Packet Filter map (eBPF map), and the network quintuple data and a connection state field are stored into a second eBPF map;

[0013] If the network packet is only an ACK packet, network quintuple data of the network packet is obtained, the network quintuple data is taken as a key, and a corresponding data connection state in the first eBPF map is modified to be successful.

[0014] Optionally, the related data of the host accessed by the external semi-connection network further includes:

[0015] Based on the probe, network quintuple data is obtained from the second eBPF map, the obtained network quintuple data is taken as a key, it is judged whether the network quintuple data exists in the first eBPF map, if exists, the network quintuple data of the external semi-connection network access is sent to a data analysis platform.

[0016] Optionally, the whitelist filtering of the related data includes:

[0017] The related data matched to the whitelist of IP, network segment and port is ignored, and does not participate in subsequent calculation.

[0018] The related data not matched to the whitelist participates in subsequent calculation.

[0019] Optionally, the IP port is spliced based on the related data filtered based on the whitelist, and includes:

[0020] The source IP is grouped, and the IP and port of the accessed host in the group are spliced to obtain the IP port.

[0021] Optionally, based on the spliced IP port, whether there is an external IP to scan the protected asset includes:

[0022] The spliced IP port is de-duplicated;

[0023] The de-duplicated IP port is merged and spliced, and a characteristic value is calculated;

[0024] According to the calculated characteristic value and the corresponding weight value, the threat total score of the IP port to the network port scanning of the protected asset is determined;

[0025] If any IP port threat total score is greater than a preset threshold, it is determined that the any IP port exists network scanning behavior to the protected asset.

[0026] Optionally, after identifying whether there is an external IP scanning the protected asset, the method further comprises:

[0027] alerting any IP port;

[0028] and / or

[0029] based on the probe receiving the issued interception policy, writing the interception policy into an eBPF map, and based on the probe receiving traffic data packets, parsing network quintuple data, and reading the interception policy from the eBPF map, matching the parsed network quintuple data, and discarding traffic data packets that match successfully, to achieve interception.

[0030] The embodiment of the present application also provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to realize the steps of the method for identifying external semi-connection scanning as described above.

[0031] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the method as described above.

[0032] The embodiment of the present application splices IP ports based on relevant data filtered based on a white list, and identifies whether there is an external IP scanning the protected asset based on the spliced IP ports. The method of the present application can effectively identify that an external host scans the protected asset in a semi-connection network port opening manner.

[0033] The above description is only a summary of the technical solutions of the present application, in order to more clearly understand the technical means of the present application, which can be implemented according to the content of the specification, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0034] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the attached drawings refer to the same or similar components. In the drawings:

[0035] Figure 1 The basic flow of the method for identifying external semi-connection scanning of the embodiment of the present application is shown;

[0036] Figure 2 The overall flow of the method for identifying external semi-connection scanning of the embodiment of the present application is shown. Detailed Implementation

[0037] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0038] This application provides a method for identifying external semi-connected scans, such as... Figure 1 The process includes the following steps:

[0039] In step S101, data related to the host (protected asset) being accessed by an external half-connection network is collected. In a specific example, data on the host being accessed by an external half-connection network can be collected and reported in real time using Linux eBPF XDP technology or other methods such as system kernel hooking. Host port listening data can also be collected to improve the accuracy of the model.

[0040] The relevant data is calculated and analyzed using data analysis platforms such as Flink and Spark. In step S102, the relevant data is filtered using a whitelist, specifically by whitelisting based on IP address, network segment, port, etc.

[0041] In step S103, the IP port is concatenated based on the relevant data filtered by the whitelist. Specifically, the data can be grouped according to the source IP or other custom fields, and then the IP of the accessed asset within the group is concatenated with the port to produce the IP port (IP:PORT).

[0042] In step S104, based on the concatenated IP ports, it is determined whether there are external IPs scanning the protected asset (host). Specifically, all IPs:ports can be deduplicated and the total number counted. For IPs:ports on ports not being listened to by the host, the total number counted as well. In some specific examples, information entropy or other feature dimensions can be used in the calculation to determine whether there are external IPs scanning the protected asset.

[0043] This application's embodiments construct IP ports by concatenating relevant data filtered through a whitelist; based on the concatenated IP ports, it identifies whether external IPs are scanning the protected asset. The method of this application can effectively identify external hosts performing half-connection network port scans on the protected asset.

[0044] In the embodiments of the present application, the Linux eBPF XDP technology is used to collect external IP initiated semi-connection network request data to the protected assets as an example for illustrative description. Other ways such as system kernel Hook can also be used to collect external IP initiated semi-connection network request data to the protected assets, which are not limited herein. In some embodiments, the related data of the host accessed by the external semi-connection network is collected by using the probe deployed on the host. In the embodiments, the probe (hereinafter referred to as Agent) is used to collect the received semi-connection network request data. In specific examples, the Agent is divided into kernel mode Agent and user mode Agent. The kernel mode Agent is implemented by using Linux eBPF (extended Berkeley Packet Filter) XDP technology, which can be attached to a specified network card. When a SYN packet arrives at the network card, the kernel network protocol stack receives the SYN data before the SYN packet is obtained, which is used to collect semi-connection network request data. In some embodiments, the related data of the host accessed by the external semi-connection network includes:

[0045] In the case that the specified network card receives a network packet, the probe is used to judge the network packet. If the network packet is a SYN packet, the network quintuple data of the network packet is obtained, the network quintuple data is used as the key, and the network quintuple data is stored in the first extended Berkeley Packet Filter map (eBPF map). The network quintuple data and the connection state field are stored in the second eBPF map. For example, in a specific scenario, when the specified network card receives a network packet, the kernel mode Agent judges whether the data packet is a SYN packet (i.e. semi-connection network connection request packet). If it is a SYN packet, the network quintuple data of the packet is obtained: source IP, source port, protocol type, target IP, and target port. The network quintuple is used as the key to store in the eBPF map A, and the network quintuple data and the connection state field (default as failed) are stored in the ebpf map B.

[0046] If the network packet is only an ACK packet, network quintuple data of the network packet is obtained, and corresponding data connection state in the first eBPF map is modified to success by taking the network quintuple data as a key. Specifically, if the kernel agent judges whether the network packet is only an ACK packet (not including Push and Fin), if it is only an ACK packet, it indicates that the network connection has been established, and the network quintuple data of the packet, i.e., source IP, source port, protocol type, target IP, and target port, is obtained, and the corresponding data connection state in the eBPF map A is modified to success by taking the network quintuple as a key.

[0047] In some embodiments, the related data collected by the host accessed by the external semi-connection network further includes: based on the probe, obtaining network quintuple data from the second eBPF map, and judging whether the network quintuple data exists in the first eBPF map by taking the obtained network quintuple data as a key, if it exists, the network quintuple data of the external semi-connection network access is sent to the data analysis platform. In a specific example, the user agent can obtain the network quintuple data from the eBPF map B in real time, and every specified time, for example, one minute, the network quintuple is used as a key to judge whether the key exists in the eBPF map A, if it exists, the network quintuple data of the semi-connection network request data is sent to the Flink, Spark, and other data analysis platforms.

[0048] In some embodiments, the whitelist filtering of the related data includes: ignoring the related data matched to the whitelist of IP, network segment, and port, and not participating in subsequent calculation; and the related data not matched to the whitelist participates in subsequent calculation. Specifically, it can be implemented by using a whitelist filtering engine, and the port monitoring data collected and reported by the calculation host side in real time is consumed by the Flink, Spark, and other data analysis platforms, including the semi-connection inbound network access data. First, the record of the whitelist filtering engine is performed, and the calculation logic is to ignore the data matched to the whitelist of IP, network segment, and port, and not participate in subsequent calculation, and for the data not matched to the whitelist, continue to enter the following flow for calculation.

[0049] The data filtered by the whitelist enters the grouping aggregation logic, such as Figure 2As shown, in some embodiments, based on the filtered relevant data based on the whitelist, the IP port is spliced to include: grouping according to the source IP, and splicing the IP and port of the accessed host in the group to obtain the IP port. When grouping according to the source IP, the network scanning access behavior of one source IP to all protected hosts can be effectively placed in a calculation group, so that the network scanning behavior of the source IP is better analyzed. After completing the grouping, the IP:PORT of the accessed asset IP and Port in the group is spliced to obtain IP:PORT. In specific applications, other custom grouping configurations can also be used.

[0050] In some embodiments, based on the spliced IP port, whether an external IP scans a protected asset is identified, including:

[0051] The spliced IP port is de-duplicated, and specifically, the IP:PORT data spliced in the group is de-duplicated. The total number of de-duplicated N1 in the successful connection state is calculated, and the total number of de-duplicated N2 in the failed connection state is calculated.

[0052] The de-duplicated IP port is merged and spliced, and the characteristic value is calculated.

[0053] According to the calculated characteristic value and the corresponding weight value, the threat total score of the IP port scanning the network port of the protected asset is determined. Specifically, all de-duplicated IP:PORT in the group can be merged and spliced to calculate the information entropy or other characteristic values, for example, the source IP grouping calculation:

[0054] The total number of de-duplicated N3 of the unmonitored port and the successful connection state is calculated;

[0055] The total number of de-duplicated N4 of the unmonitored port and the failed connection state is calculated.

[0056] Optionally, all de-duplicated IP:PORT in the group can be merged and spliced to calculate the information entropy to obtain N5 (approximately close to 1, indicating that the data is high in discrete degree and high in data distribution degree);

[0057] Then, all the characteristic values are calculated according to the weight value to obtain the threat total score of the IP scanning the network port of the protected asset: (N4 / N2+N2 / (N1+N2)+N4 / (N3+N4))*(N2+N4) / n1*N5, wherein N4 / N2 represents the proportion of access to non-existent ports, the higher the threat, N4 / N2+N2 / (N1+N2) represents the failure proportion, the higher the threat, N4 / (N3+N4) represents the failure proportion, the higher the threat, and n1 represents a coefficient 1, default 1000, which can be customized.

[0058] If the threat total score of any IP port is greater than a preset threshold, it is determined that the any IP port exists a network scanning behavior on the protected asset.

[0059] If the threat total score is greater than or equal to a preset threshold, for example, 400, it is considered that the IP port exists a network scanning behavior on the protected asset, including a half-connection network scanning. The specific preset threshold can be adjusted according to the running situation, or after running the model for a certain period of deep learning, the preset threshold is configured again.

[0060] In some embodiments, after identifying whether the external IP exists a scanning behavior on the protected asset, the method further comprises:

[0061] Alarm is given to the any IP port. In the case that the aforementioned detection is greater than the preset threshold, an alarm of the IP port performing a half-connection or other network scanning on the protected asset is generated. For the alarm, white list processing can also be selected, and the white list is automatically generated according to the alarm and user selection of IP, network segment, port and other fields to generate a white list strategy.

[0062] And / or

[0063] Based on the interception policy received by the probe, the interception policy is written into the eBPF map, and based on the probe, after receiving the traffic data packet, network quintuple data is parsed and obtained, and the interception policy is read from the eBPF map, the parsed network quintuple data is matched, and the traffic data packet matched successfully is discarded to achieve interception.

[0064] Specifically, automatic interception or manual issuance of an interception policy can be set to enable interception, and the access record of the IP port can be blocked by eBPF XDP, or other ways such as iptables can be used for interception. The XDP interception is described in the embodiments of the application, and other ways are not described here. The use of eBPF XDP to block traffic is described as follows: the user state agent receives the interception policy issued, and writes the interception policy into the eBPF map. After the kernel state agent receives the traffic data packet, network quintuple data: source IP, source port, protocol type, target IP, target port is parsed and obtained, the interception policy is read from the eBPF map, the network quintuple and the interception policy are matched, and the matched data packet is discarded to achieve the blocking effect; if the interception policy is not met, the traffic data packet is passed.

[0065] The method of the embodiments of the present application realizes high-performance external entry direction semi-connection network connection data collection at the network card driver level through XDP, and can accurately detect IP semi-connection network scanning and the like on protected assets through network access behavior analysis, thereby solving the problems that the conventional host security detection cannot cover semi-connection network scanning and even full-connection network scanning is difficult to detect when the host listens to few ports. For example, even if full-connection scanning is performed when the host listens to few ports, the amount of captured logs is far from enough to support behavior analysis, and the method of the present application can capture the requests that have not successfully established a connection, and can guarantee effective detection capability in the case that the target host has low survival and listens to few ports.

[0066] The embodiments of the present application also provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to realize the steps of the method for identifying external semi-connection scanning as described above.

[0067] The embodiments of the present application also provide a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the method as described above.

[0068] It should be noted that in the embodiments of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive containing, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0069] The serial numbers of the above embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and a general hardware platform, of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the methods described in the embodiments of the present application.

[0071] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection of the present application.

Claims

1. A method of identifying an external half-connection scan, characterized by, The method comprises the following steps: Collecting relevant data of the host accessed by the external semi-connection network; White list filtering of the relevant data; Based on the relevant data filtered by the white list, the IP port is spliced; Based on the spliced IP port, it is identified whether there is an external IP scanning the protected asset; The relevant data of the host accessed by the external semi-connection network is collected by using the probe deployed on the host, comprising: In the case that the specified network card receives network data packet, the network data packet is judged based on the probe; If the network data packet is a SYN data packet, the network quintuple data of the network data packet is obtained, the network quintuple data is taken as the key, and the network quintuple data is stored in the first extended Berkeley Packet Filter map (eBPF map), and the network quintuple data and the connection state field are stored in the second eBPF map; If the network data packet is only an ACK data packet, the network quintuple data of the network data packet is obtained, and the corresponding data connection state in the first eBPF map is modified to success based on the network quintuple data as the key; The relevant data of the host accessed by the external semi-connection network further comprises: Based on the probe, the network quintuple data is obtained from the second eBPF map, and the network quintuple data is used as the key to judge whether the network quintuple data exists in the first eBPF map, if exists, the network quintuple data of the external semi-connection network is sent to the data analysis platform; Based on the spliced IP port, it is identified whether there is an external IP scanning the protected asset, comprising: The spliced IP port is de-duplicated; The de-duplicated IP port is merged and spliced, and the characteristic value is calculated; According to the calculated characteristic value and the corresponding weight value, the threat total score of the IP port scanning the protected asset is determined; If any IP port threat total score is greater than a preset threshold, it is determined that the any IP port exists network scanning behavior to the protected asset; The threat total score of the IP port scanning the protected asset is: (N4 / N2+N2 / (N1+N2)+N4 / (N3+N4))*(N2+N4) / n1*N5 Wherein, N1 is the total de-duplicated number of the connection state being success, N2 is the total de-duplicated number of the connection state being failure, N3 is the total de-duplicated number of the unmonitored port and the connection state being success, N4 is the total de-duplicated number of the unmonitored port and the connection state being failure, N5 is the information entropy normalized value calculated by grouping all de-duplicated IP:PORT after merging and splicing, N4 / N2 represents the access proportion of non-existent port, the higher the threat is, N4 / N2+N2 / (N1+N2) represents the failure proportion, the higher the threat is, N4 / (N3+N4) represents the failure proportion, the higher the threat is, and n1 represents the coefficient.

2. The method of identifying an external semi-join scan of claim 1, wherein, The white list filtering of the relevant data comprises: Ignore IP, network segment, port matching to the relevant data of the whitelist, do not participate in subsequent calculation; For the relevant data that does not match the whitelist, participate in subsequent calculation.

3. The method of identifying an external semi-join scan of claim 2, wherein, Based on the relevant data filtered by the whitelist, splice IP port including: Grouping according to the source IP, and splicing the IP and port of the accessed host in the group to obtain the IP port.

4. The method of identifying an external semi-join scan of claim 3, wherein, After identifying whether there is an external IP scanning the protected assets, it also includes: Alarm for the any IP port; And / or Based on the interception policy received by the probe, write to the eBPF map, and based on the network five tuple data obtained by the probe after receiving the traffic data packet, read the interception policy from the eBPF map, match the parsed network five tuple data, discard the traffic data packet matched successfully, to realize interception.

5. A computer device, comprising: The memory and the processor, the memory has a computer program stored thereon, and the computer program is executed by the processor to realize the steps of the method for identifying external half connection scanning according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium has a computer program stored thereon, and the computer program is executed by the processor to realize the steps of the method according to any one of claims 1 to 4.

Images