Asset surveying and mapping method and device

Abstract

The embodiment of the invention discloses an asset surveying and mapping method and device. The method comprises the following steps: passively capturing network traffic of a to-be-detected system; operating system identification is carried out on the network flow, if an identification result cannot be obtained, a detection packet is actively sent to carry out operating system detection, fingerprint feature matching is carried out in a pre-constructed operating system level fingerprint identification library, and the operating system level fingerprint identification library comprises TCP features; and performing application identification on the network flow, matching fingerprint features in a pre-constructed application feature fingerprint database, and if the fingerprint features cannot be matched, performing identification based on a trained neural network model. And comprehensive surveying and mapping of assets can be efficiently and harmlessly realized through a surveying and mapping means with passive detection as a main part and active detection as an auxiliary part. The problem that asset surveying and mapping comprehensiveness, surveying and mapping efficiency and surveying and mapping safety need to be improved in the prior art is solved.

Description

Technical Field

[0001] The present disclosure relates to the field of network technology, and in particular to an asset mapping method and device. Background Art

[0002] In modern cyberspace security, IT-related mapping plays an important role.

[0003] Common asset mapping methods include active detection and passive detection, or a combination of active and passive detection, as follows: 1. Port scanning: This is a basic method to determine the open ports of the target host. Port scanning can help you understand which TCP or UDP ports are open and try to establish connections with these ports; 2. Protocol identification: Based on port scanning, the protocol that may be running is matched through the open ports, and the corresponding protocol connection is attempted to be established to further explore the information of the target host; 3. Banner crawling: Banner is the identification information displayed by the network service when it is accessed. By crawling the banner, you can know the service type and its version information, which helps to identify specific software and services; 4. Certificate analysis: Analyze the SSL / TLS certificate used by the server to obtain information about the server's identity and encryption strength; 5. Feature matching: Collect asset metadata, such as title, header, etc., and then use these features to match the known asset feature library to identify the type and attributes of the asset; 6. Comprehensive analysis: A single feature is often not sufficient to accurately determine the nature of an asset, so it is necessary to conduct a comprehensive analysis combining multiple features to improve the accuracy of identification.

[0004] Manual expansion of the asset fingerprint database alone can no longer meet actual needs; and active detection methods cannot map some assets that have turned on the host firewall; in addition, active detection methods perform very poorly for scanning some UDP services, mainly because they take too long; but relying solely on passive sniffing cannot solve the problem of application encryption (mainly websites). Summary of the invention

[0005] The main purpose of the present disclosure is to provide an asset mapping method and device to solve the problems existing in the related technology.

[0006] In order to achieve the above-mentioned purpose, according to the first aspect of the present disclosure, there is provided an asset mapping method, comprising passively capturing the network traffic of a system to be detected; identifying the operating system of the network traffic, and if no identification result can be obtained, actively sending a detection packet to detect the operating system, wherein fingerprint feature matching is performed in a pre-built operating system level fingerprint recognition library, and the operating system level fingerprint recognition library includes TCP features; identifying the application of the network traffic, wherein fingerprint feature matching is performed in a pre-built application feature fingerprint library, and if no match is found, identification is performed based on a trained neural network model.

[0007] Optionally, the method further includes actively sending a detection packet to detect a preset type of traffic in the network traffic if the network traffic is transmitted in an encrypted manner, so as to identify the corresponding application.

[0008] Optionally, the detection information is acquired based on a UDP unicast protocol or a broadcast protocol to perform identification based on the information.

[0009] Optionally, the TCP features include TCP version, initial lifetime, Window size, TCP option or IPv6 extension header length, Window ratio, maximum segment size MSS, TCP option layout, and information on whether segmentation is allowed.

[0010] Optionally, if no match is found, identification based on the trained neural network model includes: preprocessing the network traffic to obtain text in the network traffic; inputting the text into the trained neural network model to identify the manufacturer information and product information corresponding to the text.

[0011] According to a second aspect of the present disclosure, there is provided an asset surveying and mapping device, comprising a network traffic passive acquisition unit, configured to passively capture the network traffic of a system to be detected; an operating system identification unit, configured to identify the operating system of the network traffic, and if no identification result can be obtained, actively send a detection packet to detect the operating system, wherein fingerprint feature matching is performed in a pre-built operating system level fingerprint identification library, and the operating system level fingerprint identification library includes TCP features; an application identification unit, configured to identify the application of the network traffic, wherein fingerprint feature matching is performed in a pre-built application feature fingerprint library, and if no match is found, identification is performed based on a trained neural network model.

[0012] Optionally, the device further comprises an active detection unit configured to actively send detection packets for detecting preset types of traffic in the network traffic if the network traffic is transmitted in an encrypted manner, so as to identify the corresponding application.

[0013] Optionally, the detection information is acquired based on a UDP unicast protocol or a broadcast protocol to perform identification based on the information.

[0014] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described in any one of the implementation modes of the first aspect.

[0015] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the method described in any one implementation of the first aspect.

[0016] The asset mapping method and device of this embodiment include the following steps: passively capturing the network traffic of the system to be detected; identifying the operating system of the network traffic, and actively sending a detection packet to detect the operating system if no identification result can be obtained, wherein fingerprint feature matching is performed in a pre-built operating system level fingerprint recognition library, and the operating system level fingerprint recognition library includes TCP features; identifying the application of the network traffic, wherein fingerprint feature matching is performed in a pre-built application feature fingerprint library, and if no match is found, identification is performed based on a trained neural network model. By using a mapping method that is mainly passive detection and supplemented by active detection, comprehensive asset mapping can be achieved efficiently and harmlessly. This overcomes the problem that the comprehensiveness, efficiency, and safety of asset mapping in the related art need to be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a flow chart of an asset mapping method according to an embodiment of the present disclosure; Figure 2 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] According to an embodiment of the present disclosure, a method for asset mapping is provided, such as Figure 1 As shown, it includes the following steps 101 to 103: Step 101: Passively capture the network traffic of the system to be detected.

[0023] In this embodiment, the relevant equipment is connected to the existing network, and asset mapping is performed by passively capturing relevant network communication data packets.

[0024] Step 102: Identify the operating system of the network traffic. If no identification result can be obtained, actively send a detection packet to detect the operating system, wherein fingerprint feature matching is performed in a pre-built operating system level fingerprint recognition library, and the operating system level fingerprint recognition library includes TCP features.

[0025] In this embodiment, a general asset mapping fingerprint library is established through the historically collected asset / model fingerprint library. These fingerprint libraries mainly include fingerprint libraries at the operating system level and general application level. The general application fingerprint library includes types such as Web applications and database applications.

[0026] Establish an operating system-level TCP fingerprint identification library, and use the passive operating system fingerprint library to map the fingerprints of related assets. Perform operating system identification on the operating system fingerprint. If the operating system cannot be identified, actively send a detection packet to detect the related system. This detection is performed using NMAP.

[0027] As an optional implementation of this embodiment, the TCP features include TCP version, initial lifetime, Window size, TCP option or IPv6 extension header length, Window ratio, maximum segment size MSS, TCP option layout, and information on whether segmentation is allowed.

[0028] In this optional implementation, TCP features are transport layer protocol features, and relevant features of TCP include: TCP version, i.e., v4 or v6; initial survival time, i.e., TTL, such as 64, 128, 255, etc., which may have different values ​​in different operating systems; Window size, which is generally a multiple of MSS (mentioned below); TCP option or IPv6 extension header length; Window scale, i.e., WinScale; Maximum segment size, i.e., Maximum Segment Size, abbreviated as MSS, which is generally around 1400-1500 bytes; TCP option layout: a combination of the above contents, including nop option, MSS, Window Scale, Selective ACK, whether SACK is allowed (SACK Permit), etc.; others such as whether segmentation is allowed.

[0029] Step 103: Identify the application of the network traffic, wherein the fingerprint features are matched in a pre-built application feature fingerprint library. If a match cannot be found, identification is performed based on a trained neural network model.

[0030] In this embodiment, an application asset mapping feature fingerprint library is established, which mainly includes company manufacturer information, product version information, etc. The passive network traffic is identified by using the feature, and if it cannot be identified, a neural network model is used for identification.

[0031] Furthermore, for some asset web applications and other application traffic with significant characteristics (such as cameras, network video recorders, etc.), neural networks are used to establish deep recognition components for their manufacturer and model data, so that neural networks can be used to identify related texts when fixed fingerprint patterns cannot be used for recognition. Before training the neural network, training samples are constructed. For example, the organization of texts such as manufacturers and models is carried out using mixed generated placeholders. For example, for "Tianrongxin Technology Group", in addition to retaining this text as the input of the neural network, other texts such as "TianXxin Technology Group", "XXxin Technology Group", "XXX Technology Group" are also used for replacement training to obtain better generalization effect; the above method is also applicable to neural network learning for models, such as for Dahua camera DH-IPC-HDBW3433R-Z-400, it can be replaced with "DH-xxx-xxxxxxxxx-x-xxx, etc. for training; the purpose of using deep neural networks here is to better achieve generalization, rather than just using regular patterns; the data for training this deep neural network also contains some other corpora that are not manufacturers, products or models, in order to obtain better differentiation.

[0032] This embodiment does not completely rely on the asset fingerprint library established in advance, but uses the data packet response obtained by passive traffic to perform multiple auxiliary detections on the parts where the relevant manufacturers and models may appear. Further, after the detection results are obtained, the fingerprint corresponding to the asset is stored in the fingerprint library to update the fingerprint library.

[0033] To a certain extent, the system information of related assets can be inferred, and the related assets can be stimulated from the outside to actively send communication data in a relatively light and harmless manner, so that important information can be obtained from these data packets to achieve the purpose of comprehensive asset mapping.

[0034] As an optional implementation method of this embodiment, if a match cannot be found, identification is performed based on a trained neural network model, including: preprocessing the network traffic to obtain text in the network traffic; inputting the text into the trained neural network model to identify the manufacturer information and product information corresponding to the text.

[0035] In this optional implementation, for passive network traffic, if fingerprints cannot be directly applied to identify assets, the following method is used for processing, taking web pages as an example: crawl the website to return relevant content, remove known web page tags, scripts, etc., and pass the remaining text (including web page titles) into the neural network model to obtain whether these texts may be the text organization of manufacturers, products or product models; transmit the fingerprint information identified by the neural network back to the cloud (the cloud synchronization function needs to be activated) for use in other deployment sites.

[0036] As an optional implementation of this embodiment, if the network traffic is transmitted in an encrypted manner, a detection packet is actively sent to detect a preset type of traffic in the network traffic to identify the corresponding application.

[0037] In this optional implementation, if passive network traffic is transmitted in an encrypted manner, taking web pages as an example, a detection packet is actively sent for the part that may be a web page visit, and then the relevant content returned by the website can be captured as described above, and known web page tags, script programs, etc. are removed. The remaining text (including the web page title) is transmitted to the neural network to obtain whether the text may be the text organization of the manufacturer, product or product model.

[0038] When the network traffic is encrypted, a small amount of auxiliary detection data is actively sent and compared with the response results to obtain screening results. Asset mapping results and fingerprints are obtained automatically or with user intervention, and these fingerprints can be uploaded back to the cloud as a fingerprint library that can be updated in the future.

[0039] As an optional implementation of this embodiment, the detection information is obtained based on the UDP unicast protocol or the broadcast protocol to perform identification based on the information.

[0040] In this embodiment, for some special devices, such as instant messaging, IPC or other IoT devices, since their services generally listen to UDP ports, the device discovery protocol data is sent for discovery instead of traversing all UDP ports to avoid the problem of slow speed.

[0041] For example, for some UDP services, including some IoT devices, instant messaging devices, etc., inducement data packets are sent (through unicast or multicast), such as the network asset discovery protocol to obtain related camera devices (such as service ports 3720, 37020, etc.), and the LAN internal sending protocol (including Fei instant messaging, etc.) is sent to discover its listening service; in addition, an additional effect is that a list of illegal remote control programs that may be installed or communication programs that use illegal tunneling methods can also be obtained from these results.

[0042] Use specific UDP unicast or broadcast protocols to obtain survivable assets in the network, their manufacturers, equipment, software and hardware version information, to supplement the lack of active asset discovery capabilities and remote control and instant messaging software that may not be allowed to be installed.

[0043] Furthermore, since surveying and mapping itself involves multiple features or fingerprints, an integrated surveying and mapping rule is adopted, that is, the syntax of the overall asset surveying and mapping rule is as follows: fingerType ::= OS|Application OS::= TCPFinger|ServiceFinger|ClientFinger TCPFinger::=OSName:Ver,TTL,OptionLength,MSS,WinSize,Scale;OptionLayout #Contains TCP version, survival time, TCP option length, maximum segmentation, Window size, TCP option layout Some application services may correspond to the operating system type, such as OpenSSH service, and the port is optional. The example is as follows: #FreeBSD:"SSH-. ?-OpenSSH_([0-9.az]+)\sFreeBSD”:SSH,22 #OpenSUSE Leap: Welcome to openSUSE Leap:Telnet ServiceFinger::=OSName:Pattern,ServiceName[,Port] #For client programs, such as some browsers, operating system information may be included (User-Agent) ClientFinger::=OSName:Pattern,ClientName Application::=AppName:Pattern[,Port] This embodiment does not need to establish a large number of asset fingerprint libraries in advance, thereby saving some meaningless maintenance time for unknown devices or assets; the mapping method based on passive detection and supplemented by active detection can have a smaller impact on the assets on the user's site, and can also perform certain mapping for asset-related services with encrypted transmission; for some special devices, such as instant messaging, IPC or other Internet of Things devices, since their services generally listen to UDP ports, this patent uses the sending device discovery protocol data for discovery, rather than traversing all its UDP ports, which improves efficiency.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] According to an embodiment of the present disclosure, there is also provided an asset surveying and mapping device, including a network traffic passive acquisition unit, configured to passively capture the network traffic of a system to be detected; an operating system identification unit, configured to identify the operating system of the network traffic, and if no identification result can be obtained, actively send a detection packet to detect the operating system, wherein fingerprint feature matching is performed in a pre-built operating system level fingerprint identification library, and the operating system level fingerprint identification library includes TCP features; an application identification unit, configured to identify the application of the network traffic, wherein fingerprint feature matching is performed in a pre-built application feature fingerprint library, and if no match is found, identification is performed based on a trained neural network model.

[0046] As an optional implementation of this embodiment, the device also includes an active detection unit, which is configured to actively send detection packets to detect preset types of traffic in the network traffic if the network traffic is transmitted in an encrypted manner, so as to identify the corresponding application.

[0047] As an optional implementation of this embodiment, the detection information is obtained based on the UDP unicast protocol or the broadcast protocol to perform identification based on the information.

[0048] As an optional implementation method of this embodiment, if a match cannot be found, identification is performed based on a trained neural network model, including: preprocessing the network traffic to obtain text in the network traffic; inputting the text into the trained neural network model to identify the manufacturer information and product information corresponding to the text.

[0049] The present disclosure provides an electronic device, such as Figure 2 As shown, the electronic device includes one or more processors 31 and a memory 32. Figure 2 A processor 31 is taken as an example.

[0050] The controller may further include: an input device 33 and an output device 34 .

[0051] The processor 31, the memory 32, the input device 33 and the output device 34 may be connected via a bus or other means. Figure 2 The example of connecting through bus is taken in the following.

[0052] The processor 31 may be a central processing unit (CPU). The processor 31 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. A general-purpose processor may be a microprocessor or the processor may be any conventional processor.

[0053] The memory 32 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the control method in the embodiment of the present disclosure. The processor 31 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 32, that is, the enterprise data exchange method based on email in the above method embodiment is implemented.

[0054] The memory 32 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of a processing device operated by the server, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 32 may optionally include a memory remotely arranged relative to the processor 31, and these remote memories may be connected to a network connection device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0055] The input device 33 can receive input digital or character information, and generate key signal input related to user settings and function control of the processing device of the server. The output device 34 can include display devices such as a display screen.

[0056] One or more modules are stored in the memory 32, and when executed by one or more processors 31, the execution is as follows: Figure 1 The method shown.

[0057] Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiment method is a program that can be completed by instructing related hardware through a computer program, which can be stored in a computer-readable storage medium. When the program is executed, it can include the process of the embodiment of the above-mentioned method. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0058] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An asset mapping method, characterized in that: include: Passively capture the network traffic of the system to be detected; Identify the operating system of the network traffic, and if no identification result can be obtained, actively send a detection packet to detect the operating system, wherein fingerprint feature matching is performed in a pre-built operating system level fingerprint recognition library, and the operating system level fingerprint recognition library includes TCP features; The network traffic is subjected to application identification, wherein fingerprint features are matched in a pre-built application feature fingerprint library. If no match is found, identification is performed based on a trained neural network model.

2. The asset mapping method according to claim 1, characterized in that: The method also includes actively sending a detection packet to detect a preset type of traffic in the network traffic if the network traffic is transmitted in an encrypted manner, so as to identify the corresponding application.

3. The asset mapping method according to claim 2, characterized in that: The detection information is acquired based on the UDP unicast protocol or the broadcast protocol to perform identification based on the information.

4. The asset mapping method according to claim 1, characterized in that: The TCP features include TCP version, initial survival time, Window size, TCP option or IPv6 extension header length, Window ratio, maximum segment size MSS, TCP option layout, and information on whether segmentation is allowed.

5. The asset mapping method according to claim 1, characterized in that: If no match is found, identification based on the trained neural network model includes: Preprocess the network traffic to obtain text in the network traffic; input the text into a trained neural network model to identify the manufacturer information and product information corresponding to the text.

6. An asset mapping device, characterized in that: include: A network traffic passive acquisition unit is configured to passively capture the network traffic of the system to be detected; An operating system identification unit is configured to identify the operating system of the network traffic, and actively send a detection packet to detect the operating system if no identification result can be obtained, wherein fingerprint feature matching is performed in a pre-built operating system level fingerprint identification library, and the operating system level fingerprint identification library includes TCP features; The application identification unit is configured to identify the application of the network traffic, wherein the fingerprint features are matched in a pre-built application feature fingerprint library. If no match is found, identification is performed based on a trained neural network model.

7. The asset surveying and mapping device according to claim 6, characterized in that: The device also includes an active detection unit, which is configured to actively send detection packets to detect preset types of traffic in the network traffic if the network traffic is transmitted in an encrypted manner, so as to identify the corresponding application.

8. The asset surveying and mapping device according to claim 7, characterized in that: The detection information is acquired based on the UDP unicast protocol or the broadcast protocol to perform identification based on the information.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 5.

10. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the method described in any one of claims 1 to 5.

Images