A network threat data entry method, device, equipment and medium

Abstract

The application discloses a network threat data entry method and device, equipment and medium, applied to the field of network security, comprising: determining the target network security device of the data to be extracted from each network security device, and determining the corresponding target microservice from the local preset microservice cluster; based on the data size and log transmission frequency of network threat logs, determine the target transmission protocol from the preset transmission protocol set; use the preset port listening instance to listen to the network port, and obtain the target network threat log through the target transmission protocol; based on the preset information extraction rule, extract the target key information from the target network threat log; bind the target key information and the device information of the target network security device, and save the bound information to the local preset data warehouse. To achieve the purpose of reducing the workload of network security service personnel, realizing sustainable expansion, sustainable optimization of data collection and entry.

Description

Technical Field

[0001] This invention relates to the field of network security, and in particular to a method, apparatus, device, and medium for recording network threat data. Background Technology

[0002] With the increasing number of cybersecurity vendors in the market, enterprises may purchase cybersecurity equipment from multiple vendors simultaneously. Faced with multiple brands and versions of cybersecurity equipment, enterprises need to invest more cybersecurity service personnel to review network threat logs generated by these devices one by one and extract useful information. This results in problems such as heavy workload, high repetition, complex tracing, and susceptibility to statistical errors.

[0003] In existing technologies, network threat logs are collected from different network security devices, preprocessed, and then stored for later searching and viewing by network security service personnel. However, since different manufacturers and versions of network security devices generate various types of network threat logs, upgrading or expanding the network security devices in the system may cause compatibility issues.

[0004] Therefore, how to reduce the workload of cybersecurity service personnel while achieving sustainable expansion and optimization of data collection and entry is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method for extracting network threat data, which can reduce the workload of network security service personnel while achieving sustainable expansion and optimization of data collection and entry. The specific solution is as follows:

[0006] Firstly, this application provides a method for inputting network threat data, applied to a preset input system, wherein the preset input system establishes connections with different network security devices through different network ports; wherein the method includes:

[0007] The target network security device from which the data to be extracted is determined from each of the network security devices, and the target microservice corresponding to the target network security device is determined from the local preset microservice cluster.

[0008] Based on the data volume and log transmission frequency of the network threat logs in the target network security device, a corresponding target transmission protocol is determined from a preset transmission protocol set.

[0009] The target microservice is used to listen to the network port corresponding to the target network security device through a preset port listening instance, and the target network threat log of the target network security device is obtained through the target transmission protocol.

[0010] extract target key information from the target network threat log based on a preset information extraction rule; the target key information includes attacker key information, attacked target key information, network attack behavior information, network attack domain name information, network attack behavior corresponding protection behavior information, key request information corresponding to a target network request carrying a network attack, and key response information;

[0011] The target microservice binds the target key information to device information of the target network security device, and saves the bound information to a preset data warehouse.

[0012] Optionally, the target transmission protocol is determined from a preset transmission protocol set based on a data size of the network threat log in the target network security device and a log transmission frequency, and the target transmission protocol includes:

[0013] When the data size is greater than a preset size and the log frequency is greater than a preset frequency, the target transmission protocol is determined to be a TCP protocol.

[0014] When the data size is not greater than the preset size and the log frequency is not greater than the preset frequency, the target transmission protocol is determined to be a UDP protocol.

[0015] Optionally, the target network security device corresponding network port is listened to by the target microservice and a preset port listening instance, and the target network threat log of the target network security device is obtained through a target transmission protocol, and the target transmission protocol includes:

[0016] The target network security device corresponding network port is listened to by the target microservice and a DatagramSocket, and the target network threat log of the target network security device is obtained through a UDP protocol.

[0017] Or, the target network security device corresponding network port is listened to by the target microservice and a Socket, and the target network threat log of the target network security device is obtained through a TCP protocol.

[0018] Optionally, the target key information is extracted from the target network threat log by the target microservice based on a preset information extraction rule, and the target key information includes:

[0019] The target network threat log is split based on a split function by the target microservice, and the target key information is extracted.

[0020] Or, the target microservice formats the target network threat log based on a JSONObject function, and extracts the target key information;

[0021] Or, the target microservice extracts the target key information from the target network threat log based on a regular expression.

[0022] Optionally, the binding of the target key information and the device information of the target network security device by the target microservice comprises:

[0023] The target microservice binds the target key information with the device IP, device name, device port, device type, creation time, and unique identification of the target network security device.

[0024] Optionally, after saving the bound information into the local preset data warehouse, the method further comprises:

[0025] Based on a query condition, the bound information in the preset data warehouse is queried and exported; the query condition is the target key information or the device information.

[0026] Optionally, the attacker key information includes source IP, source port, and source MAC; the attacked key information includes target IP, target port, and target MAC; the network attack behavior information includes attack type, threat level, attack state, and occurrence time; the network attack domain name information includes domain name and XFF header; the protection behavior information corresponding to the network attack behavior includes protection action; the key request information and key response information corresponding to the target network request carrying the network attack include url, host, and HTTP protocol request method, request body, request header, response body, and response header.

[0027] In a second aspect, the present application provides a network threat data entry device, applied to a preset entry system, the preset entry system being connected with different network security devices through different network ports; wherein the device comprises:

[0028] A microservice determination module is configured to determine a target network security device to be extracted from each of the network security devices, and determine a target microservice corresponding to the target network security device from a preset microservice cluster locally.

[0029] A protocol determination module is configured to determine a corresponding target transmission protocol from a preset transmission protocol set based on the data size and log transmission frequency of the network threat log in the target network security device.

[0030] The log acquisition module is configured to listen to the network port corresponding to the target network security device through the target microservice and by using a preset port listening instance, and acquire a target network threat log of the target network security device through a target transmission protocol.

[0031] The information extraction module is configured to extract target key information from the target network threat log based on a preset information extraction rule through the target microservice, wherein the target key information includes attacker key information, attacked party key information, network attack behavior information, network attack domain name information, protection behavior information corresponding to the network attack behavior, key request information corresponding to a target network request carrying the network attack, and key response information.

[0032] The information saving module is configured to bind the target key information and device information of the target network security device through the target microservice, and save the bound information to a preset data warehouse in the local.

[0033] In a third aspect, the present application provides an electronic device, comprising:

[0034] A memory configured to save a computer program;

[0035] A processor configured to execute the computer program to implement the network threat data entry method.

[0036] In a fourth aspect, the present application provides a computer readable storage medium configured to save a computer program; wherein the computer program is executed by a processor to implement the network threat data entry method.

[0037] In the present application, the preset input system includes establishing a connection with different network security devices through different network ports, different network security devices correspond to a single microservice in the preset microservice cluster of the preset input system, and the appropriate target transmission protocol is determined in the preset transmission protocol based on the single data size of the network security device and the log transmission frequency; the target microservice continuously monitors the network port connected with the target network security device, and obtains the target network threat log of the target network security device through the target transmission protocol; the target microservice extracts the target key information including attacker key information, attacked key information, network attack behavior information, network attack domain name information, network attack behavior corresponding protection behavior information, network attack carrying target network request corresponding key request information and key response information from the target network threat log based on the preset information extraction rule; the device information of the target network security device is bound to the above-mentioned target key information through the target microservice, and the bound information is saved to the local preset data warehouse. As can be seen from the above, the input system contains multiple microservice modules, each microservice module corresponds to a network security device, forming a distributed cluster architecture, and the microservice module only collects and processes the data byte stream sent by one network security device to obtain readable network threat logs. At this time, if the network security device in the system changes due to version upgrade or model replacement, only the corresponding microservice module needs to be replaced, so that sustainable expansion and sustainable optimization can be realized. In addition, after the microservice module extracts the information field in the network threat log, only the key information and the related information of the corresponding network security device are bound and entered into the data warehouse, so that the network security service personnel can omit the data screening time, directly query and export the required key information in the data warehouse by setting the query condition for subsequent operation, greatly reducing the workload. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only belong to the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0039] Figure 1 A network threat data input method flow chart disclosed by the present application;

[0040] Figure 2 The technical architecture of the network threat data input method disclosed by the present application;

[0041] Figure 3 The microservice step diagram disclosed by the present application;

[0042] Figure 4 A specific network threat data entry method flowchart disclosed by the present application;

[0043] Figure 5 A network threat data entry device schematic diagram disclosed by the present application;

[0044] Figure 6 An electronic device structure diagram disclosed by the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0046] Currently, the number of network security manufacturers in the market is increasing, and an enterprise may purchase network security devices of multiple manufacturers. In the face of network security devices of multiple brands and versions, the enterprise needs to invest more network security service personnel to check network threat logs generated by the network security devices one by one and extract valid information, and therefore there are problems such as heavy workload, high repetition, complex traceability, and easy statistical errors.

[0047] Therefore, in the prior art, network threat logs in different network security devices are collected and preprocessed and then stored, so as to be searched and checked by subsequent network security service personnel. However, different manufacturers and different versions of network security devices may generate multiple types of network threat logs, and version upgrading or expansion of network security devices in the system may cause compatibility problems.

[0048] Therefore, the present application provides a network threat data entry method to reduce the workload of network security service personnel while realizing sustainable expansion and optimization of data collection and entry.

[0049] Referring to Figure 1 The embodiments of the present application disclose a network threat data entry method, which can include:

[0050] Step S11, determining a target network security device to be extracted from each network security device, and determining a target microservice corresponding to the target network security device from a local preset microservice cluster;

[0051] In the network security device connected to the entry system, the target security device to be extracted is determined, and the target microservice corresponding to the target security device in the local preset microservice cluster is determined.

[0052] Each network security device is connected with the preset entry system through different network interfaces, and the network security device includes an IP protocol password machine, a security router, a line password machine, a firewall and the like, is an important tool for guaranteeing network security, can prevent unauthorized access, detect and resist network attacks, and monitor and record network activities.

[0053] It should be noted that the microservices in the preset microservice cluster are relatively independent, as shown in Figure 2 Each network security device in the data layer of the preset entry system corresponds to a microservice, and the work process of a single microservice does not affect other microservices. Even if a microservice fails, due to its independence, it will not cause the entire system to crash like a monolithic application.

[0054] In addition, network threat logs generated by different network security manufacturers and different versions of network security devices differ in format. Therefore, in order to adapt to the corresponding network security device, the target microservice also needs to select appropriate transmission protocols, port listening instances and information extraction rules according to the format of the network threat log generated by the network security device.

[0055] Step S12, determining a corresponding target transmission protocol from a preset transmission protocol set based on the data size and the log transmission frequency of the network threat log in the target network security device;

[0056] In this embodiment, the target transmission protocol used is determined according to the single data size and the transmission frequency of the network threat log in the target network security device.

[0057] In the embodiment, when the data quantity size is greater than the preset quantity and the log frequency is greater than the preset frequency, the target transmission protocol is determined as a TCP protocol (Transmission Control Protocol); when the data quantity size is not greater than the preset quantity and the log frequency is not greater than the preset frequency, the target transmission protocol is determined as a UDP protocol (User Datagram Protocol). It should be noted that the TCP protocol is a connection-oriented, reliable, and byte stream-based transmission layer protocol. For a single network threat log with large data quantity and high transmission frequency, the data integrity and accuracy are crucial, and any data loss or error may affect the accurate analysis and judgment of the network threat. Therefore, when the data quantity and the log frequency of the network threat log to be received are greater than the preset values, the TCP protocol is selected as the target transmission protocol. The UDP is a connectionless transmission layer protocol. For a network threat log with small data quantity and low transmission frequency, the connection does not need to be frequently established and maintained, and the data can be sent out faster to improve the transmission efficiency. In addition, compared with 20 bytes or more bytes of the TCP header, the UDP header has only 8 bytes, which can carry more effective data in limited data quantity, is suitable for log transmission with small data quantity, and can improve the utilization rate of network bandwidth. Therefore, when the data quantity and the log frequency of the network threat log to be received are not greater than the preset values, the UDP protocol is selected as the target transmission protocol.

[0058] In step S13, the target network security device corresponding to the target network security port is listened to by the target micro service and the preset port listening instance, and the target network threat log of the target network security device is obtained through the target transmission protocol.

[0059] In the embodiment, in order to be able to identify and receive the request for the service itself, the target micro service needs to listen to the network port connected with the target network security port by using the preset port listening instance, and obtain the network threat log generated by the target network security port based on the target transmission protocol.

[0060] To monitor network ports, a preset port monitoring instance can be selected based on the aforementioned target transport protocol. In one specific implementation, the target microservice uses a DatagramSocket to monitor the network port corresponding to the target network security device, and obtains the target network threat logs of the target network security device via the UDP protocol. It should be noted that a DatagramSocket is a communication socket specifically designed for the UDP protocol, and its operation fully conforms to the characteristics of the UDP protocol, such as directly sending and receiving datagrams, without guaranteeing data order, integrity, or reliability. DatagramSocket implements UDP communication by manipulating DatagramPackets. When sending data, the data is encapsulated in a DatagramPacket and then sent out via a DatagramSocket; when receiving data, the DatagramSocket listens on a designated port and extracts and processes the data from the received DatagramPacket.

[0061] In another specific implementation, the target microservice listens on the network port corresponding to the target network security device using a Socket, and obtains the target network threat logs of the target network security device via the TCP protocol. It should be noted that TCP is a connection-oriented, reliable, byte-stream-based transport layer protocol, while Socket acts as a protocol adaptation layer to some extent. Socket can translate application layer communication requirements into operations that the underlying network protocol can understand. For example, in TCP-based Socket communication, Socket handles operations such as the three-way handshake to establish a connection, sequence number management during data transmission, acknowledgment, and the four-way handshake to close the connection. Therefore, data sending and receiving can be achieved simply by using the interface provided by Socket, without directly handling the complex details of the TCP protocol.

[0062] It should be noted that the target network threat logs obtained by the microservice from network security devices are data byte streams, not in a form readable by the microservice. Therefore, as... Figure 3 As shown, after obtaining the data byte stream corresponding to the target network threat log generated by the target network security device through the target transmission protocol TCP or UDP, it is necessary to convert the data byte stream into a string by encoding it using GBK or UTF-8 using the encoding method provided by the network security vendor. If the target network threat log is encrypted, it is also necessary to decrypt the string into a readable log based on the encryption method provided by the network security vendor.

[0063] In step S14, the target microservice extracts target key information from the target network threat log based on a preset information extraction rule. The target key information includes attacker key information, attacked target key information, network attack behavior information, network attack domain name information, network attack behavior corresponding protection behavior information, key request information corresponding to a network attack carrying target network request, and key response information.

[0064] In this embodiment, the target microservice can extract target key information from the target network threat log obtained above by using a preset information extraction rule. It can be understood that the network threat log may contain a large amount of information, and the format and content thereof are often very complex, containing many details that are not critical for analyzing a specific network threat. Therefore, after extracting the target network threat log to obtain all information, in order to improve the efficiency of analyzing network threats, it is necessary to extract target important information from all information.

[0065] In addition, since the formats of network threat logs are different, in order to obtain all information of the target network threat log, as shown in the following table, after converting and decrypting the obtained target network threat log, it is necessary to extract information fields of the target network threat log based on the corresponding network threat log interpretation provided by the network security device manufacturer. It can be understood that the method of extracting information includes split function, JSONObject function, regular expression, etc., and one or a combination of several methods can be used to extract all information in the target network threat log. Figure 3

[0066] In the first specific embodiment, the target microservice extracts the target key information by splitting the target network threat log based on the split function. It can be understood that the network threat log usually has a certain format, and the information is separated by a specific separator. For example, a common Syslog format log, in which the timestamp, device identifier, message level, and event description are separated by spaces. The main function of the split function is to split a string into multiple substrings according to the specified separator and return a list containing these substrings. Therefore, for the above-mentioned Syslog format network threat log, the split function can be used to split it according to the space to obtain a list containing each part, and the target key information can be extracted based on the list.

[0067] ​In the second specific implementation, the target key information is extracted by the target microservice based on a JSONObject function to format the target network threat log. It can be understood that the target network threat log can be in a standard JSON (JavaScript Object Notation, JS key-value pair data) format. The JSON format data has a hierarchical structure composed of key-value pairs, in which the key is a string and the value can be a string, a number, a Boolean value, an array, an object, and the like. The JSONObject function is a tool specially used for processing JSON data, which can understand the hierarchical structure of JSON and the representation of key-value pairs to facilitate access and extraction of information in the target network threat log, and is not further filtered to extract the target key information.

[0068] In the third specific implementation, the target key information is extracted by the target microservice based on a regular expression from the target network threat log. When it is necessary to extract information with a specific pattern from the log, a regular expression can also be directly used. The regular expression is a tool for describing character patterns, which allows users to define a series of rules to match specific patterns in text. In the network threat log, information usually has a certain pattern. For example, an IP address follows a specific combination of numbers and separator patterns, dates and times have common formats, and attack types can contain specific keywords. By using the regular expression, these patterns can be accurately defined to find information that meets the pattern from complex log text and determine the target key information.

[0069] In this embodiment, the attacker key information includes a source IP, a source port, and a source MAC; the attacked key information includes a target IP, a target port, and a target MAC; the network attack behavior information includes an attack type, a threat level, an attack state, and an occurrence time; the network attack domain name information includes a domain name and an XFF (X-Forwarded-For, HTTP request header field) header; the protection behavior information corresponding to the network attack behavior includes a protection action; and the key request information and the key response information corresponding to the target network request carrying the network attack include a url (Uniform Resource Locator, uniform resource locator), a host, a request method, a request body, a request header, a response body, and a response header of an HTTP protocol.

[0070] Among them, the source IP, the source port, and the source MAC in the attacker key information are used to identify the network attack initiator, so as to track whether the attack is from an internal network or an external network. If a large amount of malicious traffic is detected to come from the same external IP address, measures can be taken to block it or further investigate its source.

[0071] The target IP, target port, and target MAC in the attacker's key information can determine the target threatened by the attack, helping to assess which network resources (such as servers, databases, key application systems, etc.) are at risk. For example, when the target IP corresponds to the core database server of the enterprise, higher-level defense and protection measures need to be taken immediately, and the possible data leakage risk needs to be assessed.

[0072] Network attack behavior information includes attack type, threat level, attack status, and occurrence time. Among them, the explicit attack type is crucial for choosing the appropriate response strategy. Different types of attacks require different defense and mitigation measures. For example, for SQL (Structured Query Language) injection attacks, the application's database query statements need to be checked and repaired; for malware infection, malware removal and system hardening may be required.

[0073] Network attack domain name information includes domain name and XFF. Among them, the domain name plays a key role in network attack tracing. When malicious domain names appear in network threat logs, security service personnel can track attackers through domain name registration information. Different network attacks may use the same malicious domain name as a command and control center, so by analyzing domain name-related logs, security service personnel can associate multiple seemingly independent attack events. XFF header information can help security service personnel more accurately analyze the source and path of network traffic.

[0074] In addition, the key request information and key response information corresponding to the target network request carrying the network attack can also help security service personnel analyze the network threat situation. Among them, the URL can help security service personnel determine the entry point of the attack, and establishing a malicious URL blacklist can not only identify malicious attacks but also help discover new threat patterns. Host helps to determine the target of attack, distinguish the attack situation of different servers, and adjust security policies according to the service classification.

[0075] HTTP protocol request method can help security service personnel distinguish operation types, check whether the operation permission is compliant, whether it is within the user's permission range, and ensure compliance with relevant regulations and standards. HTTP protocol request body is an important basis for detecting injection attacks and data tampering risks; HTTP protocol request header can be used to verify user credentials and identify unauthorized access attempts to discover identity theft or abuse of authority. HTTP protocol response body can determine whether the attack is successful and assess the degree of data leakage; HTTP protocol response header can assess the server's health status and check the security policy execution;

[0076] Step S15, binding the target key information and the device information of the target network security device by the target microservice, and saving the bound information to the local preset data warehouse.

[0077] In this embodiment, as shown in the data layer of the preset entry system, after obtaining the target key information, the target key information needs to be bound with the device information of the target network security device and stored in the local preset data warehouse. It can be understood that binding the target key information with the device information of the target network security device can help trace the source of network threats. Figure 2

[0078] Further, as the first line of defense for network security protection, network security devices record information such as the entry and exit of network traffic, detection of abnormal behavior, etc. For example, when a malicious IP address is found to be attempting an attack, by binding with the firewall device, it can be seen how the IP address broke through the firewall rules or was detected under which firewall policy, which helps to understand the attack path and the means that may be adopted by the attacker.

[0079] In this embodiment, storing the bound target key information in the data warehouse can coordinate the management of network threat logs generated by different network security devices, and can reduce the workload of security service personnel in collecting and analyzing data. In addition, the information recorded by each network security device is only a partial perspective of the network security situation. For example, firewall logs mainly record network access control information, including allowed or denied connections, source IP and destination IP, etc.; IDS logs focus on the details of detecting intrusion behavior, such as attack type, attack signature, etc. Storing these different sources of information in the same data warehouse can integrate these partial perspectives, thereby providing a more comprehensive understanding of the network security situation.

[0080] It can be understood that the data warehouse stores network security history records composed of target key information from multiple network threat log information. By analyzing these network security history records, long-term trends and change rules of network threats can be found.

[0081] ​From the above, the input system includes a plurality of microservice modules, each of which corresponds to a network security device, forming a distributed cluster architecture. The microservice module only collects and processes the data byte stream sent by one network security device to obtain readable logs. At this time, if the network security device in the system changes due to version upgrade or model replacement, causing the type of network threat log to change, only the corresponding microservice module needs to be replaced, thus realizing sustainable expansion and optimization. In addition, after the microservice module extracts the information field in the network threat log, only the key information is bound with the relevant information of the corresponding network security device and then input into the data warehouse. Therefore, network security service personnel can omit the time for screening data, directly query and export the required key information in the data warehouse by setting query conditions for subsequent operation, greatly reducing the workload.

[0082] Reference Figure 4 As shown in the figure, in order to speed up the work efficiency of security service personnel and ensure the comprehensiveness of the analysis result, the embodiment of the present application further provides a specific network threat data input method, which can include:

[0083] Step S21, determining a target network security device to be extracted from each network security device, and determining a target microservice corresponding to the target network security device from a local preset microservice cluster;

[0084] Step S22, determining a corresponding target transmission protocol from a preset transmission protocol set based on the data size and log transmission frequency of the network threat log in the target network security device;

[0085] Step S23, listening to the network port corresponding to the target network security device through the target microservice and using a preset port listening instance, and obtaining the target network threat log of the target network security device through the target transmission protocol;

[0086] Step S24, extracting target key information from the target network threat log based on a preset information extraction rule through the target microservice; the target key information includes attacker key information, attacked target key information, network attack behavior information, network attack domain name information, network attack behavior corresponding protection behavior information, key request information and key response information carried by the target network request of the network attack;

[0087] Step S25, binding the target key information with the device IP, device name, device port, device type, creation time and unique identifier of the target network security device through the target microservice, and saving the bound information to a local preset data warehouse.

[0088] In this embodiment, the device IP, device name, device port, device type, creation time and unique identifier of the target network security device are bound to the target key information, and are uniformly stored in the preset data warehouse locally.

[0089] The security service personnel can determine the specific network security device through which the attack traffic passes through the device IP of the target network security device, thereby constructing an attack path, combining it with the device name, and also helping the security personnel to quickly locate a specific device in a complex network environment and determine the role of different types of devices in the attack event. In addition, the device name can be combined with the unique identifier to ensure that the target key information stored in the preset data warehouse is consistent with the corresponding target network security device configuration.

[0090] The device type and port information can help the security service personnel to evaluate whether the existing security policy is effective according to the bound target key information. For example, for a specific port corresponding to a service that is often attacked, the firewall rule or intrusion detection rule can be strengthened.

[0091] In addition, since older devices may have more security risks, the creation time can also be a factor for the security service personnel to assess the risk of the target security device.

[0092] Step S26, based on the query condition, querying and exporting the bound information in the preset data warehouse; the query condition is the target key information or the device information.

[0093] In this embodiment, as shown in Figure 2 In the application layer of the preset input system, the preset data warehouse can filter and export the bound information that meets the requirements based on the query condition and display it to the security service personnel. It can be understood that the query condition can be one or more of the target key information or the device information.

[0094] It can be understood that by querying based on the target key information and the device information in the preset data warehouse, not only can the possible security threats be quickly filtered out, which helps to intercept the threats before they cause serious damage, but also after the security threat is confirmed, the relevant target key information and device information can be used for precise emergency response.

[0095] Further, based on the query and extraction functions in the preset data warehouse, different target key information in the network threat log can be associated and analyzed with the device information of the target network security device, past security events can be queried and traced, similar attack patterns or device abnormal conditions can be found, to help security service personnel clearly depict the overall picture of the attack, including the attack path, attack means and possible attack targets, and find the cause of the security vulnerability. For example, by extracting all relevant target key information and bound device information of a specific device in the past period of time, analyzing the configuration changes and traffic pattern changes of the device before and after each security event, and the like, the root cause of the security vulnerability, such as device configuration error, compatibility problem after software update, and the like, can be found.

[0096] The specific implementation process of steps S21 to S24 can refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0097] As can be seen, the bound information is saved in the preset data warehouse, at this time, the preset data warehouse saves the target key information and device information generated by different devices at different times, so that the security service personnel can filter the data in the preset data warehouse based on specific query conditions in the target key information and device information, to obtain more accurate and comprehensive data, thereby speeding up the work efficiency while ensuring the comprehensiveness of the analysis result.

[0098] Correspondingly, referring to Figure 5 The embodiment of the application further provides a network threat data input device, which is applied to a server and can include:

[0099] The micro-service determination module 11 is configured to determine a target network security device of the to-be-extracted data from each network security device, and determine a target micro-service corresponding to the target network security device from a preset micro-service cluster locally.

[0100] The protocol determination module 12 is configured to determine a target transmission protocol from a preset transmission protocol set based on the data size and the log transmission frequency of the network threat log in the target network security device.

[0101] The log acquisition module 13 is configured to listen to the network port corresponding to the target network security device through the target micro-service and by using a preset port listening instance, and acquire the target network threat log of the target network security device through the target transmission protocol.

[0102] The information extraction module 14 is configured to extract target key information from the target network threat log based on preset information extraction rules by the target microservice; the target key information includes attacker key information, attacked target key information, network attack behavior information, network attack domain name information, network attack behavior corresponding protection behavior information, key request information corresponding to the target network request carrying the network attack, and key response information;

[0103] The information saving module 15 is configured to bind the target key information and device information of the target network security device by the target microservice, and save the bound information to a local preset data warehouse.

[0104] As can be seen from the above, the input system includes a plurality of microservice modules, each microservice module corresponding to a network security device, forming a distributed cluster architecture. The microservice module only collects and processes the data byte stream sent by one network security device to obtain readable logs. At this time, if the network security device in the system changes due to version upgrade or model replacement, causing the type of network threat log to change, only the corresponding microservice module needs to be replaced, thus realizing sustainable expansion and sustainable optimization. In addition, after the microservice module extracts the information field in the network threat log, only the key information and the related information of the corresponding network security device are bound and recorded in the data warehouse, so that the network security service personnel can omit the data screening time, directly query and export the required key information in the data warehouse by setting the query condition for subsequent operation, greatly reducing the workload.

[0105] In some embodiments, the protocol determination module 12 includes:

[0106] The first protocol determination unit is configured to determine the target transmission protocol as the TCP protocol when the data size is greater than the preset quantity and the log frequency is greater than the preset frequency.

[0107] The second protocol determination unit is configured to determine the target transmission protocol as the UDP protocol when the data size is not greater than the preset quantity and the log frequency is not greater than the preset frequency.

[0108] In some embodiments, the log acquisition module 13 includes:

[0109] The first log acquisition unit is configured to listen to the network port corresponding to the target network security device by the target microservice and using DatagramSocket, and acquire the target network threat log of the target network security device by the UDP protocol.

[0110] A second log acquisition unit is configured to listen to the network port corresponding to the target network security device through the target microservice and using a Socket, and acquire a target network threat log of the target network security device through a TCP protocol.

[0111] In some embodiments, the information extraction module 14 comprises:

[0112] A first information extraction unit is configured to extract the target key information by splitting the target network threat log based on a split function through the target microservice.

[0113] A second information extraction unit is configured to extract the target key information by formatting the target network threat log based on a JSONObject function through the target microservice.

[0114] A third information extraction unit is configured to extract the target key information from the target network threat log based on a regular expression through the target microservice.

[0115] In some embodiments, the information saving module 15 comprises:

[0116] An information saving unit is configured to bind the target key information with a device IP, a device name, a device port, a device type, a creation time, and a unique identifier of the target network security device through the target microservice.

[0117] In some embodiments, the network threat data entry device further comprises:

[0118] An information query unit is configured to query and export the bound information in the preset data warehouse based on a query condition; the query condition is the target key information or the device information.

[0119] In some embodiments of the information extraction module 14, the attacker key information comprises a source IP, a source port, and a source MAC; the attacked key information comprises a target IP, a target port, and a target MAC; the network attack behavior information comprises an attack type, a threat level, an attack state, and an occurrence time; the network attack domain name information comprises a domain name and an XFF header; the protection behavior information corresponding to the network attack behavior comprises a protection action; and the key request information and the key response information corresponding to the target network request carrying the network attack comprise a url, a host, a request method, a request body, a request header, a response body, and a response header.

[0120] Further, the application also discloses an electronic device, Figure 6is a structural diagram of the electronic device 20 according to an exemplary embodiment, and the content in the figure should not be considered as any limitation on the use range of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps in the network threat data entry method disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the present embodiment can be an electronic computer.

[0121] In the present embodiment, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited here; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited here.

[0122] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0123] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the network threat data entry method executed by the electronic device 20 disclosed in any of the preceding embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0124] Further, the present application also discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the network threat data entry method disclosed in the preceding embodiments. For the specific steps of the method, please refer to the corresponding content disclosed in the preceding embodiments, which will not be repeated here.

[0125] The embodiments in the present specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same or similar parts between the embodiments, please refer to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and please refer to the method part for the relevant content.

[0126] Those skilled in the art will further appreciate that the units and algorithms described in connection with the examples disclosed herein can be implemented in electronic hardware, computer software, or both; and that the described examples are merely illustrative of the principles of the application. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0127] The steps of a method or algorithm described in connection with the examples disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The present application is not limited by these implementation possibilities.

[0128] Finally, it should be noted that the terms "comprises", "comprising", or other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0129] The above detailed description has set forth various examples of the technology disclosed herein. The description is made for the purpose of illustrating the general principles of the technology and its practical application, and thus described arrangements are not limited to the examples given. It is intended that the examples be considered in a descriptive sense only and not for purposes indicating the exclusive processes, methods, apparatus, or devices by which the processes, methods, apparatus, or devices are performed. It is further intended that the scope of the technology disclosed herein include all devices, apparatus, methods, and processes that are equivalent to those described.

Claims

1. A network threat data ingestion method, characterized by, The application is applied to a preset entry system which is connected with different network security devices through different network ports; wherein the method comprises: determining a target network security device for data extraction from each network security device, and determining a target microservice corresponding to the target network security device from a local preset microservice cluster; determining a corresponding target transmission protocol from a preset transmission protocol set based on the data size and log transmission frequency of the network threat log in the target network security device; listening to the network port corresponding to the target network security device through the target microservice and using a preset port listening instance, and obtaining the target network threat log of the target network security device through the target transmission protocol; extracting target key information from the target network threat log based on a preset information extraction rule through the target microservice; the target key information includes attacker key information, attacked key information, network attack behavior information, network attack domain name information, network attack behavior corresponding protection behavior information, key request information and key response information carried by the target network request of the network attack; binding the target key information and the device information of the target network security device through the target microservice, and saving the bound information to a local preset data warehouse.

2. The network threat data ingestion method as claimed in claim 1, wherein, The method comprises: when the data size is greater than a preset number and the log frequency is greater than a preset frequency, determining the target transmission protocol as a TCP protocol; when the data size is not greater than a preset number and the log frequency is not greater than a preset frequency, determining the target transmission protocol as a UDP protocol.

3. The network threat data ingestion method of claim 2, wherein, The method comprises: listening to the network port corresponding to the target network security device through the target microservice and using DatagramSocket, and obtaining the target network threat log of the target network security device through the UDP protocol; or, listening to the network port corresponding to the target network security device through the target microservice and using Socket, and obtaining the target network threat log of the target network security device through the TCP protocol.

4. The network threat data ingestion method as claimed in claim 1, wherein, The method comprises: extracting the target key information based on the split function through the target microservice and splitting the target network threat log by keyword; or, extracting the target key information by formatting the target network threat log based on the JSONObject function through the target microservice; Or, extracting the target key information from the target network threat log based on a regular expression by the target microservice.

5. The network threat data ingestion method as claimed in claim 1, wherein, The binding of the target key information and the device information of the target network security device by the target microservice comprises: Binding the target key information and the device IP, device name, device port, device type, creation time and unique identification of the target network security device by the target microservice.

6. The network threat data ingestion method as claimed in claim 5, wherein, After saving the bound information into the local preset data warehouse, it further comprises: Based on the query condition, the bound information in the preset data warehouse is queried and exported; the query condition is the target key information or the device information.

7. The network threat data ingestion method of any of claims 1 to 6, wherein, The attacker key information includes source IP, source port and source MAC; the attacked key information includes target IP, target port and target MAC; the network attack behavior information includes attack type, threat level, attack state and occurrence time; the network attack domain name information includes domain name and XFF header; the protection behavior information corresponding to the network attack behavior includes protection action; the key request information and key response information corresponding to the target network request carrying network attack include url, host and HTTP protocol request method, request body, request header, response body and response header.

8. A network threat data ingestion apparatus, comprising: Applied to a preset entry system, the preset entry system establishes a connection with different network security devices through different network ports; wherein the device comprises: A microservice determination module is configured to determine a target network security device to be extracted from each network security device, and determine a target microservice corresponding to the target network security device from a local preset microservice cluster; A protocol determination module is configured to determine a corresponding target transmission protocol from a preset transmission protocol set based on the data size and log transmission frequency of the network threat log in the target network security device; A log acquisition module is configured to listen to the network port corresponding to the target network security device through the target microservice and using a preset port listening instance, and acquire the target network threat log of the target network security device through the target transmission protocol; An information extraction module is configured to extract target key information from the target network threat log based on a preset information extraction rule by the target microservice; the target key information includes attacker key information, attacked key information, network attack behavior information, network attack domain name information, network attack behavior corresponding protection behavior information, key request information and key response information corresponding to the target network request carrying network attack; An information saving module is configured to bind the target key information and the device information of the target network security device by the target microservice, and save the bound information into a local preset data warehouse.

9. An electronic device, comprising: Comprise: A memory for saving a computer program; A processor for executing the computer program to implement the network threat data entry method of any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that, A computer program product for saving a computer program; wherein the computer program, when executed by a processor, implements the network threat data ingestion method according to any one of claims 1 to 7.

Images