An automated scanning and early warning management system for network security vulnerabilities based on big data

Abstract

The present invention discloses an automatic scanning and early warning management system for network security vulnerabilities based on big data, and specifically relates to the technical field of network security vulnerabilities. An automatic scanning and early warning management system for network security vulnerabilities based on big data includes a collection module, a processing module, and a push module, and each module is connected by a signal; the present invention obtains matching information and scanning information of network components, establishes a network component evaluation index, evaluates a new batch of deployed network components, and screens out network components that need to be scanned for vulnerabilities, thereby saving scanning resources and reducing resource waste. At the same time, by obtaining system status information and network component information, a vulnerability scanning evaluation index is established, and the screened network components are further evaluated to determine whether to immediately perform vulnerability scanning on them, so that the timing of vulnerability scanning is more reasonable, avoiding scanning when the system is under high load or during critical business periods, thereby reducing the impact on system performance.

Description

Technical Field

[0001] The present invention relates to the technical field of network security vulnerabilities, and more specifically, to a network security vulnerability automatic scanning and early warning management system based on big data. Background Art

[0002] Vulnerabilities are defects in the specific implementation of hardware, software or protocols or in system security policies, which can enable attackers to obtain user privacy and damage the system through certain technical means. Vulnerability scanning technology is designed to detect the existence of these vulnerabilities and provide users with scanning results in the form of reports, pointing out possible attacks and providing users with patching measures that can be taken for each vulnerability, helping users avoid attackers from exploiting vulnerabilities accidentally left in restricted computers, components, applications or other online resources to launch attacks.

[0003] When a batch of new network components need to be deployed to the production environment, existing technologies usually scan all components for vulnerabilities. However, comprehensive vulnerability scanning of a large number of network components may take a lot of time, affecting the normal operation of the business. At the same time, if the system is under high load, high traffic, and high concurrency, scanning the newly deployed network components immediately may affect system performance, causing the system to respond slowly or be interrupted.

[0004] In order to solve the above two defects, a technical solution is now provided. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a network security vulnerability automatic scanning early warning management system based on big data to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An automatic scanning and early warning management system for network security vulnerabilities based on big data, including a data acquisition module, a data processing module, and a data analysis module, wherein there is a connection relationship between the modules;

[0008] A data collection module is used to collect matching information and scanning information of network components and send the collected information to the data processing module for processing;

[0009] A data processing module, used for obtaining a network component evaluation index by weighted summation calculation based on the matching information and scanning information of the network component, and sending the calculation result to the data analysis module;

[0010] The data analysis module is used to compare the network component evaluation index with the network component evaluation index reference threshold value to screen out the network components that need to be scanned for vulnerabilities.

[0011] In a preferred embodiment, the data collection module is used to collect matching information and scanning information of network components;

[0012] The matching information of the network components includes the component similarity coefficient, and the scanning information includes the scanning time coefficient;

[0013] The component similarity coefficient and the scanning time coefficient are marked as .

[0014] In a preferred embodiment, the logic for obtaining the component similarity coefficient is as follows:

[0015] The basic images and software packages used in the newly deployed network components are numbered according to different types, and the numbers are represented by numbers;

[0016] Match the base images and software packages with different numbers with the base images and software packages in the deployed network components, and record the number of successful matches and mark them as , and record the total number of matches, marking it as ;

[0017] The calculation expression of component similarity coefficient is as follows .

[0018] In a preferred embodiment, the acquisition logic of the scan time coefficient is as follows:

[0019] The basic images and software packages used in the newly deployed network components are numbered according to different types, and the numbers are represented by numbers;

[0020] Match the base images and software packages with different numbers with the base images and software packages in the deployed network components. If the match is successful, obtain the time node of the last vulnerability scan of the successfully matched base image and software package from the historical records, and calculate the scan duration. The expression is as follows ,in is the time when the currently evaluated network component will be deployed, The time of the last vulnerability scan of the successfully matched basic image and software package;

[0021] Establish the average value of the scan time, the expression is , where The average scan time is the scanning time of each successfully matched basic image and software package, i represents the serial number of each successfully matched basic image and software package, i={1, 2, 3, ..., n}, n is a positive integer;

[0022] The scan time coefficient is calculated by the average scan time. The expression is as follows , where is the scan time coefficient, The scanning time for each successfully matched base image and software package, is the average scan time.

[0023] In a preferred embodiment, the data processing module is used to obtain the network component evaluation index by weighted sum calculation according to the matching information and scanning information of the network component, and send the calculation result to the data analysis module.

[0024] In a preferred embodiment, the data analysis module is used to compare the network component evaluation index with a network component evaluation index reference threshold;

[0025] If the network component evaluation index is less than the network component evaluation index reference threshold, a vulnerability scan of the network component is required.

[0026] In a preferred embodiment, the data acquisition module is also used to collect system status information and network component information;

[0027] The system status information includes the connection interruption frequency and network bandwidth utilization, and the network component information includes the interaction response time and the component priority quantization value.

[0028] In a preferred embodiment, the data processing module is further used to obtain the vulnerability scanning assessment index by weighted sum calculation based on the system status information and network component information.

[0029] In a preferred embodiment, the data analysis module is further used to compare the vulnerability scanning assessment index with a vulnerability scanning assessment index reference threshold;

[0030] If the vulnerability scan assessment index is greater than or equal to the vulnerability scan assessment index reference threshold, it needs to be scanned immediately.

[0031] Technical effects and advantages of the present invention:

[0032] 1. The present invention obtains matching information and scanning information of network components, establishes a network component evaluation index, evaluates a new batch of deployed network components, and compares the network component evaluation index with a reference threshold of the network component evaluation index to screen out network components that need to be scanned for vulnerabilities, thereby saving scanning resources, reducing waste of resources, improving scanning efficiency, and being able to respond to new deployment requirements more quickly. At the same time, there is no need to wait for all components to complete scanning, thereby speeding up the online speed, reducing business interruption time, and improving business continuity.

[0033] 2. The present invention establishes a vulnerability scanning evaluation index by acquiring system status information and network component information, evaluates the screened network components, and compares the vulnerability scanning evaluation index with a vulnerability scanning evaluation index reference threshold to determine whether to immediately perform a vulnerability scan on them. By combining the importance of network components and the operating status of the system, the timing of vulnerability scanning is made more reasonable, avoiding scanning when the system is under high load or during critical business periods, thereby reducing the impact on system performance, and giving priority to processing network components with higher importance, reducing the exposure time of vulnerabilities, and improving the security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;

[0035] Figure 1 This is a schematic diagram of the structure of Example 1 of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Embodiment 1, as Figure 1 As shown, in this embodiment, a network security vulnerability automatic scanning and early warning management system based on big data includes a data acquisition module, a data processing module, and a data analysis module, and there is a connection relationship between the modules;

[0038] A data collection module is used to collect matching information and scanning information of network components and send the collected information to the data processing module for processing;

[0039] A data processing module, used for obtaining a network component evaluation index by weighted summation calculation based on the matching information and scanning information of the network component, and sending the calculation result to the data analysis module;

[0040] The data analysis module is used to compare the network component evaluation index with the network component evaluation index reference threshold value to screen out the network components that need to be scanned for vulnerabilities.

[0041] This embodiment is based on the network security vulnerability automated scanning technology to scan the security vulnerabilities of the network components newly deployed in the production environment, to ensure the security of the newly deployed network components, and to further improve the security and reliability of the system. If all newly deployed network components are fully scanned, it will undoubtedly take a lot of time and affect the normal operation of the business. Before scanning the network components for vulnerabilities, this embodiment collects the matching information and scanning information of the network components, evaluates all newly deployed network components, and screens out the network components that need to be scanned for vulnerabilities, so as to avoid spending a lot of time scanning all network components for vulnerabilities, resulting in waste of resources and redundancy of vulnerabilities.

[0042] The data collection module collects matching information and scanning information of network components;

[0043] The matching information of the network components includes the component similarity coefficient, and the scanning information includes the scanning time coefficient;

[0044] The component similarity coefficient and the scanning time coefficient are marked as ;

[0045] The data processing module normalizes the component similarity coefficient and the scanning time coefficient to calculate the network component evaluation index, which is expressed as follows:

[0046]

[0047] In the formula, Evaluation index for network components, are the preset proportional coefficients of component similarity coefficient and scanning time coefficient respectively, where .

[0048] Component similarity coefficient: The component similarity coefficient reflects the similarity between each newly deployed network component and the already deployed network components. The higher the component similarity coefficient, the higher the network component evaluation index, and the probability of vulnerability scanning will be greatly reduced.

[0049] The similarity is mainly considered from two aspects: the base image and the software package. It should be noted that the factors considered for comparing the similarity include but are not limited to the base image and the software package, which can be determined according to the actual situation.

[0050] Number the base images and software packages used in the newly deployed network components according to different types. The numbers are represented by numbers, for example, {1,2,3...}. The numbering rule is to number the base images first and then the software packages.

[0051] In an optional example, if there are 2 types of base images and 3 types of software packages used in the newly deployed network components, they are numbered {1, 2, 3, 4, 5}, where 1 and 2 are numbers of different types of base images, and 3, 4, 5 are numbers of different types of software packages.

[0052] Match the base images and software packages with different numbers with the base images and software packages in the deployed network components, and record the number of successful matches and mark them as , and record the total number of matches, marking it as ;

[0053] The calculation expression of the component similarity coefficient is as follows: .

[0054] Scan time coefficient: The basic images and software packages used in the newly deployed network components are numbered according to different types. The numbers are represented by numbers, for example, {1,2,3...}. The numbering rules are the same as above and will not be repeated here.

[0055] The base images and software packages with different numbers are matched with the base images and software packages in the deployed network components. If the match is successful, the time node of the last vulnerability scan of the successfully matched base image and software package is obtained from the historical records, and the scanning duration is calculated. The expression is as follows: ,in is the time when the currently evaluated network component will be deployed, The time of the last vulnerability scan of the successfully matched basic image and software package;

[0056] Establish the average value of the scan time, the expression is , where is the average scanning time, is the scanning time of each successfully matched basic image and software package, i represents the serial number of each successfully matched basic image and software package, i={1, 2, 3, ..., n}, n is a positive integer;

[0057] The scan time coefficient is calculated by the average scan time, and the expression is as follows: , where is the scan time coefficient, The scanning time for each successfully matched base image and software package, is the average scan time.

[0058] The smaller the scanning time coefficient is, the higher the network component evaluation index is, and thus the probability of performing vulnerability scanning on it will be greatly reduced.

[0059] The data processing module sends the calculated network component evaluation index to the data analysis module.

[0060] The data analysis module compares the network component evaluation index with a network component evaluation index reference threshold.

[0061] If the network component evaluation index is greater than or equal to the network component evaluation index reference threshold, it means that the network component has a high similarity with the deployed network component and the vulnerability scanning time is short, so there is no need to perform vulnerability scanning on the network component;

[0062] If the network component evaluation index is less than the network component evaluation index reference threshold, it means that the network component has low similarity with the already deployed network components and may carry more unknown network security vulnerabilities, and a vulnerability scan needs to be performed on the network component.

[0063] The present invention obtains matching information and scanning information of network components, establishes a network component evaluation index, evaluates a new batch of deployed network components, and compares the network component evaluation index with a reference threshold of the network component evaluation index to screen out network components that need to be scanned for vulnerabilities, thereby saving scanning resources, reducing waste of resources, improving scanning efficiency, and being able to respond to new deployment requirements more quickly. At the same time, there is no need to wait for all components to complete scanning, thereby speeding up the online speed, reducing business interruption time, and improving business continuity.

[0064] Example 2: In the above example, by obtaining the matching information and scanning information of the network components, a new batch of deployed network components are evaluated to screen out the network components that need to be scanned for vulnerabilities. However, whether the vulnerability needs to be scanned immediately still needs further comprehensive evaluation. If the system is under high load, high traffic, and high concurrency, scanning the newly deployed network components immediately may affect the system performance, causing the system response to slow down or be interrupted. This example obtains the system status information and network component information, establishes a vulnerability scanning evaluation index, and screens out the network components that need to be scanned for vulnerabilities immediately. The specific steps are as follows:

[0065] The data acquisition module collects system status information and network component information;

[0066] System status information includes connection interruption frequency and network bandwidth utilization, and network component information includes interaction response time and component priority quantization value;

[0067] The connection interruption frequency, network bandwidth utilization, interaction response time, and component priority quantization values ​​are marked as ;

[0068] The data processing module normalizes the connection interruption frequency, network bandwidth utilization, interactive response time, and component priority quantization value to calculate the vulnerability scanning assessment index, which is expressed as follows:

[0069]

[0070] In the formula, is the vulnerability scan assessment index, are the preset proportional coefficients of component priority quantization value, connection interruption frequency, network bandwidth utilization, and interactive response time, respectively. .

[0071] A connection interruption is a situation in which a connection is abruptly terminated or interrupted during network communication due to various reasons. Frequent connection interruptions may be an indicator of system stability issues. If the system has problems such as crashes, insufficient resources, or misconfiguration, connection interruptions may increase.

[0072] There are many types of connections, such as:

[0073] TCP connection: TCP connection is usually used to transmit data, such as web page requests, file transfers, etc. Monitoring the TCP connection status can help you understand the server load and whether there are too many connection anomalies that cause system performance problems.

[0074] UDP connection: User Datagram Protocol (UDP) is a connectionless protocol suitable for scenarios such as real-time communication and broadcasting. Monitoring the UDP connection status can help you understand how the system is handling real-time data streams.

[0075] Database connection: For database servers, the connection status refers to whether the active connection established with the database is abnormal. Too many database connections may cause database performance to degrade and increase system load.

[0076] Network connections: This includes various connections established with other servers, devices or services, such as HTTP connections, FTP connections, etc.

[0077] The logic for obtaining the connection interruption frequency is as follows: Select monitoring tools including but not limited to Prometheus, Zabbix, Nagios, Grafana, etc., monitor the above different types of connections, and record the number of connection interruptions, which is marked as , select a time window and mark it as For example, one minute, five minutes or one hour. The specific time window setting can be set according to different time scenarios, which will not be described here. The calculation expression for the connection interruption frequency is: .

[0078] Network bandwidth utilization: Network bandwidth utilization refers to the ratio between the actual amount of data transmitted and the available network bandwidth. It reflects the utilization of the network connection. A high network bandwidth utilization indicates that the system may be transmitting a large amount of data. At this time, vulnerability scanning will also generate a large amount of data traffic, which may affect network performance and cause system instability or even crash. The calculation expression of network bandwidth utilization is: , where The actual amount of data transmitted refers to the amount of data transmitted through the network within a preset time period. Available bandwidth refers to the maximum transmission rate of the network connection. The actual amount of transmitted data and available bandwidth can be obtained in real time through network monitoring tools, such as Wireshark, ntop, SolarWinds and other network monitoring tools.

[0079] Interaction response time: Network components usually include applications, and the interaction response time refers to the waiting time experienced by users when interacting with applications. Specifically, the interaction response time includes the time required for the system to return a response (such as page loading, result display, etc.) after the user sends a request (such as clicking a button, submitting a form, etc.). The longer the waiting time experienced by users when interacting with applications, the worse the performance of the network component, and thus the higher the vulnerability scan assessment index.

[0080] In the above embodiment 1, if the network component evaluation index is less than the network component evaluation index reference threshold, the network components that need to be scanned for vulnerabilities are screened out. In this embodiment, the network components that need to be scanned for vulnerabilities are sorted in descending order according to the network component evaluation index to obtain a network component priority sorting table. ,in Represents the priority level of each network component.

[0081] The logic for obtaining the component priority quantization value is as follows: the network component priority in the network component priority ranking table is mapped to obtain the component priority quantization value. The higher the component priority quantization value, the more important the network component is, and the more immediate vulnerability scanning is required, thus the greater the vulnerability scanning evaluation index.

[0082] In an optional example, the mapping relationship between the network component priority level and the component priority quantization value is as described in the following table:

[0083]

[0084] Wherein, m is the total number of network components screened in Example 1.

[0085] The data processing module sends the calculated vulnerability scanning assessment index to the data analysis module.

[0086] The data analysis module compares the vulnerability scanning assessment index with a vulnerability scanning assessment index reference threshold.

[0087] If the vulnerability scan assessment index is greater than or equal to the vulnerability scan assessment index reference threshold, it means that the importance of the network component is high and the system performance is relatively stable at this time, so it can be scanned immediately;

[0088] If the vulnerability scan assessment index is less than the vulnerability scan assessment index reference threshold, it means that the stability of the system is poor at this time, and the importance of the network component is low, and it does not need to be scanned for vulnerabilities immediately, and the vulnerability scan can be delayed.

[0089] The present invention establishes a vulnerability scanning evaluation index by acquiring system status information and network component information, evaluates the screened network components, and compares the vulnerability scanning evaluation index with a vulnerability scanning evaluation index reference threshold to determine whether to immediately perform a vulnerability scan on them. By combining the importance of network components and the operating status of the system, the timing of vulnerability scanning is made more reasonable, avoiding scanning when the system is under high load or during critical business periods, thereby reducing the impact on system performance, and giving priority to processing network components with higher importance, reducing the exposure time of vulnerabilities, and improving the security of the system.

[0090] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0091] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0092] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0093] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0096] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A network security vulnerability automatic scanning and early warning management system based on big data, characterized by: It includes data acquisition module, data processing module and data analysis module, and there is a connection relationship between the modules; A data collection module is used to collect matching information and scanning information of network components and send the collected information to the data processing module for processing; A data processing module, used for obtaining a network component evaluation index by weighted summation calculation based on the matching information and scanning information of the network component, and sending the calculation result to the data analysis module; A data analysis module is used to screen out network components that need to be scanned for vulnerabilities based on the comparison between the network component evaluation index and the network component evaluation index reference threshold; A data collection module, used to collect matching information and scanning information of network components; The matching information of the network components includes the component similarity coefficient, and the scanning information includes the scanning time coefficient; The component similarity coefficient and the scanning time coefficient are marked as ; The logic for obtaining component similarity coefficients is as follows: The basic images and software packages used in the newly deployed network components are numbered according to different types, and the numbers are represented by numbers; Match the base images and software packages with different numbers with the base images and software packages in the deployed network components, and record the number of successful matches and mark them as , and record the total number of matches, marking it as ; The calculation expression of component similarity coefficient is as follows ; The logic for obtaining the scan time coefficient is as follows: The basic images and software packages used in the newly deployed network components are numbered according to different types, and the numbers are represented by numbers; Match the base images and software packages with different numbers with the base images and software packages in the deployed network components. If the match is successful, obtain the time node of the last vulnerability scan of the successfully matched base image and software package from the historical records, and calculate the scan duration. The expression is as follows ,in is the time when the currently evaluated network component will be deployed, The time of the last vulnerability scan of the successfully matched basic image and software package; Establish the average value of the scan time, the expression is , where The average scan time is the scanning time of each successfully matched basic image and software package, i represents the serial number of each successfully matched basic image and software package, i={1, 2, 3, ..., n}, n is a positive integer; The scan time coefficient is calculated by the average scan time. The expression is as follows , where is the scan time coefficient, The scanning time for each successfully matched base image and software package, is the average scanning time; The data processing module is used to obtain the network component evaluation index through weighted sum calculation according to the matching information and scanning information of the network component, and send the calculation result to the data analysis module.

2. According to the big data-based network security vulnerability automatic scanning early warning management system of claim 1, it is characterized by: A data analysis module, for comparing the network component evaluation index with a network component evaluation index reference threshold; If the network component evaluation index is less than the network component evaluation index reference threshold, a vulnerability scan of the network component is required.

3. According to the big data-based network security vulnerability automatic scanning early warning management system of claim 1, it is characterized by: The data acquisition module is also used to collect system status information and network component information; The system status information includes the connection interruption frequency and network bandwidth utilization, and the network component information includes the interaction response time and the component priority quantization value.

4. According to claim 3, the network security vulnerability automatic scanning early warning management system based on big data is characterized by: The data processing module is also used to obtain the vulnerability scanning assessment index through weighted sum calculation based on the system status information and network component information.

5. The network security vulnerability automatic scanning and early warning management system based on big data according to claim 4 is characterized by: The data analysis module is further used to compare the vulnerability scanning assessment index with a vulnerability scanning assessment index reference threshold; If the vulnerability scan assessment index is greater than or equal to the vulnerability scan assessment index reference threshold, it needs to be scanned immediately.

Images