Security vulnerability management method and system based on big data

Abstract

The invention relates to the technical field of security vulnerability analysis, in particular to a big data-based security vulnerability management method, which comprises the following steps of: constructing a real-time data acquisition strategy, capturing an internal source network equipment log, a server running state and application system flow data through a distributed log collection component, and storing the internal source network equipment log, the server running state and the application system flow data; meanwhile, vulnerability information of the exogenous threat intelligence platform is synchronized through an API interface; constructing a dynamic risk assessment model, and updating the risk assessment of the security vulnerability in real time by taking the four-dimensional model [IP address, service type, port number and software version] as a reference; and triggering a response strategy according to risk value grading, wherein the response strategy comprises automatic isolation of high-risk equipment, dynamic configuration of firewall rules and generation of a repair work order. According to the invention, through the synergistic effect of real-time data acquisition, dynamic asset modeling, intelligent risk assessment, a hierarchical response mechanism and a closed-loop verification system, the accuracy, efficiency and defense capability of security vulnerability management are significantly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of security vulnerability analysis, and in particular relates to a security vulnerability management method and system based on big data. Background Art

[0002] Existing security vulnerabilities mainly rely on devices such as firewalls. Vulnerability management generally adopts a regular update method, which lacks continuity and is not updated in a timely manner. Moreover, the feedback mechanism of security vulnerabilities is often processed by the user end after the failure, which is a post-processing and cannot be analyzed and processed in advance. It is difficult to accurately control and effectively deal with the risks of unknown vulnerabilities and effectively suppress the occurrence of security vulnerabilities.

[0003] For example, the invention patent with application number CN202411294142.3 proposes a security vulnerability analysis and management system based on big data, which includes: a data acquisition module, the data acquisition module is used to collect big data information such as operation logs, user behavior data, network traffic data, etc. from the target software and system; a vulnerability analysis module, the vulnerability analysis module is used to obtain the big data information in the data acquisition module, and obtain the protection strategy through the vulnerability analysis strategy; based on the initial integrated protection strategy of each event unit queue, determine the staged protection strategy to be activated for each vulnerability threat information; policy execution module, the policy execution module is used to execute the corresponding protection strategy automatically or under manual intervention according to the protection strategy in the vulnerability analysis module. However, the analysis of this security vulnerability is to analyze the operation logs, user behavior data, and network traffic data, and then directly enter the protection strategy, which makes it difficult to accurately control and effectively handle the risks of unknown vulnerabilities.

[0004] To this end, this application aims to provide a system that can intelligently analyze security vulnerabilities based on big data, predict the future risks of vulnerabilities, and determine and optimize management methods based on risk situations. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art, solve or at least alleviate the problems of the separation of risk control and copyright management, the reliance on biometric authentication hardware, and the lack of coordination between blockchain and artificial intelligence in the field of digital media copyright transactions, and provide a security vulnerability management method and system based on big data.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a security vulnerability management method based on big data, comprising:

[0007] Build a real-time data collection strategy, capture internal network device logs, server operation status, and application system traffic data through distributed log collection components, and synchronize vulnerability information from external threat intelligence platforms through API interfaces;

[0008] Build a dynamic risk assessment model and update the risk assessment of security vulnerabilities in real time based on the four-dimensional model of [IP address, service type, port number, software version];

[0009] Trigger vulnerability-asset intelligent correlation analysis when asset changes are detected, and start the risk assessment engine only when the impact scope of external vulnerability intelligence matches the specific service version recorded in the asset library;

[0010] Trigger response strategies based on risk level, including automatic isolation of high-risk devices, dynamic configuration of firewall rules, and generation of repair work orders;

[0011] After repair, the effectiveness of protection is verified through containerization and sandbox testing, and successful cases are converted into disposal rules and written into the knowledge base.

[0012] Preferably, the real-time data collection mechanism includes: using distributed log collection system components to collect endogenous data, and synchronizing the vulnerability number, hazard level and affected product information of the National Information Security Vulnerability Sharing Platform and the Weibo Online Threat Intelligence Platform.

[0013] Preferably, the dynamic asset fingerprint library updates asset portraits regularly and reorganizes asset information immediately when a new service deployment or version upgrade is detected.

[0014] Preferably, the risk assessment engine adopts a composite calculation model:

[0015] Risk value = basic CVSS score × asset exposure factor.

[0016] Preferably, the hierarchical response strategy includes:

[0017] When the risk value is greater than the first threshold, device isolation, malicious IP blocking, and automatic patch deployment will be performed;

[0018] When the risk value is between the first and second thresholds, dynamically configure WAF rules to limit traffic and push manual review tickets;

[0019] When the risk value is less than the second threshold, a repair work order is generated and an overdue automatic escalation mechanism is set;

[0020] The second threshold is smaller than the first threshold.

[0021] Preferably, the operations of the containerized verification sandbox include:

[0022] Clone the production environment configuration and inject vulnerability attack vectors. If the unauthorized access request returns a successful repair status code, the repair is considered effective. Otherwise, a disposal policy upgrade is triggered.

[0023] Preferably, when successful cases are converted into disposal rules and written into the knowledge base, the risk model parameters are optimized to reduce the CVSS weight coefficient of false positive events by 0.2 and increase the behavioral anomaly detection sensitivity threshold of missed positive events by 15%.

[0024] Preferably, the knowledge base system constructs a decision support system based on historical cases, and uses a relational database to store structured repair case records. The core fields include key dimensions such as vulnerability identifier, affected software name, version range restriction, and verified effective repair measures.

[0025] A security vulnerability management system based on big data, comprising:

[0026] Data collection module, used to obtain internal and external source data through distributed log components and API interfaces;

[0027] Asset fingerprint management module, used to maintain dynamically updated four-dimensional asset portraits;

[0028] Risk assessment engine, which performs vulnerability-asset correlation analysis and risk value calculation;

[0029] The response strategy execution module calls automated tools to perform protection operations according to risk levels;

[0030] Repair verification module, testing the effectiveness of repairs through containerized sandbox and updating the knowledge base.

[0031] The repair verification module is integrated with the container orchestration framework, supports rapid cloning of the production environment and simulates attack vector injection.

[0032] The present invention significantly improves the accuracy, efficiency and defense capabilities of security vulnerability management through the synergy of real-time data collection, dynamic asset modeling, intelligent risk assessment, hierarchical response mechanism and closed-loop verification system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is the main flow chart of the present invention.

[0034] Figure 2 This is a flow chart of the security vulnerability repair process of the present invention.

[0035] Figure 3 This is a block diagram of the security vulnerability management system of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0037] Example 1

[0038] This embodiment provides an intelligence-driven network security vulnerability management system based on a big data environment, including the following steps:

[0039] First, a real-time data collection mechanism is established to capture internal network device logs, server operation status and application system traffic data through distributed log collection components. At the same time, external intelligence data sources are synchronized through API interfaces, including vulnerability numbers, hazard levels and affected product information from the National Information Security Vulnerability Sharing Platform and the Weibo Online Threat Intelligence Platform.

[0040] Secondly, a dynamic asset fingerprint library is built, and the asset portrait is updated every 5 minutes based on the four-dimensional model [IP address, service type, port number, software version]. When a newly deployed service or version upgrade is detected, the asset information reorganization is immediately triggered.

[0041] Then, an intelligent vulnerability-asset correlation analysis is performed. The risk assessment engine is started when and only when the product range affected by the external vulnerability intelligence intersects with the specific service version recorded in the asset library. The engine uses a composite calculation model to output a risk value = basic CVSS score × asset exposure coefficient, where the core database exposure coefficient is set to 1.5 and the office terminal exposure coefficient is set to 0.8.

[0042] The response strategy is triggered based on the risk value classification: when the risk value is greater than the first threshold, the high-risk disposal process is automatically executed, including immediate isolation of the device, blocking of malicious IP addresses, and patch deployment through the manufacturer's API. When the risk value is between the second threshold and the first threshold, the Web application firewall rules are dynamically configured to implement access restriction and push manual review work orders. When the risk value is less than the second threshold, a 72-hour repair work order is generated and an automatic upgrade mechanism is set for overdue work orders.

[0043] Among them, the real-time data collection mechanism achieves comprehensive integration of internal and external source data through a distributed architecture. In terms of internal data collection, an agent-based log collection framework is adopted, and lightweight collection agents are deployed on each network device, physical server, virtual machine and container instance to continuously capture full-dimensional data including security audit logs of network devices, server CPU, memory, disk and other operating indicators, and HTTP request headers and response characteristics of application systems. These raw data are transmitted to the central processing platform through message queues. After format standardization, outlier cleaning and feature extraction, they are stored in a high-performance time series database. In terms of external intelligence synchronization, a standardized API connection channel is established, and the national information security vulnerability sharing platform and commercial threat intelligence services are automatically polled every hour to obtain key information such as the standard vulnerability number, hazard level, affected product version range, vulnerability exploitation characteristics and repair suggestions. After semantic analysis and ontology mapping, this external intelligence is associated with the internal source data and indexed to form a unified security data lake. The entire collection process adopts a streaming processing architecture to ensure data freshness at the thousandth of a second, providing real-time data support for subsequent analysis.

[0044] The asset fingerprint library, as the core infrastructure of the present invention, adopts a four-dimensional model to accurately characterize asset characteristics. The model uses IP address as the index basis and combines service type, port number and software version to build a complete asset portrait. At the data collection layer, a dual mechanism of active detection and passive monitoring is deployed: the active detection module performs a full network scan every 5 minutes, and determines the service type and version information of the open port through protocol handshake and version identification algorithm; the passive monitoring module continuously analyzes the protocol characteristics and application identification in the network traffic to capture asset change events. When the system detects new server deployment, container startup, software version upgrade or network topology adjustment, it immediately triggers the asset library reorganization process. This dynamic update mechanism ensures that the asset view is always synchronized with the actual environment, achieving 99.5% accuracy in testing. The asset library adopts a distributed key-value storage architecture, supports millisecond-level update response, and establishes a version history tracing function, which can trace back the asset status at any point in time, providing a complete chain of evidence for event investigation.

[0045] The risk assessment engine is the intelligent decision-making center of the present invention. It adopts a condition-triggered association analysis model. When the system receives new vulnerability intelligence, it first extracts key attributes such as the affected version range, and then scans all devices matching the version range in the asset library. Only when there is an exactly matching asset can the engine start the risk assessment calculation. This design avoids invalid scanning and improves system efficiency by more than 30%. In the risk calculation stage, a composite assessment model is adopted: the basic CVSS score is multiplied by the asset exposure coefficient to determine the final risk value. The exposure coefficient is set according to the asset's business importance. The core database server is weighted 1.5 times, the Internet-exposed application server is weighted 1.2 times, the internal management system maintains a baseline value of 1.0, and the office terminal equipment is downgraded to 0.8. This dynamic weighting mechanism successfully avoided 85% of risk misjudgments in bank system testing. For example, the risk value of an Oracle vulnerability in a database server increased from 7.8 to 11.7 after weighting, triggering the highest level response. When the calculated result exceeds 10.0, it is automatically processed as the highest risk, ensuring that critical assets receive priority protection.

[0046] A tiered response system based on risk values ​​enables precise resource scheduling. For ultra-high-risk vulnerabilities (above the first threshold), the system immediately initiates an emergency response protocol. First, the target device's network connection is isolated by modifying switch configurations and host firewall rules. At the same time, attack signature filtering rules are deployed on the perimeter firewall to block malicious IP addresses. An automated script is then invoked to distribute security patches and restart services. The entire process is completed within 120 seconds, triggering the highest security alert. For high-risk vulnerabilities (between the second and first thresholds), the system implements defensive control measures: dynamically configuring protection rules in the web application firewall to intercept attack signature traffic, implementing access rate limits on suspicious sources, and generating remediation tickets requiring manual review by the security team. A 30-minute timeout automatically escalates the timeout. For medium- and low-risk vulnerabilities (below the second threshold), a standardized response process is implemented: a 72-hour remediation ticket is created, with a responsible person automatically assigned and marked as "pending" in the asset library. The system sends an alert 12 hours before the expiration date. If the issue is not addressed within the deadline, the priority is automatically escalated and the supervisor is notified. This tiered mechanism has optimized resource allocation and saved 40% of security operations costs on e-commerce platforms.

[0047] Example 2

[0048] After the repair is completed, the containerized verification sandbox is started, the production environment configuration is cloned and the corresponding vulnerability attack vector (such as SQL injection payload) is injected. When an unauthorized access request is detected and a repair success status code is returned, the repair is considered effective, otherwise the disposal strategy upgrade is triggered; finally, the successful case is converted into a disposal rule and written into the knowledge base, and the risk model parameters are optimized simultaneously - false positive events reduce the CVSS weight coefficient by 0.2, and missed events increase the behavioral anomaly detection sensitivity threshold by 15%.

[0049] Kubernetes-based containerized sandbox technology is used in the repair verification phase. Upon receiving a repair completion notification, the system automatically extracts the target environment's detailed configuration from the asset library, including parameters such as the operating system version, software dependencies, and network topology. The system then precisely clones the production environment in an isolated sandbox cluster. Verifiers then select corresponding attack vector scripts based on the vulnerability signature library, simulating real-world attack scenarios for penetration testing. Effectiveness is determined using a dual standard: for authentication vulnerabilities, the system checks whether unauthorized access returns a successful repair prohibition status code. For code execution vulnerabilities, the system logs are monitored for traces of malicious payload execution. Verification results trigger different disposition processes: Successful defense cases immediately generate disposition rules and are written to the knowledge base. For example, the experience of remediating an Apache vulnerability was transformed into a persistent rule that mandates a mandatory upgrade to version 2.0 or higher when detecting a Log4j2 vulnerability. Verification failures trigger a 0.5 risk value increase and a reassignment of a work order, while also notifying the security team for analysis. In actual testing of financial systems, this mechanism has eliminated 12% of repair blind spots.

[0050] The knowledge base system builds a decision support system based on historical cases and uses a relational database to store structured repair case records. The core fields include key dimensions such as vulnerability identifier, affected software name, version range restriction, and verified effective repair measures. The system sets up an automated rule generation engine. When the container sandbox confirms that the protection is effective, it extracts vulnerability features and disposal solutions to generate standard rules. These rules form reusable knowledge assets. In the rule application stage, the system establishes a multi-level matching mechanism: new vulnerability alerts first scan historical similar cases in the knowledge base, give priority to matching records with exactly the same vulnerability numbers, then match the version range of the same software, and finally refer to the disposal solutions of similar products.

[0051] The present invention establishes a continuously evolving learning system. When a false alarm occurs, the system automatically starts a parameter calibration process: it reduces the CVSS basic score weight of the corresponding vulnerability type by 0.2, and simultaneously adjusts the sensitivity of the correlation analysis to avoid the recurrence of similar errors. In a case where an office system falsely reported an Oracle vulnerability, this mechanism lowered the weight from the second threshold, and subsequent similar alarms decreased by 90%. When a missed alarm occurs, the system increases the abnormal behavior detection threshold by 15%, enhances the depth of log analysis, expands the monitoring coverage, and stores all parameter adjustment records in the audit log. Version backtracking analysis is supported, and the security team can view the historical adjustment trajectory to evaluate the optimization effect. During a six-month trial run, this mechanism promoted a 42% increase in system accuracy, achieving a fundamental shift from static rules to dynamic learning.

[0052] The system of the present invention adopts a layered distributed architecture design and is divided into four functional areas at the hardware deployment level. The data collection layer deploys three high-performance servers to form a log collection cluster, which is connected to the core switch through a 10 Gigabit fiber optic network to ensure the daily collection throughput of billions of logs. The analysis and computing layer is equipped with a GPU acceleration server to run the risk assessment engine and machine learning model. The distributed memory database cluster is connected to the RDMA high-speed network to store asset fingerprints. The response execution layer adopts a dual-active control node architecture to achieve high-availability execution of automated scripts and connects to the firewall management system through a dedicated channel. The verification sandbox layer constructs a three-node Kubernetes cluster. Each computing node is configured with a 32-core CPU, 256GB of memory and an NVMe storage pool. An isolated network domain is created through VXLAN technology. In actual deployment, financial customers choose a private deployment model to integrate the four-layer architecture into the infrastructure of two locations and three centers, while small and medium-sized enterprises can adopt a modular cloud service solution to subscribe to functional components on demand.

[0053] The system workflow forms a complete closed loop. After obtaining the original logs and threat intelligence from the data collection layer, they are input into the analysis and calculation layer for asset modeling and risk assessment; the generated disposal instructions are passed to the response execution layer to dispatch security equipment; after the repair is completed, the verification sandbox is triggered to verify the effect; the final results are deposited into the knowledge base to optimize subsequent decision-making.

[0054] This system can effectively reduce the risk of potential security vulnerabilities, form a quantifiable security value system, and continuously improve proactive defense capabilities by combining threat hunting and attack simulation technologies.

[0055] The contents not described in detail in the specification belong to the existing technology known to those skilled in the art, and the model parameters of each electrical appliance are not specifically limited, and conventional equipment can be used. In this technical solution, the electrical control components not mentioned are not shown in the figure because they belong to the existing technology and are not described here.

[0056] The above are only preferred specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A security vulnerability management method based on big data, characterized in that: include: Build a real-time data collection strategy, capture internal network device logs, server operation status, and application system traffic data through distributed log collection components, and synchronize vulnerability information from external threat intelligence platforms through API interfaces; Build a dynamic risk assessment model and update the risk assessment of security vulnerabilities in real time based on the four-dimensional model of [IP address, service type, port number, software version]; Trigger response strategies based on risk level, including automatic isolation of high-risk devices, dynamic configuration of firewall rules, and generation of repair work orders; After repair, the effectiveness of protection is verified through containerization and sandbox testing, and successful cases are converted into disposal rules and written into the knowledge base.

2. The security vulnerability management method according to claim 1, characterized in that: Vulnerability-asset intelligent correlation analysis is triggered when an asset change is detected, and the risk assessment engine is launched only when the impact scope of external vulnerability intelligence matches the specific service version recorded in the asset library.

3. The security vulnerability management method according to claim 1, wherein: The real-time data collection mechanism includes: using distributed log collection system components to collect endogenous data, and synchronizing the vulnerability numbers, hazard levels and affected product information of the National Information Security Vulnerability Sharing Platform and the Weibo Online Threat Intelligence Platform.

4. The security vulnerability management method according to claim 1, wherein: The dynamic asset fingerprint library updates asset portraits regularly and reorganizes asset information immediately when a new service deployment or version upgrade is detected.

5. The security vulnerability management method according to claim 1, wherein: The risk assessment engine uses a composite calculation model: Risk value = basic CVSS score × asset exposure factor.

6. The security vulnerability management method according to claim 1, wherein: The tiered response strategy includes: When the risk value is greater than the first threshold, device isolation, malicious IP blocking, and automatic patch deployment will be performed; When the risk value is between the first and second thresholds, dynamically configure WAF rules to limit traffic and push manual review tickets; When the risk value is less than the second threshold, a repair work order is generated and an overdue automatic escalation mechanism is set; The second threshold is smaller than the first threshold.

7. The security vulnerability management method according to claim 1, characterized in that: The operations of the containerized verification sandbox include: Clone the production environment configuration and inject vulnerability attack vectors. If the unauthorized access request returns a successful repair status code, the repair is considered effective. Otherwise, a disposal policy upgrade is triggered.

8. The method according to claim 1, characterized in that While converting successful cases into handling rules and writing them into the knowledge base, the risk model parameters are optimized to reduce the CVSS weight coefficient of false positive events by 0.2 and increase the behavioral anomaly detection sensitivity threshold of missed positive events by 15%.

9. A security vulnerability management system based on big data, used to implement the security vulnerability management method according to claims 1-8, characterized in that: include: Data collection module, used to obtain internal and external source data through distributed log components and API interfaces; Asset fingerprint management module, used to maintain dynamically updated four-dimensional asset portraits; Risk assessment engine, which performs vulnerability-asset correlation analysis and risk value calculation; The response strategy execution module calls automated tools to perform protection operations according to risk levels; Repair verification module, testing the effectiveness of repairs through containerized sandbox and updating the knowledge base.

10. The system according to claim 9, characterized in that The repair verification module integrates the K container orchestration framework, supports rapid cloning of production environments and simulates attack vector injection.

Images