A method and system for constructing a unified network security data platform

Abstract

This invention discloses a method and system for constructing a unified network security data platform, belonging to the field of network security technology. The method includes: receiving at least one type of heterogeneous security log data through an access layer; converting the heterogeneous logs into event objects conforming to the OCSF standard by a conversion engine, forming a conversion path from ECS to OCSF; semantically enhancing the OCSF event objects using the MITRE ATT&CK framework by an enhancement layer, automatically associating them with attack technology IDs; converting the enhanced OCSF event objects into threat intelligence packages conforming to the STIX 2.1 standard by an export layer, forming an export path from OCSF to STIX; and providing a RESTful interface through an API layer, which has a built-in security verification module for penetration testing and stress testing of the system. This invention integrates the three major standards ECS, OCSF, and STIX to construct a complete processing pipeline from data access, standardization, semantic enhancement to threat intelligence export, and incorporates a security verification mechanism, effectively solving the problem of difficult unified processing and sharing of heterogeneous security data, improving security operation efficiency and system deployment reliability.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a middleware system for uniformly processing heterogeneous log data from different security devices and its construction method. Background Technology

[0002] As enterprises deepen their digital transformation, cybersecurity threats are becoming increasingly complex. To address these threats, enterprises typically deploy various security devices from different vendors, such as firewalls, intrusion detection systems (IDS), and endpoint detection and response (EDR) platforms. The logs and alerts generated by these devices vary in format; for example, many Elastic Stack-based systems use Elastic Common Schema (ECS), while traditional devices may use Syslog or CEF formats.

[0003] Currently, there is a lack of a system capable of efficiently and uniformly accessing, standardizing, enhancing, and exporting this heterogeneous data. The main technical problems are as follows: 1. Inconsistent data standards: Data from different sources, due to their format differences, are difficult to integrate, analyze, and query, forming "data silos." 2. Lack of semantic association: Raw logs typically only contain low-level fields and cannot be automatically associated with high-level, understandable attack tactics and techniques (such as those defined in the MITRE ATT&CK framework). This forces security analysts to spend a significant amount of time on manual analysis, resulting in low efficiency. 3. Difficulty in intelligence export: Analyzed data cannot be easily converted into internationally accepted standard threat intelligence formats (such as STIX 2.1), limiting the sharing and automated consumption of intelligence within the enterprise and among external partners. 4. High system deployment risk: Existing solutions lack systematic security and performance verification before deployment, potentially leading to security vulnerabilities or performance bottlenecks in the production environment, impacting business continuity.

[0004] To address the aforementioned issues, some solutions exist in existing technologies. For example, Chinese patent CN120832866B discloses a format standardization method for sharing cybersecurity threat intelligence. However, this method primarily focuses on the conversion from multi-source data to STIX intelligence, without proposing a complete processing pipeline that organically integrates the three major standards: ECS, OCSF, and STIX. It also fails to address the semantic enhancement issue during the standardization process. Furthermore, this solution does not mention mechanisms for verifying the security and performance of the system itself. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method and system for constructing a unified network security data platform.

[0006] Technical solution This invention provides a method for constructing a unified network security data platform, comprising the following steps: Step S1: Receiving at least one type of heterogeneous security log data through the access layer, wherein the heterogeneous security log data includes at least logs conforming to the ECS standard; Step S2: The conversion engine converts the heterogeneous security log data into event objects conforming to the OCSF standard based on predefined field mapping rules, forming a conversion path from ECS standard logs to OCSF standard events; Step S3: The enhancement layer performs semantic enhancement on the standardized OCSF event objects using the MITRE ATT&CK framework, specifically including: extracting key features from the OCSF event objects, matching the extracted key features with a predefined attack pattern library associated with MITRE ATT&CK technology IDs, and adding enriched objects containing technology IDs and tactical information to the successfully matched OCSF event objects; Step S4: The export layer converts the semantically enhanced OCSF event objects into STIX-compliant objects. 2.1 Standard threat intelligence package, forming an export path from OCSF standard events to STIX standard intelligence; Step S5: The API layer provides a RESTful interface for interacting with the above steps. The API layer has a built-in security verification module for performing penetration testing and stress testing on the API interface.

[0007] Furthermore, in step S2, for source data fields that cannot be directly mapped, they are stored in custom fields of the OCSF event object.

[0008] Furthermore, the penetration tests performed by the security verification module include at least authentication bypass testing, SQL injection testing, and XSS attack testing; the stress tests include at least throughput testing, concurrency testing, and memory stability testing.

[0009] This invention also provides a unified network security data platform system for implementing the above method, comprising: an access module configured to receive heterogeneous security log data; a conversion module configured to convert heterogeneous security log data into event objects conforming to the OCSF standard based on predefined field mapping rules; an enhancement module configured to perform semantic enhancement on the OCSF event objects using the MITRE ATT&CK framework; an export module configured to convert the semantically enhanced OCSF event objects into threat intelligence packages conforming to the STIX 2.1 standard; an API module configured to provide a RESTful interface, wherein the API module has a built-in security verification unit; and a storage module configured to persist the processed data.

[0010] Beneficial effects Compared with existing technologies, this invention has the following beneficial effects: 1. It achieves complete integration of three standards (ECS-OCSF-STIX): This invention constructs an end-to-end processing pipeline from data access to threat intelligence export, integrating the log collection advantages of ECS, the data standardization advantages of OCSF, and the intelligence sharing advantages of STIX into a complete and standardized closed-loop system. 2. It enhances the semantic value of data: By innovatively introducing the MITRE ATT&CK framework into the processing flow of OCSF standard events, it achieves automatic semantic enhancement of raw logs, mapping low-level fields (such as command lines and ports) to high-level attack techniques (such as T1059.001), greatly improving the readability and analytical value of the data. 3. It improves the reliability and deployability of the system: By embedding a security verification module in the API layer, the system itself possesses verifiable and testable characteristics. This enables the system to automatically complete security vulnerability scanning and performance baseline testing before being deployed to the production environment, effectively reducing deployment risks. 4. Established intelligence sharing channels: By supporting the export of STIX 2.1 standard, internal security analysis results can be seamlessly connected to external threat intelligence platforms, promoting collaboration within the security ecosystem. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the overall architecture of the unified network security data platform system provided in an embodiment of the present invention.

[0012] like Figure 1 As shown, the system includes: API module (105), access module (101), conversion module (102), enhancement module (103), export module (104) and storage module (106).

[0013] The API module (105) serves as the unified entry point of the system, providing a RESTful interface to receive requests from external systems (such as SOC and SIEM platforms). This module integrates a security verification unit, configured to perform penetration testing and stress testing.

[0014] The access module (101) is connected to the API module (105) and is used to receive heterogeneous security log data, which includes at least logs conforming to the ECS standard.

[0015] The conversion module (102) is connected to the access module (101) and is configured to convert heterogeneous logs into OCSF standard events based on field mapping rules, forming a conversion path from ECS to OCSF.

[0016] The enhancement module (103) is connected to the conversion module (102) and configured to use the MITRE ATT&CK framework for semantic enhancement, and to add MITRE ATT&CK technology ID enrichment objects to successfully matched events.

[0017] The export module (104) is connected to the enhancement module (103) and configured to convert OCSF events into STIX 2.1 threat intelligence packages, forming an export path from OCSF to STIX.

[0018] The storage module (106) is connected to the conversion module (102), the enhancement module (103) and the export module (104) respectively, and is configured to persist the data during the processing.

[0019] Figure 1 Solid arrows indicate the direction of data flow, while dashed arrows indicate persistent data storage.

[0020] Figure 2 A data processing flowchart of a unified network security data platform system provided in an embodiment of the present invention.

[0021] like Figure 2 As shown, the process includes the following steps: Step S201 (Data Access): The system receives log data that conforms to the ECS standard as input to the processing flow.

[0022] Step S202 (Data Normalization Transformation): Based on predefined field mapping rules, convert ECS logs into OCSF standard event objects. This step specifically includes the following field mappings: @timestamp → time source.ip → src_endpoint.ip process.command_line → process.cmd_line x_custom_field → unmapped Step S203 (Semantic Enhancement): Semantic enhancement is performed on the standardized OCSF event objects using the MITRE ATT&CK framework. This step specifically includes: Extract key features, such as extracting command line content from the process.cmd_line field; The extracted key features are matched against a predefined attack pattern library. For example, command lines containing the pattern "powershell*-enc" are matched against MITRE ATT&CK technical ID T1059.001. For successfully matched events, enrichment objects are added. These enrichment objects contain a technology ID, technology name, confidence score, and corresponding tactical information.

[0023] Step S204 (Threat Intelligence Export): Convert the semantically enhanced OCSF event object into a threat intelligence package compliant with the STIX 2.1 standard. This step specifically includes: Convert OCSF events into STIX 2.1 observed-data objects; Transform the technical information in the enriched object into an attack-pattern object; Create a relationship object and associate the observed-data object with the attack-pattern object; All STIX objects are encapsulated into a single bundle as the final output.

[0024] Figure 2 The execution order of each step is indicated by a downward arrow. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1 This embodiment describes the specific structure and workflow of a unified network security data platform system. For example... Figure 1As shown, the unified network security data platform system of the present invention includes: an API module (105), an access module (101), a conversion module (102), an enhancement module (103), an export module (104), and a storage module (106). The API module (105) serves as the unified entry point of the system, providing a RESTful interface to receive requests from external systems (such as the SOC security operations center and the SIEM security information and event management platform). The API module (105) integrates a security verification unit configured to perform penetration testing and stress testing to verify the security and performance indicators of the system interface. The access module (101) is connected to the API module (105) and is used to receive heterogeneous security log data, which includes at least logs conforming to the ECS standard. The conversion module (102) is connected to the access module (101) and is configured to convert heterogeneous security log data into event objects conforming to the OCSF standard based on predefined field mapping rules, forming a conversion path from ECS standard logs to OCSF standard events. For source data fields that cannot be directly mapped, the conversion module (102) stores them in the unmapped field of the OCSF event object to ensure data integrity. The enhancement module (103) is connected to the conversion module (102) and configured to use the MITRE ATT&CK framework to perform semantic enhancement on the standardized OCSF event objects, adding enriched objects containing the MITRE ATT&CK technology ID to the successfully matched OCSF event objects. The export module (104) is connected to the enhancement module (103) and configured to convert the semantically enhanced OCSF event objects into threat intelligence packages conforming to the STIX 2.1 standard, forming an export path from OCSF standard events to STIX standard intelligence. The storage module (106) is connected to the conversion module (102), the enhancement module (103), and the export module (104) respectively and configured to persist the data during the processing. Figure 1 Solid arrows indicate data flow, while dashed arrows indicate persistent data storage. The following section combines... Figure 2Taking a specific ECS log as an example, the data processing flow of this system is explained in detail. Step S201 (Data Access): The API module (105) receives an ECS format log sent by an external system, such as a log containing a process creation event, which records information such as process name, command line, and source IP. Step S202 (Data Standardization Conversion): The conversion module (102) converts the ECS log into an OCSF standard event object according to predefined field mapping rules. The specific mapping includes: mapping the @timestamp field in the ECS log to the time field of the OCSF event object; mapping the source.ip field in the ECS log to the src_endpoint.ip field of the OCSF event object; mapping the process.command_line field in the ECS log to the process.cmd_line field of the OCSF event object; and storing custom fields in the ECS log that cannot be directly mapped (such as x_custom_field) into the unmapped field of the OCSF event object. After conversion, the original ECS log is transformed into an OCSF event object with a unified structure and standard fields. Step S203 (Semantic Enhancement): The enhancement module (103) extracts key features from the generated OCSF event object. For example, the command line content "powershell.exe -enc ..." is extracted from the process.cmd_line field. The enhancement module (103) matches this feature with a predefined attack pattern library. This pattern library has a pre-defined mapping relationship between the regular expression powershell.*-enc and the MITRE ATT&CK technology ID T1059.001 (PowerShell). After a successful match, the enhancement module (103) generates an enrichment array for the OCSF event object, which contains the technology ID "T1059.001", the technology name "PowerShell", the confidence score "0.9", and the related tactic "Execution". Step S204 (Threat Intelligence Export): The export module (104) converts the semantically enhanced OCSF event object into a threat intelligence package in STIX 2.1 format. Specifically: the OCSF event object itself is transformed into an observed-data object; the technical information in the enriched object is transformed into an attack-pattern object; a relationship object is created to associate the observed-data object with the attack-pattern object; finally, all STIX objects are encapsulated into a bundle and returned to the caller through the API module (105).Meanwhile, the storage module (106) can persist the raw data, intermediate results and final results in the processing to the database for historical query and tracing.

[0027] Example 2 This embodiment focuses on describing the specific implementation of the system's security verification module. For example... Figure 1 As shown, the API module (105) integrates a security verification unit. This security verification unit can perform security verification tasks during system deployment, periodically after system startup, or when triggered by maintenance personnel. The security verification unit has multiple pre-built test cases, including penetration test cases and stress test cases. Penetration test cases include at least: Authentication bypass test: Simulating an attacker accessing a protected endpoint without an API Key or with an incorrect API Key, verifying whether the system returns a 401 Unauthorized status code. SQL injection test: Constructing a request containing SQL injection payloads such as 'OR 1=1', sending it to interfaces such as / api / v1 / mitre / technique / , verifying whether the system returns a 400 or 404 error instead of executing an SQL query. XSS attack test: Constructing a request containing... <script>alert(1)< / script> The system performs a request payload test to verify whether the output is escaped and the script is not reflected into the response. Stress test cases should include at least the following: Throughput test: Under a preset number of concurrent threads (e.g., 20 threads), send a large number of requests to the / health or / api / v1 / transform interface, recording the number of requests per second (RPS) the system can handle. Concurrency test: Gradually increase the number of concurrent threads (e.g., from 10 threads to 50 threads), observe the changes in the system's average response time, P95 / P99 latency, and error rate, and identify the system's optimal operating point and saturation zone. Memory stability test: Under continuous stress, periodically collect process memory usage data to verify whether there are memory leaks or abnormal growth. After the security verification unit completes all test cases, it generates a structured test report, including the pass / fail status of each test case, the actual response code, and key performance indicators (such as throughput and latency). Operations personnel can use this report to determine whether the system meets the security and performance requirements for production environment deployment. If the test fails, the system can issue an alarm and prevent the current version from going live. Through the built-in security verification mechanism described above, this invention enables the system to automatically complete security vulnerability scanning and performance baseline testing before being deployed to the production environment, effectively reducing deployment risks.

[0028] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art can make various improvements and modifications without departing from the spirit and principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for constructing a unified network security data center, characterized in that, Includes the following steps: Step S1: Receive at least one type of heterogeneous security log data through the access layer, wherein the heterogeneous security log data includes at least logs conforming to the ECS (Elastic Common Schema) standard; Step S2: The transformation engine converts the heterogeneous security log data into event objects that conform to the OCSF (Open Cybersecurity Schema Framework) standard. The transformation process is based on predefined field mapping rules, which map fields in the source data to corresponding fields in the target OCSF event object, forming a transformation path from ECS standard logs to OCSF standard events. Step S3: The enhancement layer uses the MITRE ATT&CK framework to perform semantic enhancement on the standardized OCSF event objects. The semantic enhancement includes at least: extracting key features from the OCSF event objects, matching the extracted key features with a predefined attack pattern library, and adding enriched objects containing MITRE ATT&CK technology IDs to the successfully matched OCSF event objects. Step S4: The export layer converts the semantically enhanced OCSF event object into a threat intelligence package conforming to the STIX 2.1 standard, forming an export path from OCSF standard events to STIX standard intelligence; Step S5: The API layer provides RESTful interfaces for interacting with the above steps. The API layer has a built-in security verification module for performing penetration testing and stress testing on the API interface to verify the security and performance of the system.

2. The method of claim 1, wherein, The field mapping rules in step S2 are a configurable set of mapping rules. For source data fields that cannot be directly mapped, they are stored in the custom fields of the OCSF event object to ensure data integrity.

3. The method of claim 1, wherein, The semantic enhancement in step S3 further includes: calculating a confidence score based on the matching result, and storing the matched MITRE ATT&CK tactical information and the technology ID together in the enriched object.

4. The method of claim 1, wherein, The penetration tests performed by the security verification module include at least authentication bypass testing, SQL injection testing, XSS attack testing, and path traversal testing; the stress tests include at least throughput testing, concurrency testing, and memory stability testing.

5. The method of claim 1, wherein, It also includes step S6: persisting semantically enhanced OCSF event objects by a storage layer that supports SQLite or PostgreSQL databases.

6. A unified network security data hub system for implementing the method of any one of claims 1 to 5, characterized in that, include: Access module: configured to receive heterogeneous security log data; The conversion module is configured to convert the heterogeneous security log data into event objects that conform to the OCSF standard based on predefined field mapping rules, forming a conversion path from ECS standard logs to OCSF standard events. Enhancement module: Configured to use the MITRE ATT&CK framework to perform semantic enhancement on OCSF event objects, adding enriched objects containing MITRE ATT&CK technology IDs to successfully matched OCSF event objects; Export module: Configured to convert semantically enhanced OCSF event objects into threat intelligence packages compliant with the STIX 2.1 standard, forming an export path from OCSF standard events to STIX standard intelligence; API module: Configured to provide RESTful interfaces. The API module has a built-in security verification unit for penetration testing and stress testing of the API interface. Storage module: Configured to persist the processed data.

7. The system of claim 6, wherein, The enhancement module further includes: a feature extraction unit, used to extract process command lines, network ports, and file names as key features from OCSF event objects; and a pattern matching unit, used to match the key features with a predefined attack pattern library associated with MITRE ATT&CK technology IDs.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 5.

Images