A parallel embedding-based heterogeneous vulnerability data fusion method and system

Abstract

The present application belongs to but is not limited to the technical field of information security, and particularly relates to a heterogeneous vulnerability data fusion method and system based on parallel embedding, comprising: S1, collecting vulnerability data for a target environment: operating and configuring a vulnerability scanning engine management module, issuing a scanning task to the managed vulnerability scanning engine or tool, and obtaining various vulnerability data related to the target system; S2, collecting, analyzing and extracting the original vulnerability data, and storing the data into a database; S3, performing data structure conversion on the preliminary formatted vulnerability data: the heterogeneous vulnerability data conversion module converts each vulnerability scanning result into a corresponding graph data structure for the various vulnerability data results obtained in step S2; S4, fusing the multiple subgraph data generated in step S3; and S5, extracting the fusion result generated in step S4 and generating a report.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of information security technology, and in particular relates to a method and system for fusion of heterogeneous vulnerability data based on parallel embedding. Background Art

[0002] In practice, inconsistent results are often found between vulnerability tools (such as web vulnerabilities) from different vendors and types, requiring manual analysis of scan results. This is tedious and time-consuming for post-test data summarization and organization. Currently, the main approaches for automating the fusion of vulnerability scan results include static matching and mapping or relying on national standards for format specification. Among these, relying on CVE vulnerability numbers for association suffers from poor universality and cannot comprehensively and reliably fuse large amounts of heterogeneous vulnerability data.

[0003] In view of the above analysis, the technical problems that need to be solved urgently in the existing technology are:

[0004] Existing vulnerability detection tools work independently and output heterogeneous data formats, making it difficult to perform integrated analysis on the same platform. This results in network security teams spending a long time and being inefficient when analyzing multi-source vulnerability data.

[0005] Traditional vulnerability scanning methods are often limited to single-dimensional vulnerability detection and fail to fully explore the potential correlations between different vulnerabilities.

[0006] In existing technologies, security analysts need to spend a lot of time manually organizing and analyzing reports generated by different tools, making it difficult to respond to rapidly changing network security threats in real time.

[0007] Traditional methods rely on static reports and manual analysis, making it difficult to respond to new threats in dynamic network environments in real time. Summary of the Invention

[0008] In response to the problems existing in the prior art, the present invention provides a method and system for fusion of heterogeneous vulnerability data based on parallel embedding, which mainly provides a solution and system for the management of multiple vulnerability scanning tools and fusion of heterogeneous result data in network security tests.

[0009] The present invention is implemented as follows: a heterogeneous vulnerability data fusion method based on parallel embedding, comprising:

[0010] S1. Collect vulnerability data for the target environment: operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engine or tool, and obtain various vulnerability data related to the target system (vulnerabilities, source code defects, backdoors);

[0011] S2. Collect, parse, and extract raw vulnerability data, and store the data in a database: Because the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and parsing module automatically parses the format and extracts attribute values ​​for each vulnerability data. After parsing and extracting data, the processing results and the raw data are stored in a document-based database.

[0012] S3. Convert the initially formatted vulnerability data into a data structure: The heterogeneous vulnerability data conversion module converts each vulnerability scan result from the various vulnerability data analyzed in step S2 into a corresponding graph data structure. Furthermore, to improve the accuracy of subsequent fusion steps, the heterogeneous vulnerability data conversion module also receives data from the threat intelligence knowledge base and associates and expands the vulnerability data from step S2.

[0013] S4. Fuse the multiple subgraph data generated in step S3: The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize the entities in the multiple subgraphs, merge the similar data and repeated entities in the multiple subgraphs according to the similarity of the vectors, and after obtaining the fusion results of the multiple subgraphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence.

[0014] S5. Extract the fusion results generated in step S4 and generate a report: The vulnerability result report generation module reads the blank fields required by the report template, traverses the comprehensive graph generated in step S4 and performs necessary calculations, and finally generates a fused vulnerability report.

[0015] Furthermore, when a network security analyst faces a target system, they create a unified task through the vulnerability scanning engine management module. This task includes the vulnerability types to be scanned and the configuration information required for various vulnerability scanning tools. Once the network security analyst completes the scanning task, each vulnerability scanning engine begins sending pre-configured messages of different types to the target system, analyzing the returned information and generating relevant results.

[0016] Furthermore, S3 specifically includes:

[0017] S301: First, traverse all types of vulnerability data, and traverse according to the vulnerability type selected when creating the scanning task in step S1;

[0018] S302, converting vulnerability data into a subgraph;

[0019] S303: After the entity type is extracted, a secondary dictionary mapping process is set up, namely "extracted entity type - intermediate entity type - preset entity type", so that all vulnerability result data forms structured data in a unified format;

[0020] S304. The basic components of the threat intelligence library include the National Information Security Vulnerability Database (CNNVD), the Common Vulnerability and Exposure Database (CVE), the Common Weakness Enumeration (CWE), and other vulnerability libraries with wide application scope and recognized reasonable scoring methods, and each vulnerability data has a unique ID and other attributes.

[0021] Furthermore, S4 specifically includes:

[0022] First, the subgraph is vectorized. The subgraph is expressed as G = (E, R, T), where R represents the node in the graph data structure, R represents the edge in the graph data structure, and T represents the attribute of the node. The three subgraphs generated in step 3 can be expressed as G1 = (E1, R1, T1), G2 = (E2, R2, T2), and G3 = (E3, R3, T3). After the formal definition, the formal definition of the problem to be solved by the present invention (e1, e2) can also be given, e1∈E1, e2∈E2, that is, to determine whether any entity pair (e1, e2) is the same entity in the target system. If it is the same entity, the different subgraphs are connected through the entity as a common point pair, and the entity pair is merged into one entity node, retaining the original node information.

[0023] Furthermore, in the subgraphs generated in step 3, two subgraphs are randomly selected to start the fusion process. That is, all entity pairs (e1, e2) are enumerated and vectorized using the TransE model to judge, and the initial fusion result G'1 is obtained. Then, one subgraph is randomly selected from the remaining subgraphs and fused with G'1 to obtain G'2. This process is repeated until all subgraphs are fused to obtain G' n-1 , n is the number of subgraphs generated in step 3.

[0024] Furthermore, the construction process of the TransE model, translation model and judgment model used in the fusion process mainly includes model building and model training.

[0025] The model is constructed using the TransE model as the basic model to complete the vectorization of entities and entity relationships (i.e., embedding). In addition, the TransE model is followed by a cascade translation model M, which is implemented using a multi-layer fully connected neural network to perform linear transformation on the vectorized entity pair (e1, e2). In this embodiment, Tr(e1) and Tr(e2) represent the results of vectorizing the entities using the TransE model. Spatial translation of Tr(e1) can be expressed as M. ijTr(e1), where i and j represent the dimensions before and after the transformation, respectively. In this embodiment, i and j are both 512. Finally, the judgment model is also implemented using a multi-layer fully connected neural network and cascaded after the translation model M. Its formal expression is f(M ij Tr(e1), Tr(e2)), the model output is binary, i.e. 0 or 1, representing whether the entity pair (e1, e2) can be fused.

[0026] The key parts of model training are the construction of the loss function and the training data set. From the formal definition of the problem to be solved by the present invention in S401, it can be seen that the ultimate goal of the model is to make the vector distance between the same or similar entities as close as possible, and the vector distance between different entities as far as possible. Therefore, in this embodiment, the following loss function is used to train the TransE model and the translation model.

[0027]

[0028] In the above formula, S and S' represent the positive sample dataset and negative sample dataset, respectively. The positive sample dataset contains identical or similar entity pairs, while the negative sample dataset contains identical or similar entity pairs. The judgment model is a general binary classification model, trained using the binary cross entropy function as the loss function, as shown in the following formula.

[0029]

[0030] Furthermore, S5 specifically includes:

[0031] S501. First, a breadth-first search algorithm is used to traverse the result data. A starting vertex is randomly selected, and all adjacent vertices are first visited before expanding outward layer by layer. Since the result data is based on the expanded graph data, supplementary nodes such as the host, operating system, and basic middleware are removed during the traversal process, leaving only the fused vulnerable node information.

[0032] S502: Substitute the above traversal results into a report template in a list form to form a vulnerability result fusion report.

[0033] Another object of the present invention is to provide a heterogeneous vulnerability data fusion system based on parallel embedding for implementing the heterogeneous vulnerability data fusion method based on parallel embedding, comprising:

[0034] Vulnerability Data Engine Management Module: Operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engines or tools, and obtain various vulnerability data related to the target system (vulnerabilities, source code defects, backdoors);

[0035] Vulnerability Data Collection and Parsing Module: Collects, parses, and extracts raw vulnerability data, and stores the data in a database. Because the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and parsing module automatically performs format parsing and attribute value extraction for each vulnerability data. After parsing and data extraction are completed, the processing results and the raw data are stored simultaneously in a document-based database.

[0036] Heterogeneous Vulnerability Data Conversion Module: This module converts each vulnerability scan result into a corresponding graph data structure based on the various vulnerability data obtained through analysis. Furthermore, to improve the accuracy of subsequent fusion steps, the module also receives data from the threat intelligence knowledge base and associates and expands the vulnerability data.

[0037] Heterogeneous vulnerability data fusion module: fuses the generated multiple subgraph data. The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize the entities in multiple subgraphs, and merge similar data and repeated entities in multiple subgraphs according to the degree of vector similarity. After obtaining the fusion results of multiple subgraphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence.

[0038] Vulnerability result report generation module: extracts the generated fusion results and generates a report: The vulnerability result report generation module reads the blank fields required by the report template, traverses the generated comprehensive graph and performs necessary calculations, and finally generates a fused vulnerability report.

[0039] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the heterogeneous vulnerability data fusion method based on parallel embedding.

[0040] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the heterogeneous vulnerability data fusion method based on parallel embedding.

[0041] Another object of the present invention is to provide an information data processing terminal, which includes the heterogeneous vulnerability data fusion system based on parallel embedding.

[0042] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0043] First, the present invention solves the technical problems in existing experimental work, such as low efficiency and poor accuracy of vulnerability data fusion, easy occurrence of a large amount of vulnerability duplication information, poor scalability of vulnerability scanning result fusion, and simple association methods. The system is used to efficiently and accurately complete the unified management of vulnerability scanning, result fusion, and scalability.

[0044] Second, technical problem solving:

[0045] 1) Solve the problem of difficulty in integrating heterogeneous data from multiple tools

[0046] Existing vulnerability detection tools operate independently, outputting heterogeneous data formats, making it difficult to integrate and analyze them on a single platform. This results in time-consuming and inefficient analysis of multi-source vulnerability data for network security teams. This invention standardizes the raw data generated by different scanning tools through a unified result collection interface and formatting method, resolving the incompatibility issue of heterogeneous data and laying the foundation for subsequent data fusion and threat analysis.

[0047] 2) Improve the relevance and depth of vulnerability detection results

[0048] Traditional vulnerability scanning methods are often limited to detecting vulnerabilities in a single dimension, failing to fully explore the potential correlations between different vulnerabilities. This invention, by constructing a subgraph structure of vulnerability data and leveraging a threat intelligence library for correlation expansion, can reveal the interrelationships between different vulnerabilities and possible attack paths, improving the depth of vulnerability detection and the scientific nature of decision-making.

[0049] 3) Reduce the workload of manual analysis and decision-making

[0050] In existing technologies, security analysts spend a significant amount of time manually organizing and analyzing reports generated by various tools, making it difficult to respond to rapidly evolving network security threats in real time. This invention utilizes parallel embedding technology to uniformly represent heterogeneous vulnerability data as high-dimensional feature vectors and automatically generate vulnerability situation maps, significantly reducing the workload of manual data processing and enabling rapid vulnerability assessment and decision support.

[0051] 4) Realize real-time and high efficiency

[0052] Traditional methods, relying on static reports and manual analysis, struggle to respond to new threats in dynamic network environments in real time. This invention combines real-time monitoring, data fusion, and situational awareness technologies to rapidly assess and dynamically manage target system vulnerabilities, significantly improving the efficiency and accuracy of vulnerability detection.

[0053] Technological advancements:

[0054] 1) Significant improvement in data fusion capabilities

[0055] This invention solves the problem of integrating heterogeneous data from multiple tools and proposes a unified vulnerability representation method based on parallel embedding, which can analyze vulnerability information from multiple dimensions and provide a more comprehensive and accurate reference for enterprise security protection.

[0056] 2) Intelligent Development of Vulnerability Detection

[0057] By expanding subgraph information in the threat intelligence library and automatically generating vulnerability situation maps, the present invention shifts vulnerability analysis from passive static detection to active dynamic perception, improving the intelligence level of security analysis and laying the foundation for predictive security protection.

[0058] 3) Efficient and low-cost network security solutions

[0059] Compared with traditional methods, this invention significantly reduces the time cost and manpower investment of vulnerability detection through automated collection, analysis and decision-making processes, while providing real-time and accurate security response capabilities, and has good industrial application value.

[0060] 4) Universal technical solutions applicable to multiple scenarios

[0061] This invention is not only suitable for vulnerability detection of a single system, but can also be extended to security management of complex network environments, cross-cloud systems and heterogeneous platforms, promoting the widespread application of network security technology in multiple fields such as industry, finance, and government affairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a method for fusion of heterogeneous vulnerability data based on parallel embedding provided by an embodiment of the present invention;

[0063] Figure 2 This is a structural diagram of a heterogeneous vulnerability data fusion system based on parallel embedding provided by an embodiment of the present invention;

[0064] Figure 3 Schematic diagram of a preset security entity model used in the heterogeneous vulnerability data fusion method based on parallel embedding provided by an embodiment of the present invention;

[0065] Figure 4 It is a schematic diagram of the process of data transfer transformation in the heterogeneous vulnerability data fusion method based on parallel embedding provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0067] Example 1: Enterprise Intranet Vulnerability Detection and Management

[0068] In an enterprise intranet environment, network administrators need to perform vulnerability detection on the entire network system to prevent potential threats, including the security of servers, databases, and employee terminal devices.

[0069] 1) Configure the scan task

[0070] Network administrators create a scan task in the vulnerability scanning engine management module, select weak password scanning, host vulnerability scanning, and application vulnerability scanning tools, and configure scan parameters such as target IP address range, weak password dictionary, and vulnerability type. After the task is launched, the scanning tool sends probe packets to collect security data from servers and terminal devices.

[0071] 2) Collect scan data

[0072] The raw result data generated by the scanning tool is automatically collected through the API interface. For example, the information returned by the database scanning tool may include unencrypted sensitive forms and outdated SQL engine versions. All raw result data is imported into the system and initially formatted in preparation for subsequent analysis.

[0073] 3) Data fusion and analysis

[0074] The system constructs multiple subgraphs based on the scan results. For example, one subgraph represents host vulnerability information, while another represents database vulnerability information. Subgraph data is then expanded using the threat intelligence library, for example by associating known Common Vulnerability Encoding (CVE) numbers with patch information. Finally, parallel embedding technology is used to fuse all subgraph information to generate a unified vulnerability landscape, helping administrators quickly understand the security status of their intranet.

[0075] 4) Decision support

[0076] Based on the integrated vulnerability data, the system provides priority remediation recommendations, such as recommending immediate database engine updates or modifying weak password configurations for high-risk vulnerabilities. It also provides report export capabilities to support companies in subsequent rectification and compliance reviews.

[0077] Example 2: Vulnerability Assessment in a Cross-Cloud Environment

[0078] A large internet company uses a multi-cloud architecture, encompassing both public and private clouds. The security team needs to conduct security assessments on containerized applications across these cloud environments to ensure the reliability and security of cloud services.

[0079] 1) Configure scanning tools and policies

[0080] Security teams use the method described in this paper to create scanning tasks, including weak password scanning for containers in public cloud services and source code vulnerability scanning tools in private clouds. They configure the cloud service provider's API access permissions and container target list, and set the path and scanning rules for the source code scanning tool. Once the task is launched, the tool automatically detects security vulnerabilities across the cloud environment.

[0081] 2) Scan result collection

[0082] Scanning tools in public clouds use APIs to return container vulnerability information, such as out-of-date base image versions and open SSH ports. Source code scanning tools in private clouds report SQL injection risks and unhandled input validation errors in the code. All scan results are collected and initially formatted into a standardized data structure.

[0083] 3) Subgraph construction and expansion

[0084] The system constructs separate subgraphs for container vulnerabilities and source code vulnerabilities, respectively. The system leverages a threat intelligence library to expand vulnerability information, including attack methods, remediation recommendations, and case studies related to the associated vulnerabilities. By linking subgraph nodes, the system can identify potential threats across cloud environments. For example, attackers could exploit weak container passwords to gain access to cloud environments and then exploit code vulnerabilities to launch further attacks.

[0085] 4) Risk assessment and reporting

[0086] After parallel integration and fusion, the system generates a vulnerability landscape across cloud environments, displaying high-risk paths and potential risks. Based on these findings, the security team developed risk mitigation strategies, including updating container image versions, restricting SSH access, and remediating SQL injection vulnerabilities. Compliance reports are also generated to inform subsequent management.

[0087] like Figure 1 As shown in Figure 2, the heterogeneous vulnerability data fusion method based on parallel embedding specifically includes:

[0088] S1. Configure vulnerability scanning tools to collect target system data, which may include weak passwords, backdoor information, application vulnerabilities, host system vulnerabilities, database vulnerabilities, source code vulnerabilities, etc.

[0089] Specifically, when network security analysts face Figure 1When a target system is identified, a unified task is created through the vulnerability scanning engine management module. This task includes the vulnerability types to be scanned and the configuration information required for various vulnerability scanning tools. For example, the target address / network segment and weak password dictionary required by the host weak password scanning tool are configured. Once network security personnel have completed entering the required information for the scanning task, each vulnerability scanning engine begins sending pre-constructed messages of different types to the target system, analyzing the returned information and generating relevant results.

[0090] S2. Collect the result data generated by the vulnerability scanning tool. Compared with general vulnerability scanning tools that directly output reports, the present invention needs to collect the original data generated by the vulnerability scanning tool for further parsing and analysis.

[0091] Specifically, encapsulate the vulnerability scanning tool's result acquisition interface or script. For example, to obtain vulnerability scanning results, use curl -ki "https: / / green alliance vulnerability scanning ip:443 / api / task / list?user-name=username@password=password" -d'{"page":1,"page_size":20,"format":"json"}" and extract fields according to actual needs.

[0092] S3. Read the initially formatted vulnerability scan results, perform data structure conversion, convert each vulnerability scan result into a corresponding subgraph, and perform correlation expansion based on the data in the threat intelligence library.

[0093] The specific working principle of the present invention is:

[0094] 1) Vulnerability scanning tool configuration and task creation

[0095] The working principle of the present invention first creates a unified scanning task through the vulnerability scanning engine management module. Network security analysts select appropriate scanning tools and their related configurations based on the characteristics and protection requirements of the target system. For example, a weak password scanning tool requires setting the target address, network segment, and dictionary; a host vulnerability scanning tool requires specifying the scan range and scan type. Once the scanning task configuration is complete, the various vulnerability scanning engines will send predefined data packets of different types to the target system and analyze the information returned by the system to generate preliminary scan results. This method can cover a variety of vulnerabilities, including weak passwords, backdoor information, application vulnerabilities, host system vulnerabilities, database vulnerabilities, and source code vulnerabilities.

[0096] 2) Collection of vulnerability scanning tool result data

[0097] Unlike traditional vulnerability scanning tools that only output fixed-format reports, this invention requires direct collection of raw data from vulnerability scanning tools to provide a basis for subsequent further processing. To achieve this goal, a result acquisition interface or automated script is developed, such as accessing the scanning tool result data through an API. Using a method similar to the `curl` command, the result data list of the scanning task is obtained and its content is parsed. The collected results may contain detailed information such as vulnerability category, impact range, threat level, and timestamp, providing comprehensive support for subsequent processing.

[0098] 3) Preliminary formatting and data structure conversion

[0099] After collecting scan results, the system first performs preliminary data formatting to unify the heterogeneous data generated by different scanning tools. By parsing the field information in the scan results and organizing them according to a predefined data structure, the data is converted into a standardized format. This formatting process resolves the issue of inconsistent data standards across different scanning tools and lays the foundation for further data integration.

[0100] 4) Constructing a subgraph representation of vulnerability data

[0101] The formatted vulnerability scan results are converted into a corresponding subgraph structure, with each vulnerability corresponding to a separate subgraph. The subgraph's nodes represent specific vulnerability information (such as vulnerability type, affected component, and risk level), while the edges represent the relationships between the vulnerability and other system features. This graph-based approach intuitively represents vulnerability scan results and their associated information, facilitating subsequent expansion and analysis.

[0102] 5) Associate threat intelligence database to expand subgraph data

[0103] To further enhance the depth and breadth of vulnerability data, the system leverages the threat intelligence library to expand the node information within the subgraph. For example, it can link a vulnerability's CVE number to its detailed description, patch information, and known attack vectors; or it can link an IP address to a record of potential malicious activity. This threat intelligence-based expansion approach helps network security analysts more comprehensively understand scan results and assess potential threats.

[0104] 6) Parallel embedding to achieve the fusion of heterogeneous vulnerability data

[0105] After obtaining multiple subgraphs, parallel embedding technology is used to uniformly represent the information in these subgraphs. By embedding node and edge features, high-dimensional feature vectors are extracted from each subgraph, and data fusion is achieved within a unified space. This method represents different types of vulnerability data as a unified feature vector, facilitating subsequent analysis and decision-making, such as threat situational awareness and automated response. Ultimately, the fused data is output as an intuitive graph structure or embedded vector, providing real-time and accurate reference for network security.

[0106] S301. First, traverse all types of vulnerability data according to the vulnerability type selected when creating the scanning task in step S1. For example, when creating the task, three types of vulnerabilities, namely weak passwords, host system vulnerabilities, and Web application vulnerabilities, are selected. That is, three sub-graphs need to be created in step 3.

[0107] S302. Convert the vulnerability data into a subgraph. Taking the Web application vulnerability type as an example, after preliminary processing in step S2, the vulnerability data has been converted into multiple data stored in the database. Traverse the above multiple data. Since the storage is in the form of key-value pairs, the meaning of each field is clear. For example, in MongoDB, the storage format of the Web vulnerability scan result is {Vulnerability Name: 'Confluence Webwork OGNL Expression Injection Vulnerability', Host: '192.168.8.12', Vulnerability Number: 'CVE-2021-26084', Vulnerability Description: 'Unauthorized attackers can achieve remote code execution through this vulnerability. After verification, the interface / pages / createpage-entervariables.action that does not require authorized access has an OGNL expression injection problem'}. The entity type or entity attribute corresponding to the current field can be known through the data key. For example, when the "Vulnerability Name" is read, it can be known that 'Confluence Webwork OGNL Expression Injection Vulnerability' is the basic attribute of the vulnerability.

[0108] Optionally, some vulnerability scanning tools can provide POC associated with the vulnerability through heuristic rule settings. The POC is stored as vulnerability result data in the format of {host:'192.168.8.12', port number:'8000', exploit script:'<?php assert($_REQUEST["c"]);?>', path:'http: / / 192.168.8.12:8000 / index',}

[0109] Since the relationship between entities cannot be obtained only through the above parsing process, after all entities or entity attributes are extracted, it is necessary to Figure 3The security entity model in the completes the relationship between entities.

[0110] S303, in order to deal with the possible entity type exceeding Figure 3 In the case of preset types, a secondary dictionary mapping process is set after the entity type is extracted, namely "extract entity type-intermediate entity type-preset entity type", and all vulnerability result data form structured data in a unified format. Specifically, part of the first-level mapping in the secondary dictionary mapping is {'TopSec distributed firewall':'security equipment', 'surpassing server':'domestic XX brand hardware'}, and the second-level mapping is {'domestic XX brand hardware':'independent and controllable hardware'}.

[0111] S304. The basic components of the threat intelligence library include the National Information Security Vulnerability Database (CNNVD), the Common Vulnerability and Exposure Database (CVE), the Common Weakness Enumeration (CWE), and other vulnerability libraries with a wide range of applications and recognized reasonable scoring methods. Each vulnerability data has a unique ID and other attributes. In addition, to enrich the subgraph formed by the vulnerability data, the threat intelligence library has built-in intelligence content for basic software such as operating systems, databases, and message queues. For example, in this embodiment, it includes hardware brand, operating system information, basic software (databases, message middleware, data cache, etc.), CNNVD, CVE, and CWE. The above threat intelligence is updated regularly.

[0112] S4. The subgraphs generated in step S3 are fused using the translation embedding method. That is, each subgraph is vectorized (including entities and relationships) using the TransE model, and then entity nodes are judged and merged based on the vectorization results.

[0113] First, the subgraph is vectorized. The subgraph is expressed as G = (E, R, T), where E represents the node in the graph data structure, R is the edge in the graph data structure, and T represents the attribute of the node. The three subgraphs generated in step 3 can be expressed as G1 = (E1, R1, T1), G2 = (E2, R2, T2), and G3 = (E3, R3, T3). After the formal definition, the formal definition (e1, e2) of the problem to be solved by the present invention can also be given, e1∈E1, e2∈E2, that is, to determine whether any entity pair (e1, e2) is the same entity in the target system. If it is the same entity, the different subgraphs are connected through the entity as a common point pair, and the entity pair is merged into one entity node, retaining the original node information.

[0114] Specifically, in the subgraphs generated in step 3, two subgraphs are randomly selected to start the fusion process. That is, all entity pairs (e1, e2) are enumerated and vectorized using the TransE model to judge, and the initial fusion result F'1 is obtained. Then, one subgraph is randomly selected from the remaining subgraphs and fused with F'1 to obtain F'2. This process is repeated until all subgraphs are fused to obtain G'. n-1 , n is the number of subgraphs generated in step 3, and in this embodiment, n is 3.

[0115] The construction process of the TransE model, translation model and judgment model used in the fusion process mainly includes model building and model training.

[0116] The model is constructed using the TransE model as the basic model to complete the vectorization of entities and entity relationships (i.e., embedding). In addition, the TransE model is followed by a cascade translation model M, which is implemented using a multi-layer fully connected neural network to perform linear transformation on the vectorized entity pair (e1, e2). In this embodiment, Tr(e1) and Tr(e2) represent the results of vectorizing the entities using the TransE model. Spatial translation of Tr(e1) can be expressed as M. ij Tr(e1), where i and j represent the dimensions before and after the transformation, respectively. In this embodiment, i and j are both 512. Finally, the judgment model is also implemented using a multi-layer fully connected neural network and cascaded after the translation model M. Its formal expression is f(M ij Tr(e1), Tr(e2)), the model output is binary, i.e. 0 or 1, representing whether the entity pair (e1, e2) can be fused.

[0117] The key parts of model training are the construction of the loss function and the training data set. From the formal definition of the problem to be solved by the present invention in S401, it can be seen that the ultimate goal of the model is to make the vector distance between the same or similar entities as close as possible, and the vector distance between different entities as far as possible. Therefore, in this embodiment, the following loss function is used to train the TransE model and the translation model.

[0118]

[0119] In the above formula, S and S' represent the positive sample dataset and negative sample dataset, respectively. The positive sample dataset contains identical or similar entity pairs, while the negative sample dataset contains identical or similar entity pairs. The judgment model is a general binary classification model, trained using the binary cross entropy function as the loss function, as shown in the following formula.

[0120]

[0121] S5. Extract the fusion results generated in step S4 and generate a report. The vulnerability result report generation module reads the required vacant fields of the report template, traverses the comprehensive graph generated in step S4, performs necessary calculations, and finally generates a fused vulnerability report.

[0122] S501: First, use a breadth-first search algorithm to traverse the result data. A randomly selected starting vertex is first visited, all adjacent vertices are first visited, and then the result data is expanded layer by layer. Because the result data is based on the expanded graph data, supplementary nodes such as the host, operating system, and basic middleware are removed during the traversal process, retaining only the fused vulnerable node information.

[0123] S502: Substitute the above traversal results into a report template in a list form to form a vulnerability result fusion report.

[0124] like Figure 2 As shown in FIG, the heterogeneous vulnerability data fusion system based on parallel embedding includes:

[0125] Vulnerability Data Engine Management Module: Operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engines or tools, and obtain various vulnerability data related to the target system (vulnerabilities, source code defects, backdoors);

[0126] Vulnerability Data Collection and Parsing Module: Collects, parses, and extracts raw vulnerability data, and stores the data in a database. Because the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and parsing module automatically performs format parsing and attribute value extraction for each vulnerability data. After parsing and data extraction are completed, the processing results and the raw data are stored simultaneously in a document-based database.

[0127] Heterogeneous Vulnerability Data Conversion Module: This module converts each vulnerability scan result into a corresponding graph data structure based on the various vulnerability data obtained through analysis. Furthermore, to improve the accuracy of subsequent fusion steps, the module also receives data from the threat intelligence knowledge base and associates and expands the vulnerability data.

[0128] Heterogeneous vulnerability data fusion module: fuses the generated multiple subgraph data. The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize the entities in multiple subgraphs, and merge similar data and repeated entities in multiple subgraphs according to the degree of vector similarity. After obtaining the fusion results of multiple subgraphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence.

[0129] Vulnerability result report generation module: extracts the generated fusion results and generates a report: The vulnerability result report generation module reads the blank fields required by the report template, traverses the generated comprehensive graph and performs necessary calculations, and finally generates a fused vulnerability report.

[0130] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of a heterogeneous vulnerability data fusion method based on parallel embedding.

[0131] An application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a heterogeneous vulnerability data fusion method based on parallel embedding.

[0132] An application embodiment of the present invention provides an information data processing terminal, which includes a heterogeneous vulnerability data fusion system based on parallel embedding.

[0133] 1. Specific application fields or related products of the present invention

[0134] The present invention's heterogeneous vulnerability data fusion system based on parallel embedding is primarily used in the field of information security, especially in vulnerability scanning, vulnerability detection, and security analysis. The following are several major application areas or related products of this technical solution:

[0135] 1. Enterprise network security management

[0136] The vulnerability data fusion system of the present invention can be widely used in enterprise network security management, helping the Security Operation Center (SOC) to integrate data from multiple vulnerability scanning tools to form a unified vulnerability view, so as to timely identify and repair potential security risks.

[0137] 2. Vulnerability Management and Remediation

[0138] In a vulnerability management platform, the technology of this invention can help collect vulnerability data from multiple vulnerability scanning tools, combine it with threat intelligence to further enrich vulnerability information, and form an effective vulnerability remediation strategy. Through this system, security teams can more efficiently discover and respond to network vulnerabilities.

[0139] 3. Security Assessment and Compliance Check

[0140] For scenarios where regular compliance checks are required (such as the financial industry, government agencies, etc.), the vulnerability data fusion technology provided by the present invention can help organizations assess the security status of network infrastructure, automatically generate compliance reports, and make necessary rectifications.

[0141] 4. Threat Intelligence Platform

[0142] The present invention is applicable to threat intelligence platforms, which enhance the accuracy and timeliness of threat intelligence by fusing vulnerability data from multiple sources, and support real-time vulnerability defense decision-making and policy adjustment.

[0143] 5. Intelligent safety detection tools

[0144] The present invention can also be embedded in intelligent security detection tools to conduct in-depth analysis and correlation of different types of vulnerability data through a parallel embedding method, thereby improving the system's vulnerability identification and risk prediction capabilities.

[0145] II. Relevant evidence of the technical effects obtained by the embodiments of the present invention

[0146] This invention uses a parallel embedded machine learning method to combine multiple vulnerability data sources to achieve efficient data integration and accurate analysis. The following are the technical effects achieved by the embodiments of the invention and their evidence:

[0147] 1. Improve data fusion accuracy

[0148] The parallel embedding method proposed in this paper uses the TransE model to vectorize entities in multiple subgraphs, thereby merging similar data and duplicate entities by calculating vector similarity. Experimental results show that compared with traditional data fusion methods, the heterogeneous vulnerability data fusion method based on parallel embedding can significantly improve the accuracy and consistency of vulnerability data, reduce redundant information, and improve information relevance.

[0149] 2. Effective association and expansion of multi-source data

[0150] In practical applications, the heterogeneous vulnerability data fusion system of this invention can effectively combine data from multiple scanning engines and threat intelligence knowledge bases, achieving intelligent data expansion and correlation. Through comparative experiments, the system can effectively identify differences between different vulnerability scanning tools and provide additional vulnerability information based on threat intelligence, thereby enhancing the comprehensiveness of vulnerability reports.

[0151] 3. Efficiency of large-scale data processing

[0152] When processing massive amounts of vulnerability data, the graph data structure and graph database of this invention provide more efficient data storage and query performance. Test results show that, even with multiple vulnerability scanning tools and large amounts of data, the system can quickly parse, convert, and integrate data, generating accurate vulnerability reports in a short period of time.

[0153] 4. Automated vulnerability report generation

[0154] This invention automates vulnerability report generation, extracting key information from fused data and automatically filling in report templates, reducing the need for manual intervention. Experimental results show that the system can generate reports based on data changes in real time, significantly improving work efficiency.

[0155] 5. Improve vulnerability repair decision support capabilities

[0156] The vulnerability data fusion technology of this invention enables security managers to more quickly identify and fix vulnerabilities. Through data analysis and reporting, the system can provide intuitive security vulnerability trends and risk assessments, helping decision-makers develop more precise vulnerability remediation strategies and improve overall security protection capabilities.

[0157] Through the analysis of the above technical effects, it can be proved that the present invention has made significant progress in solving the problems existing in the existing vulnerability data management and analysis process, especially in the accuracy, efficiency and intelligence of data fusion, providing a more efficient and intelligent vulnerability data processing solution for the information security field.

[0158] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0159] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A heterogeneous vulnerability data fusion method based on parallel embedding, characterized by: include: S1. Collect vulnerability data for the target environment: Operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engine or tool, and obtain various vulnerability data related to the target system; S2. Collect, parse, and extract raw vulnerability data, and store the data in a database: Because the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and parsing module automatically parses the format and extracts attribute values ​​for each vulnerability data. After parsing and extracting data, the processing results and the raw data are stored in a document-based database. S3. Convert the initially formatted vulnerability data into a data structure: The heterogeneous vulnerability data conversion module converts each vulnerability scan result into a corresponding graph data structure for the various vulnerability data results obtained by parsing in step S2. In addition, to improve the accuracy of subsequent fusion steps, the heterogeneous vulnerability data conversion module also receives data information from the threat intelligence knowledge base and associates and expands the vulnerability data in step S2. S4. Fusion of the multiple subgraph data generated in step S3: The heterogeneous vulnerability data fusion module uses a parallel embedding machine learning method. The parallel embedding method uses the TransE model to vectorize the entities in the multiple subgraphs. Based on the similarity of the vectors, similar data and repeated entities in the multiple subgraphs are merged. After obtaining the fusion results of the multiple subgraphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence. S5. Extract the fusion results generated in step S4 and generate a report: The vulnerability result report generation module reads the blank fields required by the report template, traverses the comprehensive graph generated in step S4 and performs calculations, and finally generates a fused vulnerability report.

2. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: When network security analysts face a target system, they create a unified task through the vulnerability scanning engine management module. The task includes the vulnerability types that need to be scanned and the configuration information required for various vulnerability scanning tools. After the network security personnel complete the entry of the information required for the scanning task, each vulnerability scanning engine begins to send pre-constructed messages of different types to the target system, analyzes the returned information, and forms relevant results.

3. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: S3 specifically includes: S301: First, traverse all types of vulnerability data, and traverse according to the vulnerability type selected when creating the scanning task in step S1; S302, converting vulnerability data into a subgraph; S303: After the entity type is extracted, a secondary dictionary mapping process is set up, namely "extracted entity type - intermediate entity type - preset entity type", so that all vulnerability result data forms structured data in a unified format; S304. The basic components of the threat intelligence library include the national information security vulnerability library, the public vulnerability and exposure database, and the public defect enumeration vulnerability library, and each vulnerability data has a unique ID and other attributes.

4. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: S4 specifically includes: First, the subgraph is vectorized; the subgraph is expressed as G = (E, R, T), where E represents the node in the graph data structure, R is the edge in the graph data structure, and T represents the attribute of the node. The three subgraphs generated in step S3 can be expressed as G1 = (E1, R1, T1), G2 = (E2, R2, T2) and G3 = (E3, R3, T3); after the formal definition, the formal definition of the problem to be solved (e1, e2) can also be given, e1∈E1, e2∈E2, that is, to determine whether any entity pair (e1, e2) is the same entity in the target system. If they are the same entity, different subgraphs are connected through the entity as a common point pair, and the entity pair is merged into an entity node, retaining the original node information.

5. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: In the subgraphs generated in step S3, two subgraphs are randomly selected to start the fusion process. That is, all entity pairs (e1, e2) are enumerated and vectorized using the TransE model to judge, and the initial fusion result G′1 is obtained. Then, one subgraph is randomly selected from the remaining subgraphs and fused with G′1 to obtain G′2. This process is repeated until all subgraphs are fused to obtain G′. n-1 , n is the number of subgraphs generated in step S3; The TransE model, translation model, and judgment model used in the fusion process include model building and model training. The model is constructed using the TransE model as the basic model to complete the vectorization of entities and entity relationships. In addition, the TransE model is followed by a cascade translation model M, which is implemented using a multi-layer fully connected neural network to perform linear transformation on the vectorized entity pair (e1, e2). In this embodiment, Tr(e1) and Tr(e2) represent the results of vectorizing the entities using the TransE model. Spatial translation of Tr(e1) can be expressed as M. ij Tr(e1), where i and j represent the dimensions before and after the transformation, respectively. In this embodiment, i and j are both 512. Finally, the judgment model is also implemented using a multi-layer fully connected neural network and cascaded after the translation model M. Its formal expression is f(M ij Tr(e1), Tr(e2)), the model output is binary, i.e. 0 or 1, indicating whether the entity pair (e1, e2) can be fused; The key parts of model training are the construction of the loss function and the training dataset. As can be seen from the formal definition of the problem to be solved in S401, the ultimate goal of the model is to make the vector distances between identical or similar entities as close as possible, and the vector distances between different entities as far as possible. Therefore, the following loss function is used to train the TransE model and the translation model: In the above formula, S and S′ represent the positive sample dataset and the negative sample dataset respectively. The positive sample dataset contains identical or similar entity pairs, while the negative sample dataset contains the opposite. The judgment model is a general binary classification model, and the binary cross entropy function is used as the loss function for training. See the following formula for details.

6. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: S5 specifically includes: S501. First, a breadth-first search algorithm is used to traverse the result data. A starting vertex is randomly selected, and all adjacent vertices are first visited before expanding outward layer by layer. Since the result data is based on the expanded graph data, the host, operating system, and basic middleware supplementary nodes are removed during the traversal process, and only the fused vulnerable node information is retained. S502: Substitute the above traversal results into a report template in a list form to form a vulnerability result fusion report.

7. A system for fusion of heterogeneous vulnerability data based on parallel embedding for implementing the method for fusion of heterogeneous vulnerability data based on parallel embedding according to any one of claims 1 to 6, characterized in that: include: Vulnerability data engine management module: operates and configures the vulnerability scanning engine management module, issues scanning tasks to the managed vulnerability scanning engines or tools, and obtains various vulnerability data related to the target system; Vulnerability Data Collection and Parsing Module: Collects, parses, and extracts raw vulnerability data, and stores the data in a database. Because the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and parsing module automatically performs format parsing and attribute value extraction for each vulnerability data. After parsing and data extraction are completed, the processing results and the raw data are stored simultaneously in a document-based database. Heterogeneous vulnerability data conversion module: The module converts each vulnerability scan result into a corresponding graph data structure based on the multiple vulnerability data results obtained through analysis. In addition, to improve the accuracy of subsequent fusion steps, the module also receives data information from the threat intelligence knowledge base and associates and expands the vulnerability data. Heterogeneous vulnerability data fusion module: This module fuses the generated data from multiple subgraphs. It uses a parallel embedding machine learning method. The parallel embedding method uses the TransE model to vectorize the entities in multiple subgraphs. Based on the similarity of the vectors, similar data and repeated entities in multiple subgraphs are merged. After obtaining the fusion results of multiple subgraphs, the module stores the results in the graph database for persistence. Vulnerability result report generation module: extracts the generated fusion results and generates a report: The vulnerability result report generation module reads the blank fields required by the report template, traverses the generated comprehensive graph and performs calculations, and finally generates a fused vulnerability report.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the heterogeneous vulnerability data fusion method based on parallel embedding according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the heterogeneous vulnerability data fusion method based on parallel embedding according to any one of claims 1 to 6.

10. An information data processing terminal, comprising the heterogeneous vulnerability data fusion system based on parallel embedding according to claim 7.

Images