Heterogeneous vulnerability data fusion method and system based on parallel embedding

Abstract

The invention belongs to but is not limited to the technical field of information security, and particularly relates to a heterogeneous vulnerability data fusion method and system based on parallel embedding, and the method comprises the steps: S1, collecting vulnerability data for a target environment: operating and configuring a vulnerability scanning engine management module, issuing a scanning task to a managed vulnerability scanning engine or tool, and storing the vulnerability data in a database; obtaining various vulnerability data related to the target system; s2, collecting, analyzing and extracting original vulnerability data, and storing the data into a database; s3, performing data structure conversion on the preliminarily formatted vulnerability data, wherein a heterogeneous vulnerability data conversion module converts various vulnerability data results obtained through analysis in the step S2 and each vulnerability scanning result into a corresponding graph data structure; s4, performing 0-line fusion on the multiple sub-graph data generated in the step S3; and S5, extracting a fusion result generated in the step S4, and generating a report.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of information security technology, and in particular relates to a method and system for fusion of heterogeneous vulnerability data based on parallel embedding. Background Art

[0002] In actual use, the results of vulnerability tools (Web vulnerabilities, etc.) from different manufacturers and types are often inconsistent, and the content of the scan results needs to be analyzed manually. The above work is trivial and time-consuming for summarizing and organizing the data after the test. At present, the main methods for automating the fusion of vulnerability scan results are static matching mapping methods or relying on national standards for format specification. Among them, relying on CVE vulnerability numbers for association has the problem of poor universality and cannot comprehensively and reliably fuse a large amount of heterogeneous vulnerability data.

[0003] In view of the above analysis, the technical problems that need to be solved urgently in the prior art are:

[0004] Existing vulnerability detection tools work independently and output data in heterogeneous formats, making it difficult to conduct integrated analysis on the same platform. This causes network security teams to spend a long time and be inefficient when analyzing multi-source vulnerability data.

[0005] Traditional vulnerability scanning methods are often limited to single-dimensional vulnerability detection and fail to fully explore the potential correlations between different vulnerabilities.

[0006] In the existing technology, security analysts need to spend a lot of time manually organizing and analyzing reports generated by different tools, making it difficult to respond to rapidly changing network security threats in real time.

[0007] Traditional methods rely on static reports and manual analysis, making it difficult to respond to new threats in dynamic network environments in real time. Summary of the invention

[0008] In view of the problems existing in the prior art, the present invention provides a method and system for fusion of heterogeneous vulnerability data based on parallel embedding, which mainly provides a solution and system for the management of multiple vulnerability scanning tools and fusion of heterogeneous result data in network security experiments.

[0009] The present invention is implemented as follows: a heterogeneous vulnerability data fusion method based on parallel embedding, comprising:

[0010] S1. Collect vulnerability data for the target environment: operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engine or tool, and obtain various vulnerability data related to the target system (vulnerabilities, source code defects, backdoors);

[0011] S2. Collect, parse and extract the original vulnerability data, and store the data in the database: Since the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and parsing module automatically performs format parsing and attribute value extraction for each vulnerability data; after completing the parsing and data extraction, the processing results and the original data are stored in the document database at the same time;

[0012] S3. Convert the data structure of the initially formatted vulnerability data: The heterogeneous vulnerability data conversion module converts each vulnerability scan result into a corresponding graph data structure for the multiple vulnerability data results obtained by parsing in step S2. In addition, in order to improve the accuracy of the subsequent fusion steps, the heterogeneous vulnerability data conversion module also receives data information from the threat intelligence knowledge base and associates and expands the vulnerability data in step S2;

[0013] S4. Fusion of multiple subgraph data generated in step S3: The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize entities in multiple subgraphs, merge similar data and repeated entities in multiple subgraphs according to the similarity of the vectors, and after obtaining the fusion results of multiple subgraphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence.

[0014] S5. Extract the fusion result generated in step S4 and generate a report: The vulnerability result report generation module reads the missing fields required by the report template, traverses the comprehensive graph generated in step S4 and performs necessary calculations, and finally generates a fused vulnerability report.

[0015] Furthermore, when network security analysts face a target system, they create a unified task through the vulnerability scanning engine management module, which includes the vulnerability types to be scanned and the configuration information required for various vulnerability scanning tools. After network security personnel complete the input of the information required for the scanning task, each vulnerability scanning engine begins to send pre-constructed messages of different types to the target system, analyzes the returned information, and forms relevant results.

[0016] Furthermore, S3 specifically includes:

[0017] S301, first traverse various vulnerability data, and traverse according to the vulnerability type selected when creating the scanning task in step S1;

[0018] S302, converting vulnerability data into a subgraph;

[0019] S303, after the entity type is extracted, a secondary dictionary mapping process is set, namely, "extracted entity type - intermediate entity type - preset entity type", so that all vulnerability result data form structured data in a unified format;

[0020] S304. The basic components of the threat intelligence library include the National Information Security Vulnerability Database (CNNVD), the Common Vulnerability and Exposure Database (CVE), the Common Weakness Enumeration (CWE) and other vulnerability libraries with wide application range and recognized reasonable scoring methods, and each vulnerability data has a unique ID and other attributes.

[0021] Furthermore, S4 specifically includes:

[0022] First, the subgraph is vectorized. The subgraph is expressed as G = (E, R, T), where R represents the node in the graph data structure, R represents the edge in the graph data structure, and T represents the attribute of the node. The three subgraphs generated in step 3 can be expressed as G1 = (E1, R1, T1), G2 = (E2, R2, T2) and G3 = (E3, R3, T3). After the formal definition, the formal definition (e1, e2) of the problem to be solved by the present invention can also be given, e1∈E1, e2∈E2, that is, to determine whether any entity pair (e1, e2) is the same entity in the target system. If it is the same entity, the different subgraphs are connected through the entity as a common point pair, and the entity pair is merged into an entity node, retaining the original node information.

[0023] Furthermore, in the subgraphs generated in step 3, two subgraphs are randomly selected to start the fusion process, that is, all entity pairs (e1, e2) are enumerated and vectorized using the TransE model to obtain the initial fusion result G'1, and then one subgraph is randomly selected from the remaining subgraphs to be fused with G'1 to obtain G'2, and so on, until all subgraphs are fused to obtain G' n-1 , n is the number of subgraphs generated in step 3.

[0024] Furthermore, the construction process of the TransE model, translation model and judgment model used in the fusion process mainly includes model building and model training.

[0025] The model building uses the TransE model as the basic model to complete the vectorization of entities and entity relationships (i.e., Embedding). In addition, the TransE model is cascaded with a translation model M, and the translation model is implemented using a multi-layer fully connected neural network to achieve linear transformation of the vectorized entity pair (e1, e2). In this embodiment, Tr(e1) and Tr(e2) represent the results of vectorizing the entities using the TransE model, and the spatial translation of Tr(e1) can be expressed as M. ijTr(e1), where i and j represent the dimensions before and after the transformation, respectively. In this embodiment, i and j are both 512. Finally, the judgment model is also implemented using a multi-layer fully connected neural network and cascaded after the translation model M. Its formal expression is f(M ij Tr(e1), Tr(e2)), the model output is binary, i.e. 0 or 1, representing whether the entity pair (e1, e2) can be fused.

[0026] The key parts of model training are the construction of loss functions and training data sets. From the formal definition of the problem to be solved by the present invention in S401, it can be seen that the ultimate goal of the model is to make the vector distance between the same or similar entities as close as possible, and the vector distance between different entities as far as possible. Therefore, in this embodiment, the following loss function is used to train the TransE model and the translation model.

[0027]

[0028] In the above formula, S and S' represent the positive sample data set and the negative sample data set respectively. The positive sample data set contains the same or similar entity pairs, while the negative sample data set contains the opposite. The judgment model is a general binary classification model, and the binary cross entropy function can be used as the loss function for training. See the following formula for details.

[0029]

[0030] Furthermore, S5 specifically includes:

[0031] S501. First, a breadth-first search algorithm is used to traverse the result data, a starting vertex is randomly selected, all adjacent vertices are visited first, and then the result data is expanded outward layer by layer. Since the result data is based on the expanded graph data, the supplementary nodes such as the host, operating system, and basic middleware are removed during the traversal process, and only the fused vulnerable node information is retained.

[0032] S502: Substitute the above traversal results into a report template in a list form to form a vulnerability result fusion report.

[0033] Another object of the present invention is to provide a heterogeneous vulnerability data fusion system based on parallel embedding for implementing the heterogeneous vulnerability data fusion method based on parallel embedding, comprising:

[0034] Vulnerability data engine management module: operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engine or tool, and obtain various vulnerability data related to the target system (vulnerabilities, source code defects, backdoors);

[0035] Vulnerability data collection and analysis module: collects, analyzes and extracts the original vulnerability data, and stores the data in the database; since the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and analysis module automatically performs format analysis and attribute value extraction for each vulnerability data; after completing the analysis and data extraction, the processing results and the original data are stored in the document database at the same time;

[0036] Heterogeneous vulnerability data conversion module: The heterogeneous vulnerability data conversion module converts each vulnerability scan result into a corresponding graph data structure for the multiple vulnerability data results obtained through parsing. In addition, in order to improve the accuracy of the subsequent fusion steps, the heterogeneous vulnerability data conversion module also receives data information from the threat intelligence knowledge base and associates and expands the vulnerability data;

[0037] Heterogeneous vulnerability data fusion module: fuses the generated multiple sub-graph data. The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize the entities in multiple sub-graphs, and merge similar data and repeated entities in multiple sub-graphs according to the similarity of the vectors. After obtaining the fusion results of multiple sub-graphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence.

[0038] Vulnerability result report generation module: extracts the generated fusion results and generates a report: The vulnerability result report generation module reads the missing fields required by the report template, traverses the generated comprehensive graph and performs necessary calculations, and finally generates a fused vulnerability report.

[0039] Another object of the present invention is to provide a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the heterogeneous vulnerability data fusion method based on parallel embedding.

[0040] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the heterogeneous vulnerability data fusion method based on parallel embedding.

[0041] Another object of the present invention is to provide an information data processing terminal, which includes the heterogeneous vulnerability data fusion system based on parallel embedding.

[0042] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0043] First, the present invention solves the technical problems in existing experimental work, such as low efficiency and poor accuracy of vulnerability data fusion, easy appearance of more duplicate vulnerability information, poor scalability of vulnerability scanning result fusion, and simple association method. The present invention realizes the use of this system to efficiently and accurately complete the unified management of vulnerability scanning, result fusion and scalability.

[0044] Second, technical problem solving:

[0045] 1) Solve the problem of difficulty in integrating heterogeneous data from multiple tools

[0046] Existing vulnerability detection tools work independently and output data in heterogeneous formats, making it difficult to perform fusion analysis on the same platform, resulting in a long time and low efficiency for network security teams to analyze multi-source vulnerability data. The present invention standardizes the raw data generated by different scanning tools through a unified result collection interface and formatting processing method, solves the problem of incompatibility of heterogeneous data, and lays the foundation for subsequent data fusion and threat analysis.

[0047] 2) Improve the relevance and depth of vulnerability detection results

[0048] Traditional vulnerability scanning methods are often limited to single-dimensional vulnerability detection and fail to fully explore the potential correlation between different vulnerabilities. The present invention, by constructing a subgraph structure of vulnerability data and using the threat intelligence library for correlation expansion, can reveal the mutual relationship and possible attack paths of different vulnerabilities, thereby improving the depth of vulnerability detection and the scientific nature of decision-making.

[0049] 3) Reduce the workload of manual analysis and decision-making

[0050] In the prior art, security analysts need to spend a lot of time manually collating and analyzing reports generated by different tools, making it difficult to respond to rapidly changing network security threats in real time. The present invention uses parallel embedding technology to uniformly represent heterogeneous vulnerability data as high-dimensional feature vectors and automatically generate vulnerability situation maps, which greatly reduces the workload of manual data processing and realizes rapid vulnerability assessment and decision support.

[0051] 4) Realize real-time and high efficiency

[0052] Traditional methods rely on static reports and manual analysis, making it difficult to respond to new threats in a dynamic network environment in real time. The present invention combines real-time monitoring, data fusion and situational awareness technology to achieve rapid assessment and dynamic management of target system vulnerabilities, significantly improving the efficiency and accuracy of vulnerability detection.

[0053] Technological advancements:

[0054] 1) Significant improvement in data fusion capabilities

[0055] The present invention solves the problem of integrating heterogeneous data from multiple tools and proposes a unified vulnerability representation method based on parallel embedding, which can analyze vulnerability information from multiple dimensions and provide a more comprehensive and accurate reference for enterprise security protection.

[0056] 2) Intelligent development of vulnerability detection

[0057] By expanding sub-graph information in the threat intelligence library and automatically generating vulnerability situation maps, the present invention enables vulnerability analysis to shift from passive static detection to active dynamic perception, improves the intelligence level of security analysis, and lays the foundation for predictive security protection.

[0058] 3) Efficient and low-cost network security solutions

[0059] Compared with traditional methods, the present invention significantly reduces the time cost and manpower investment of vulnerability detection by automating the collection, analysis and decision-making processes, while providing real-time and accurate security response capabilities, and has good industrial application value.

[0060] 4) Universal technical solutions applicable to multiple scenarios

[0061] The present invention is not only suitable for vulnerability detection of a single system, but can also be extended to security management of complex network environments, cross-cloud systems and heterogeneous platforms, promoting the widespread application of network security technology in multiple fields such as industry, finance, and government affairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flow chart of a heterogeneous vulnerability data fusion method based on parallel embedding provided by an embodiment of the present invention;

[0063] Figure 2 It is a structural diagram of a heterogeneous vulnerability data fusion system based on parallel embedding provided by an embodiment of the present invention;

[0064] Figure 3 It is a schematic diagram of a preset security entity model used in a heterogeneous vulnerability data fusion method based on parallel embedding provided in an embodiment of the present invention;

[0065] Figure 4 It is a schematic diagram of the process of data transfer transformation in the heterogeneous vulnerability data fusion method based on parallel embedding provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] Example 1: Enterprise intranet vulnerability detection and management

[0068] In an enterprise intranet environment, network administrators need to perform vulnerability detection on the entire network system to prevent potential threats, including the security of servers, databases, and employee terminal devices.

[0069] 1) Configure the scanning task

[0070] The network administrator creates a scanning task through the vulnerability scanning engine management module, selects weak password scanning, host vulnerability scanning and application vulnerability scanning tools, and configures scanning parameters, such as target IP address range, weak password dictionary, vulnerability type, etc. After the task is started, the scanning tool sends detection messages to collect security data of servers and terminal devices.

[0071] 2) Collect scan data

[0072] The raw result data generated by the scanning tool is automatically collected through the API interface. For example, the information returned by the database scanning tool includes unencrypted sensitive forms, outdated SQL engine versions, etc. All raw result data is imported into the system and initially formatted in preparation for subsequent analysis.

[0073] 3) Data fusion and analysis

[0074] The system constructs multiple sub-graphs for the scan results, for example, one sub-graph is used to represent host vulnerability information, and another sub-graph is used to represent database vulnerability information. Subsequently, the sub-graph data is expanded through the threat intelligence library, such as associating known CVE numbers and patch information. Finally, all sub-graph information is integrated using parallel embedding technology to generate a unified vulnerability situation map, helping administrators quickly understand the security status of the intranet.

[0075] 4) Decision support

[0076] The system gives priority repair suggestions based on the integrated vulnerability data, such as immediately updating the database engine or modifying weak password configurations for high-risk vulnerabilities. It also provides a report export function to support companies in subsequent rectification and compliance reviews.

[0077] Example 2: Vulnerability assessment in a cross-cloud environment

[0078] A large Internet company uses a multi-cloud architecture, including public and private clouds. The security team needs to conduct security assessments on containerized applications in cross-cloud environments to ensure the reliability and security of cloud services.

[0079] 1) Configure scanning tools and policies

[0080] The security team uses the method of the present invention to create scanning tasks, including container weak password scanning in public cloud services and source code vulnerability scanning tools in private clouds. Configure the cloud service provider's API access rights and container target list, and set the path and scanning rules of the source code scanning tool. After the task is started, the tool automatically detects security vulnerabilities in the cross-cloud environment.

[0081] 2) Scanning results collection

[0082] Scanning tools in the public cloud return container vulnerability information through API interfaces, such as unupdated base image versions, open SSH ports, etc.; private cloud source code scanning tools return SQL injection risk points and unhandled input validation errors in the code. All scan results are collected by the system and initially formatted into a standardized data structure.

[0083] 3) Subgraph construction and expansion

[0084] The system builds independent subgraphs for container vulnerabilities and source code vulnerabilities, and uses the threat intelligence library to expand vulnerability information, such as the attack methods, repair suggestions, and related cases of related vulnerabilities. Through the correlation of subgraph nodes, potential threats across cloud environments are discovered. For example, attackers can enter the cloud environment through weak container passwords and then use code vulnerabilities to carry out further attacks.

[0085] 4) Risk assessment and reporting

[0086] After parallel embedding and fusion, the system generates a vulnerability situation map across cloud environments, showing high-risk paths and potential risks. Based on the results, the security team developed risk mitigation strategies, including updating container image versions, restricting SSH access rights, and fixing SQL injection vulnerabilities. At the same time, compliance reports were generated to provide a basis for subsequent management.

[0087] like Figure 1 As shown in FIG. 1 , the heterogeneous vulnerability data fusion method based on parallel embedding specifically includes:

[0088] S1. Configure vulnerability scanning tools to collect target system data, which may include weak passwords, backdoor information, application vulnerabilities, host system vulnerabilities, database vulnerabilities, source code vulnerabilities, etc.

[0089] Specifically, when network security analysts face Figure 1When the target system in the network is scanned, a unified task is created through the vulnerability scanning engine management module. The task contains the vulnerability types to be scanned and the configuration information required by various vulnerability scanning tools. For example, the target address / network segment and weak password dictionary required for configuring the host weak password scanning tool. When the network security personnel complete the input of the information required for the scanning task, each vulnerability scanning engine starts to send pre-constructed different types of messages to the target system, analyze and form relevant results based on the returned information.

[0090] S2. Collect the result data generated by the vulnerability scanning tool. Compared with the general vulnerability scanning tool that directly outputs a report, the present invention needs to collect the original data generated by the vulnerability scanning tool for further parsing and analysis.

[0091] Specifically, encapsulate the result acquisition interface or script of the vulnerability scanning tool. For example, the vulnerability scanning result is obtained as curl-ki "https: / / green alliance vulnerability scanning ip:443 / api / task / list?user-name=username@password=password"-d'{"page":1,"page_size":20,"format":"json"}", and extract fields according to actual needs.

[0092] S3. Read the initially formatted vulnerability scan results, perform data structure conversion, convert each vulnerability scan result into a corresponding subgraph, and perform correlation expansion based on the data in the threat intelligence library.

[0093] The specific working principle of the present invention is:

[0094] 1) Configuration and task creation of vulnerability scanning tools

[0095] The working principle of the present invention is to first create a unified scanning task through the vulnerability scanning engine management module. Network security analysts select appropriate scanning tools and their related configurations according to the characteristics and protection requirements of the target system. For example, the weak password scanning tool needs to set the target address, network segment and dictionary; the host vulnerability scanning tool needs to specify the scanning range and scanning type, etc. When the scanning task configuration is completed, various vulnerability scanning engines will send predefined data packets of different types to the target system, and form preliminary scanning results by analyzing the information returned by the system. This method can cover multiple vulnerabilities such as weak passwords, backdoor information, application vulnerabilities, host system vulnerabilities, database vulnerabilities and source code vulnerabilities.

[0096] 2) Collection of vulnerability scanning tool result data

[0097] Unlike traditional vulnerability scanning tools that only output reports in a fixed format, the present invention requires the direct collection of raw data from vulnerability scanning tools to provide a basis for subsequent further processing. To achieve this goal, a result acquisition interface or automated script is developed, such as accessing the scanning tool result data through an API. The result data list of the scanning task is obtained in a manner similar to the `curl` command, and its content is parsed. The collected results may contain detailed information such as vulnerability category, impact range, threat level, and timestamp, providing comprehensive support for subsequent processing.

[0098] 3) Preliminary formatting and data structure conversion

[0099] After collecting the scan results, the system first formats the data in order to unify the heterogeneous data generated by different scanning tools. By parsing the field information in the scan results, it is organized according to the predefined data structure and converted into a standardized format. This formatting process can solve the problem of inconsistent data standards between different scanning tools and lay the foundation for the next step of data fusion.

[0100] 4) Constructing a subgraph representation of vulnerability data

[0101] The formatted vulnerability scan results are converted into corresponding subgraph structures, with each vulnerability corresponding to an independent subgraph. The nodes of the subgraph can represent the specific information of the vulnerability (such as vulnerability type, affected components, risk level, etc.), and the edges can represent the relationship between the vulnerability and other system characteristics. This graph structuring method can intuitively represent the vulnerability scan results and their associated information, and facilitate subsequent expansion and analysis.

[0102] 5) Associate threat intelligence database to expand sub-graph data

[0103] To further enhance the depth and breadth of vulnerability data, the system uses the threat intelligence library to expand the node information in the subgraph. For example, by looking up the CVE number of the vulnerability, it can associate its detailed description, patch information, and known attack methods; or by associating its possible malicious behavior records through the IP address. This threat intelligence-based expansion method can help network security analysts understand the scan results more comprehensively and evaluate potential threats.

[0104] 6) Parallel embedding to achieve the fusion of heterogeneous vulnerability data

[0105] After obtaining multiple subgraphs, the parallel embedding technology is used to uniformly represent the information in different subgraphs. By embedding the features of nodes and edges, the high-dimensional feature vectors of each subgraph are extracted, and data fusion is achieved in a unified space. This method can represent different types of vulnerability data as a unified feature vector, which facilitates subsequent analysis and decision-making, such as threat situation awareness and automated response. Finally, the fused data is output in the form of an intuitive graph structure or embedded vector, providing real-time and accurate reference for network security.

[0106] S301. First, various vulnerability data are traversed according to the vulnerability types selected when creating the scanning task in step S1. For example, when creating the task, three types of vulnerabilities, namely weak passwords, host system vulnerabilities, and Web application vulnerabilities, are selected, that is, three sub-graphs need to be created in step 3.

[0107] S302, convert the vulnerability data into a subgraph. Taking the Web application vulnerability type as an example, after the preliminary processing of step S2, the vulnerability data has been converted into multiple data stored in the database. Traverse the above multiple data. In view of the storage in the form of key-value pairs, the meaning of each field is clear. For example, in MongoDB, the storage format of the Web vulnerability scan result is {Vulnerability Name: 'Confluence Webwork OGNL Expression Injection Vulnerability', Host: '192.168.8.12', Vulnerability Number: 'CVE-2021-26084', Vulnerability Description: 'Unauthorized attackers can use this vulnerability to achieve remote code execution. After verification, the interface / pages / createpage-entervariables.action that does not require authorized access has the problem of OGNL expression injection'}. The entity type or entity attribute corresponding to the current field can be known through the data key. For example, when the "Vulnerability Name" is read, it can be known that 'Confluence Webwork OGNL Expression Injection Vulnerability' is the basic attribute of the vulnerability.

[0108] Optionally, some vulnerability scanning tools can provide POC associated with the vulnerability through heuristic rule settings. The POC is stored as vulnerability result data in the format of {host:'192.168.8.12', port number:'8000', use script:'<? php assert($_REQUEST["c"]);? >', path:'http: / / 192.168.8.12:8000 / index',}

[0109] Since the relationship between entities cannot be obtained only through the above parsing process, after all entities or entity attributes are extracted, Figure 3The security entity model in the completes the relationship between entities.

[0110] S303. In order to deal with the possible entity type exceeding Figure 3 In the case of preset types, a secondary dictionary mapping process is set after the entity type is extracted, namely "extract entity type-intermediate entity type-preset entity type", and all vulnerability result data form structured data in a unified format. Specifically, part of the first-level mapping in the secondary dictionary mapping is {'TopSec distributed firewall':'security equipment', 'beyond server':'domestic XX brand hardware'}, and the second-level mapping is {'domestic XX brand hardware':'autonomous controllable hardware'}.

[0111] S304, the basic components of the threat intelligence library include the National Information Security Vulnerability Database (CNNVD), the Common Vulnerability and Exposure Database (CVE), the Common Weakness Enumeration (CWE) and other vulnerability libraries with a wide range of applications and recognized reasonable scoring methods, and each vulnerability data has a unique ID and other attributes. In addition, in order to enrich the subgraph formed by the vulnerability data, the threat intelligence library has built-in intelligence content of basic software such as operating systems, databases, and message queues. For example, in this embodiment, it contains hardware brands, operating system information, basic software (databases, message middleware, data caches, etc.), CNNVD, CVE, and CWE. The above threat intelligence is updated regularly in increments.

[0112] S4. The subgraphs generated in step S3 are fused using the translation embedding method, that is, each subgraph is vectorized (including entities and relationships) using the TransE model, and then entity nodes are judged and merged based on the vectorization results.

[0113] First, the subgraph is vectorized. The subgraph is expressed as G = (E, R, T), where E represents the node in the graph data structure, R is the edge in the graph data structure, and T represents the attribute of the node. The three subgraphs generated in step 3 can be expressed as G1 = (E1, R1, T1), G2 = (E2, R2, T2) and G3 = (E3, R3, T3). After the formal definition, the formal definition (e1, e2) of the problem to be solved by the present invention can also be given, e1∈E1, e2∈E2, that is, to determine whether any entity pair (e1, e2) is the same entity in the target system. If it is the same entity, the different subgraphs are connected through the entity as a common point pair, and the entity pair is merged into an entity node, retaining the original node information.

[0114] Specifically, in the subgraphs generated in step 3, two subgraphs are randomly selected to start the fusion process, that is, all entity pairs (e1, e2) are enumerated and vectorized using the TransE model to obtain the initial fusion result F'1, and then one subgraph is randomly selected from the remaining subgraphs to be fused with F'1 to obtain F'2, and so on, until all subgraphs are fused to obtain G' n-1 , n is the number of subgraphs generated in step 3, and n is 3 in this embodiment.

[0115] The construction process of the TransE model, translation model and judgment model used in the fusion process mainly includes model building and model training.

[0116] The model building uses the TransE model as the basic model to complete the vectorization of entities and entity relationships (i.e., Embedding). In addition, the TransE model is cascaded with a translation model M, and the translation model is implemented using a multi-layer fully connected neural network to achieve linear transformation of the vectorized entity pair (e1, e2). In this embodiment, Tr(e1) and Tr(e2) represent the results of vectorizing the entities using the TransE model, and the spatial translation of Tr(e1) can be expressed as M. ij Tr(e1), where i and j represent the dimensions before and after the transformation, respectively. In this embodiment, i and j are both 512. Finally, the judgment model is also implemented using a multi-layer fully connected neural network and cascaded after the translation model M. Its formal expression is f(M ij Tr(e1), Tr(e2)), the model output is binary, i.e. 0 or 1, representing whether the entity pair (e1, e2) can be fused.

[0117] The key parts of model training are the construction of loss functions and training data sets. From the formal definition of the problem to be solved by the present invention in S401, it can be seen that the ultimate goal of the model is to make the vector distance between the same or similar entities as close as possible, and the vector distance between different entities as far as possible. Therefore, in this embodiment, the following loss function is used to train the TransE model and the translation model.

[0118]

[0119] In the above formula, S and S' represent the positive sample data set and the negative sample data set respectively. The positive sample data set contains the same or similar entity pairs, while the negative sample data set contains the opposite. The judgment model is a general binary classification model, and the binary cross entropy function can be used as the loss function for training. See the following formula for details.

[0120]

[0121] S5, extracting the fusion result generated in step S4 and generating a report. The vulnerability result report generation module reads the missing fields required by the report template, traverses the comprehensive graph generated in step S4 and performs necessary calculations, and finally generates a fused vulnerability report.

[0122] S501. First, use the breadth-first search algorithm to traverse the result data, randomly select the starting vertex, first visit all its adjacent vertices, and then expand outward layer by layer. Since the result data is based on the expanded graph data, the supplementary nodes such as the host, operating system, and basic middleware are removed during the traversal process, and only the fused vulnerable node information is retained;

[0123] S502: Substitute the above traversal results into a report template in a list form to form a vulnerability result fusion report.

[0124] like Figure 2 As shown, the heterogeneous vulnerability data fusion system based on parallel embedding includes:

[0125] Vulnerability data engine management module: operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engine or tool, and obtain various vulnerability data related to the target system (vulnerabilities, source code defects, backdoors);

[0126] Vulnerability data collection and analysis module: collects, analyzes and extracts the original vulnerability data, and stores the data in the database; since the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and analysis module automatically performs format analysis and attribute value extraction for each vulnerability data; after completing the analysis and data extraction, the processing results and the original data are stored in the document database at the same time;

[0127] Heterogeneous vulnerability data conversion module: The heterogeneous vulnerability data conversion module converts each vulnerability scan result into a corresponding graph data structure for the multiple vulnerability data results obtained through parsing. In addition, in order to improve the accuracy of the subsequent fusion steps, the heterogeneous vulnerability data conversion module also receives data information from the threat intelligence knowledge base and associates and expands the vulnerability data;

[0128] Heterogeneous vulnerability data fusion module: fuses the generated multiple sub-graph data. The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize the entities in multiple sub-graphs, and merge similar data and repeated entities in multiple sub-graphs according to the similarity of the vectors. After obtaining the fusion results of multiple sub-graphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence.

[0129] Vulnerability result report generation module: extracts the generated fusion results and generates a report: The vulnerability result report generation module reads the missing fields required by the report template, traverses the generated comprehensive graph and performs necessary calculations, and finally generates a fused vulnerability report.

[0130] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of a heterogeneous vulnerability data fusion method based on parallel embedding.

[0131] The application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a heterogeneous vulnerability data fusion method based on parallel embedding.

[0132] An application embodiment of the present invention provides an information data processing terminal, which includes a heterogeneous vulnerability data fusion system based on parallel embedding.

[0133] 1. Specific application fields or related products of the present invention

[0134] The heterogeneous vulnerability data fusion system based on parallel embedding of the present invention is mainly used in the field of information security, especially vulnerability scanning, vulnerability detection and security analysis. The following are several main application fields or related products of this technical solution:

[0135] 1. Enterprise network security management

[0136] The vulnerability data fusion system of the present invention can be widely used in enterprise network security management, helping the Security Operation Center (SOC) to integrate data from multiple vulnerability scanning tools to form a unified vulnerability view, so as to timely identify and repair potential security risks.

[0137] 2. Vulnerability management and repair

[0138] In the vulnerability management platform, the technology of the present invention can help collect vulnerability data from multiple vulnerability scanning tools, further enrich vulnerability information with threat intelligence, and form an effective vulnerability repair strategy. Through this system, the security team can more efficiently discover and respond to network vulnerabilities.

[0139] 3. Security assessment and compliance check

[0140] For scenarios where regular compliance checks are required (such as the financial industry, government agencies, etc.), the vulnerability data fusion technology provided by the present invention can help organizations evaluate the security status of network infrastructure, automatically generate compliance reports, and make necessary rectifications.

[0141] 4. Threat Intelligence Platform

[0142] The present invention is applicable to a threat intelligence platform, which enhances the accuracy and timeliness of threat intelligence by fusing vulnerability data from multiple sources, and supports real-time vulnerability defense decision-making and strategy adjustment.

[0143] 5. Intelligent safety detection tools

[0144] The present invention can also be embedded in intelligent security detection tools to conduct in-depth analysis and association of different types of vulnerability data through a parallel embedding method, thereby improving the vulnerability identification and risk prediction capabilities of the system.

[0145] II. Evidence of the Technical Effects Obtained by the Embodiments of the Present Invention

[0146] The present invention achieves efficient data fusion and accurate analysis by combining multiple vulnerability data sources through parallel embedded machine learning methods. The following are the technical effects and evidences of the embodiments of the present invention:

[0147] 1. Improve data fusion accuracy

[0148] The parallel embedding method of the present invention uses the TransE model to vectorize entities in multiple subgraphs, thereby merging similar data and repeated entities by calculating vector similarity. Experimental results show that compared with traditional data fusion methods, the heterogeneous vulnerability data fusion method based on parallel embedding can significantly improve the accuracy and consistency of vulnerability data, reduce redundant information and improve the relevance of information.

[0149] 2. Effective association and expansion of multi-source data

[0150] In practical applications, the heterogeneous vulnerability data fusion system of the present invention can effectively combine data from multiple scanning engines and threat intelligence knowledge bases to achieve intelligent expansion and association of data. Through comparative experiments, the system can effectively identify the differences between different vulnerability scanning tools and provide additional vulnerability information based on threat intelligence, thereby enhancing the comprehensiveness of vulnerability reports.

[0151] 3. Efficiency of large-scale data processing

[0152] When processing massive vulnerability data, the graph data structure and graph database of the present invention provide more efficient data storage and query performance. Test results show that in the case of multiple vulnerability scanning tools and large-scale data, the system can quickly complete data parsing, conversion and fusion, and can generate accurate vulnerability reports in a short time.

[0153] 4. Automated vulnerability report generation

[0154] The present invention realizes automation in vulnerability report generation, can extract key information from the fused data, and automatically fill in the report template, reducing the need for manual intervention. Experimental results show that the system can generate reports in real time according to data changes, significantly improving work efficiency.

[0155] 5. Improve vulnerability repair decision support capabilities

[0156] Through the vulnerability data fusion technology of the present invention, security managers can identify and repair vulnerabilities more quickly. Through data analysis and reporting, the system can provide intuitive security vulnerability trends and risk assessments, helping decision makers make more accurate vulnerability repair strategies and improve overall security protection capabilities.

[0157] Through the analysis of the above technical effects, it can be proved that the present invention has made significant progress in solving the problems existing in the existing vulnerability data management and analysis process, especially in the accuracy, efficiency and intelligence of data fusion, providing a more efficient and intelligent vulnerability data processing solution for the information security field.

[0158] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0159] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for fusion of heterogeneous vulnerability data based on parallel embedding, characterized in that: include: S1. Collect vulnerability data for the target environment: operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engine or tool, and obtain various vulnerability data related to the target system; S2. Collect, parse and extract the original vulnerability data, and store the data in the database: Since the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and parsing module automatically performs format parsing and attribute value extraction for each vulnerability data; after completing the parsing and data extraction, the processing results and the original data are stored in the document database at the same time; S3. Convert the data structure of the initially formatted vulnerability data: The heterogeneous vulnerability data conversion module converts each vulnerability scanning result into a corresponding graph data structure for the multiple vulnerability data results obtained by parsing in step S2; in addition, in order to improve the accuracy of the subsequent fusion step, the heterogeneous vulnerability data conversion module simultaneously receives data information in the threat intelligence knowledge base and associates and expands the vulnerability data in step S2; S4. Fusion of multiple subgraph data generated in step S3: The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize entities in multiple subgraphs, merge similar data and repeated entities in multiple subgraphs according to the similarity of the vectors, and after obtaining the fusion results of multiple subgraphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence; S5. Extract the fusion result generated in step S4 and generate a report: The vulnerability result report generation module reads the missing fields required by the report template, traverses the comprehensive graph generated in step S4 and performs necessary calculations, and finally generates a fused vulnerability report.

2. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: When network security analysts face a target system, they create a unified task through the vulnerability scanning engine management module. The task includes the vulnerability types that need to be scanned and the configuration information required for various vulnerability scanning tools. When network security personnel complete the entry of the information required for the scanning task, each vulnerability scanning engine begins to send pre-constructed messages of different types to the target system, analyze the returned information, and form relevant results.

3. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: S3 specifically includes: S301, first traverse various vulnerability data, and traverse according to the vulnerability type selected when creating the scanning task in step S1; S302, converting vulnerability data into a subgraph; S303: After the entity type is extracted, a secondary dictionary mapping process is set, namely, "extracted entity type-intermediate entity type-preset entity type", so that all vulnerability result data form structured data in a unified format; S304. The basic components of the threat intelligence library include the national information security vulnerability library, the public vulnerability and exposure database, and the public defect enumeration vulnerability library, and each vulnerability data has a unique ID and other attributes.

4. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: S4 specifically includes: First, the subgraph is vectorized; the subgraph is expressed as G=(E, R, T), where E represents a node in the graph data structure, R is an edge in the graph data structure, and T represents an attribute of the node. The three subgraphs generated in step S3 can be expressed as G1=(E1, R1, T1), G2=(E2, R2, T2) and G3=(E3, R3, T3); after formal definition, the formal definition (e1, e2) of the problem to be solved by the present invention can also be given, e1∈E1, e2∈E2, that is, to determine whether any entity pair (e1, e2) is the same entity in the target system. When they are the same entity, different subgraphs are connected through the entity as a common point pair, and the entity pair is merged into an entity node, retaining the original node information.

5. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: In the subgraphs generated in step S3, two subgraphs are randomly selected to start the fusion process, that is, all entity pairs (e1, e2) are enumerated and vectorized using the TransE model to obtain the initial fusion result G′1, and then one subgraph is randomly selected from the remaining subgraphs to be fused with G′1 to obtain G′2, and so on, until all subgraphs are fused to obtain G′ n-1 , n is the number of subgraphs generated in step S3; The TransE model, translation model, and judgment model used in the fusion process mainly include model building and model training; The model building uses the TransE model as the basic model to complete the vectorization of entities and entity relationships. In addition, the TransE model is followed by a cascade translation model M, and the translation model is implemented using a multi-layer fully connected neural network to achieve linear transformation of the vectorized entity pair (e1, e2). In this embodiment, Tr(e1) and Tr(e2) represent the results of vectorizing the entities using the TransE model, and the spatial translation of Tr(e1) can be expressed as M ij Tr(e1), where i and j represent the dimensions before and after the transformation, respectively. In this embodiment, i and j are both 512. Finally, the judgment model is also implemented using a multi-layer fully connected neural network and cascaded after the translation model M. Its formal expression is f(M ij Tr(e1), Tr(e2)), the model output is binary, i.e. 0 or 1, indicating whether the entity pair (e1, e2) can be fused; The key parts of model training are the construction of loss function and training data set. From the formal definition of the problem to be solved by the present invention in S401, it can be seen that the ultimate goal of the model is to make the vector distance between the same or similar entities as close as possible and the vector distance between different entities as far as possible. Therefore, the following loss function is used in this embodiment to train the TransE model and the translation model; In the above formula, S and S′ represent the positive sample data set and the negative sample data set respectively. The positive sample data set contains the same or similar entity pairs, while the negative sample data set contains the opposite. The judgment model is a general binary classification model, and the binary cross entropy function is used as the loss function for training. See the following formula for details.

6. The method for fusion of heterogeneous vulnerability data based on parallel embedding according to claim 1, characterized in that: S5 specifically includes: S501. First, a breadth-first search algorithm is used to traverse the result data, a starting vertex is randomly selected, all adjacent vertices are visited first, and then the result data is expanded outward layer by layer. Since the result data is based on the expanded graph data, the supplementary nodes such as the host, operating system, and basic middleware are removed during the traversal process, and only the fused vulnerable node information is retained. S502: Substitute the above traversal results into a report template in a list form to form a vulnerability result fusion report.

7. A system for fusion of heterogeneous vulnerability data based on parallel embedding for implementing the method for fusion of heterogeneous vulnerability data based on parallel embedding as claimed in any one of claims 1 to 6, characterized in that: include: Vulnerability data engine management module: operate and configure the vulnerability scanning engine management module, issue scanning tasks to the managed vulnerability scanning engine or tool, and obtain various vulnerability data related to the target system; Vulnerability data collection and analysis module: collects, analyzes and extracts the original vulnerability data, and stores the data in the database; since the vulnerability collection step involves multiple engines / tools, the vulnerability data collection and analysis module automatically performs format analysis and attribute value extraction for each vulnerability data; after completing the analysis and data extraction, the processing results and the original data are stored in the document database at the same time; Heterogeneous vulnerability data conversion module: The heterogeneous vulnerability data conversion module converts each vulnerability scan result into a corresponding graph data structure for the multiple vulnerability data results obtained through parsing. In addition, in order to improve the accuracy of the subsequent fusion steps, the heterogeneous vulnerability data conversion module also receives data information from the threat intelligence knowledge base and associates and expands the vulnerability data. Heterogeneous vulnerability data fusion module: fuses the generated multiple subgraph data. The heterogeneous vulnerability data fusion module adopts a parallel embedding machine learning method. The parallel embedding method in the present invention is to use the TransE model to vectorize the entities in multiple subgraphs, merge similar data and repeated entities in multiple subgraphs according to the similarity of the vectors, and after obtaining the fusion results of multiple subgraphs, the heterogeneous vulnerability data fusion module stores the results in the graph database for persistence; Vulnerability result report generation module: extracts the generated fusion results and generates a report: The vulnerability result report generation module reads the missing fields required by the report template, traverses the generated comprehensive graph and performs necessary calculations, and finally generates a fused vulnerability report.

8. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the heterogeneous vulnerability data fusion method based on parallel embedding as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the heterogeneous vulnerability data fusion method based on parallel embedding according to any one of claims 1 to 6.

10. An information data processing terminal, comprising the heterogeneous vulnerability data fusion system based on parallel embedding according to claim 7.

Images