Formal description method and device for big data platform risk

Abstract

The invention discloses a formalized description method and device for big data platform risks, and the method comprises the steps: analyzing an audit log in each standard security domain of a target big data platform, so as to recognize and extract system entities of the target big data platform, relationships between entities and entity information, and generating a risk traceability graph of the target big data platform; retrieving the risk traceability graph to obtain suspicious entities; querying a risk traceability graph according to the information of the attack event to obtain a risk propagation chain, and taking an intersection of a risk-related entity and a suspicious entity as a risk entity; supplementing the risk propagation chain according to the components related to the risk entity and the data processing process to obtain a complete risk propagation chain; and performing formalized description on the risk of the target big data platform according to the complete risk propagation chain. The automation degree and the description accuracy can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data platform analysis, and specifically relates to a method and device for formally describing risks of a big data platform. Background Art

[0002] The proposal and development of big data platforms stems from the demand for large-scale data processing. Traditional stand-alone databases have defects such as slow processing speed, high storage cost, and single point failure when facing PB-level data. The big data platform has achieved a significant increase in data storage and processing efficiency through distributed storage and data processing. The big data platform has developed into a professional and diversified ecosystem, which has been supported by rich software and tools in the big data ecosystem in data collection, data transmission, data storage, data processing and other links.

[0003] However, big data platforms have more security risks than general computer systems. A series of formal models for describing computer system risks have been proposed at home and abroad. The formal description of risk refers to the accurate and clear description and representation of risk using mathematical and logical language, and then based on the concept and characteristics of risk, the risk is converted into a set of formalized logical calculation symbols or mathematical expressions, providing a method that can accurately describe the source, characteristics, impact, probability of occurrence and other related factors of risk, thereby helping enterprises better understand, quantify and manage risks and providing an analytical basis for risk assessment.

[0004] The current formal risk description methods often rely on high-level abstract risk indicators, which are difficult to adapt to the actual operation status of the big data platform. As a result, in the actual implementation process, the operation and maintenance personnel need to collect the operation data of the big data platform and convert it into abstract risk indicators. In addition, the current risk formalization model is often manually constructed by security experts. Excessive involvement of human factors makes the risk description results susceptible to subjective factors, resulting in low accuracy of the description results. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for formally describing risks of a big data platform, which can solve the problems that the current formal risk description method has a low degree of automation and low accuracy of the description results.

[0006] In a first aspect, an embodiment of the present invention provides a method for formally describing risks of a big data platform, the method comprising:

[0007] Analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk traceability diagram of the target big data platform, where the security assessment indicators of platform assets in the same standard security domain are the same. The risk traceability diagram includes multiple entity nodes corresponding to system entities one by one, and each entity node has at least one directed edge connected to other entity nodes. The directed edge is used to describe the relationship between two system entities.

[0008] Retrieve the risk traceability graph to obtain suspicious entities;

[0009] According to the information of the attack event, the risk traceability graph is queried to obtain the risk propagation chain, and the intersection of the risk-related entity and the suspicious entity is taken as the risk entity, where the risk-related entity is the entity in the risk propagation chain;

[0010] Supplement the risk propagation chain according to the components and data processing processes related to the risk entity to obtain a complete risk propagation chain;

[0011] The risks of the target big data platform are formally described based on the complete risk propagation chain.

[0012] In a second aspect, an embodiment of the present invention provides a device for formally describing risks of a big data platform, including:

[0013] The global adaptive processing module is used to analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, the relationship between entities and the entity information of the target big data platform, and generate a risk traceability diagram of the target big data platform, wherein the security assessment indicators of the platform assets in the same standard security domain are the same, and the risk traceability diagram includes multiple entity nodes corresponding to the system entities one by one, and each entity node has at least one directed edge connected to other entity nodes, and the directed edge is used to describe the relationship between two system entities;

[0014] The risk retrieval module is used to retrieve the risk traceability graph to obtain suspicious entities; query the risk traceability graph based on the information of the attack event to obtain the risk propagation chain, and take the intersection of the risk-related entity and the suspicious entity as the risk entity, where the risk-related entity is the entity in the risk propagation chain;

[0015] A supplementary retrieval module, which is used to supplement the risk propagation chain according to the components and data processing processes related to the risk entity to obtain a complete risk propagation chain;

[0016] Formal description module,The formal description module is used to formally describe the risks of the target big data platform based on the complete risk propagation chain.

[0017] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory is used to store computer programs; the processor can be used to execute the computer program (instructions) stored in the memory to implement the method of the first aspect above.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the method of the first aspect described above can be implemented.

[0019] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: since the risk tracing graph is automatically generated based on the audit log, when the state of the target big data platform changes, the generated risk tracing graph can also be dynamically updated accordingly without the need for human intervention for conversion; at the same time, the risk propagation chain is obtained by querying the risk tracing graph based on the attack event instead of manually querying the risk propagation path, which can improve the degree of automation; the risk formalization model mapped by the risk propagation chain further improves the objectivity of the analysis results compared to the traditional method of manually constructing the risk formalization model, thereby improving the accuracy of the formal description results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a formal description method for big data platform risks provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a scenario for generating a risk tracing diagram provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a scenario for generating a complete risk propagation chain provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of a scenario of analyzing audit logs using a large language model provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of a risk formalization description model provided by an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of a scenario for mapping a complete risk propagation chain to a risk formal description model provided by an embodiment of the present invention;

[0026] Figure 7 A flowchart of a possible implementation of a formal description method for big data platform risks provided by an embodiment of the present invention;

[0027] Figure 8A schematic diagram of a scenario for defining a standard security domain of a target big data platform provided by an embodiment of the present invention;

[0028] Fig. 9 A schematic diagram of a scenario for generating an audit log provided by an embodiment of the present invention;

[0029] Fig.10 A schematic diagram of the structure of a device for formally describing risks of a big data platform provided by an embodiment of the present invention;

[0030] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0032] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0033] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0034] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

[0035] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0037] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0038] The formal description method of big data platform risks provided by the embodiment of the present invention can be applied to electronic devices such as mobile terminals, personal laptops, supercomputers, etc. The embodiment of the present invention does not impose any restrictions on the specific type of electronic devices.

[0039] Figure 1 The flowchart of the implementation of a formal description method of a big data platform risk shown in the present invention is shown. As an example but not a limitation, the method can be applied to the above electronic device, and the method can include steps S101-S105, and each step is described below.

[0040] S101, analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk traceability diagram for the target big data platform.

[0041] In one possible implementation, the audit logs of each standard security domain in the target big data platform can be analyzed to identify the system entities, inter-entity relationships, and entity information of the target big data platform. Then, entity nodes corresponding to each entity are generated, and each entity node is connected according to the inter-entity relationship. The entity node information is supplemented according to the entity information to generate a risk traceability diagram for the target big data platform (see Figure 2 ).

[0042] Exemplarily, the system entity may be the union of a system process set, a file set, and a network link set within the target big data platform.

[0043] In one example, the relationship between entities may represent an operation of one system entity on another system entity. The entity information may include the standard security domain in which the system entity is located and the time range in which the system entity occurs.

[0044] The five-tuple consisting of system entities, relationships between entities and entity information: (head, relation, tail, location, timetuple) can be used to describe events occurring in the target big data platform; among them, head is the head entity (i.e., the operation entity), tail is the tail entity (i.e., the entity being operated), relation is the type of the operation relationship between the head entity and the tail entity (i.e., the type of the relationship between entities), location is the standard security domain to which these two entities belong, and timetuple = (begin, end) is the time range of the operation, including the start and end time of the event.

[0045] Exemplarily, the operation set between entities may include calling functions such as start, exit, read, write, invoke, kill, create, edit, delete, rename, connect, and listen.

[0046] In one example, see Figure 2 ,The risk tracing graph can include multiple entity nodes corresponding to the system entities one to one, and each entity node has at least one directed edge connecting to other entity nodes.

[0047] For example, a directed edge between entities can be used to describe the relationship between two system entities.

[0048] Specifically, the direction of the directed edge connecting the head entity and the tail entity can be determined according to the causal order, such as the calling relationship between processes, processes and resources (ie, files or network links), or the time sequence of events.

[0049] S102, searching the risk traceability graph to obtain suspicious entities.

[0050] For example, since the number of normal event logs recorded by the big data platform is often much larger than the number of attack event logs, the frequency of entities related to normal events in the risk traceability graph will be higher than that of entities related to attack events. Therefore, system entities whose frequency of appearance in the risk traceability graph is less than the preset frequency threshold can be regarded as suspicious entities, that is, candidate sets of entities related to attack events.

[0051] Optionally, the range of the preset frequency threshold may include 5%-20%, for example, the preset frequency threshold may be 10%.

[0052] S103, query the risk traceability diagram according to the information of the attack event to obtain the risk propagation chain, and take the intersection of the risk-related entity and the suspicious entity as the risk entity.

[0053] Exemplarily, the information of the attack event may include the attack location and the attack time range of the attack event.

[0054] In a possible implementation, the information of the attack event can be used as a query condition to search the risk traceability graph, and obtain a number of search result sets including entities that perform suspicious behaviors within the attack time range. Then, each entity that performs suspicious behaviors in the union of all search result sets is traversed, with the entity node corresponding to the entity in the risk traceability graph as the center, the start time and end event of the attack event as the boundary, and the directed edges in the risk traceability graph are traced forward and backward to obtain the risk propagation chain (see Figure 3 ).

[0055] For example, see Figure 3 , there can be one-to-one, one-to-many, many-to-one and many-to-many relationships in the risk propagation chain.

[0056] In one example, after an entity performs suspicious behavior, the state of at least one system entity in the target big data platform changes or attempts to change, for example, a process, file, network connection, etc. may be changed or attempted to be changed.

[0057] In a possible implementation, all entities in the risk propagation chain may be selected to obtain a risk-related entity set Risk_Related_Entities, which is then intersected with a suspicious entity set Suspicious_Entities, and finally all entities in the intersection result are marked as risk entities.

[0058] Determining potential risk points in the target big data platform based on the attack time range and frequency of occurrence can effectively simplify the discovery, location, association and tracking of big data platform risks.

[0059] Exemplarily, a risk entity may be defined as follows: for any risk-related entity belonging to the risk propagation chain, if the entity also belongs to the suspicious entity set, then the entity is a risk entity, that is, Risk_Entities=Risk_Related_Entities∧Suspicious_Entities.

[0060] S104, supplement the risk propagation chain according to the components and data processing procedures related to the risk entity to obtain a complete risk propagation chain.

[0061] In some embodiments, component nodes can be set around the risk entities in the risk propagation chain, configuration item nodes, security policy nodes, and data processing process nodes can be added around the component nodes, and then risk nodes can be added around the data processing process nodes; risk entity nodes and component nodes, component nodes and configuration item nodes, component nodes and security policy nodes, component nodes and data processing process nodes are connected, and finally the data processing process nodes and risk nodes are connected to obtain a complete risk propagation chain.

[0062] The complete risk propagation chain can provide assessors with a complete, comprehensive, and highly abstract view of the target big data platform's operating status and risk propagation, significantly reducing the workload of security operations and maintenance personnel.

[0063] Exemplarily, the component nodes around the entity node corresponding to the risk entity correspond one-to-one to the relevant components of the risk entity. The relevant components of the risk entity are components that may be used in the event represented by the risk entity. Accordingly, the edge connecting the entity node and the component node can be used to represent the relationship between the risk entity and the component.

[0064] For example, the configuration item nodes and security policy nodes around the component node can be used to represent the configuration item and security policy of the component corresponding to the component node. Accordingly, the edge connecting the component node and the configuration item node, and the edge connecting the component node and the security policy node can respectively represent the relationship between the component and the configuration item, and the relationship between the component and the security policy, such as the meaning of belonging to or including. The configuration item node and the security policy node can be used to subsequently discover the cause of the risk.

[0065] For example, the data processing process nodes around the component node can represent the data processing process to which the component corresponding to the component node belongs, that is, the component used in the data processing process. Accordingly, the edge connecting the component node and the data processing process node can represent the relationship between the component and the data processing process.

[0066] For example, the risk nodes around the data processing process node can represent the type of risk that may be generated when executing the data processing process. Accordingly, the edge connecting the data processing process node and the risk node can represent the relationship between the data processing process and the risk. The data processing process node and the risk node can be used as a basis for subsequent risk assessment.

[0067] In a possible implementation, the configuration file and security policy of the component can be analyzed to obtain the configuration items, configuration values ​​and possible risk types of the component. The configuration value of each configuration item is saved to the configuration item node, and the corresponding risk type is saved to the security policy node. After that, the relationship between the component and the risk entity is extracted from the audit log of the security domain where the component is located, and the relationship between the data processing process and the risk, and the relationship between the data processing process and the component are analyzed.

[0068] Exemplarily, the data processing process can be divided into four types: data collection, storage and synchronization, processing, and application. The types of risks that may arise in different data processing processes can be shown in the following Table 1:

[0069] Table 1

[0070]

[0071] In one example, see Figure 4 , a large language model can be used to analyze the configuration files and security policies of each component in the target big data platform to solve the problems existing in the traditional rule-based analysis method when analyzing a wide variety of component configuration files and security policies that are constantly updated.

[0072] For details, see Figure 4 , the content of each component's configuration file and security policy can be input into the large language model for processing. If the content length is too long, it is segmented so that the segmented content can be processed by the large language model, thereby ensuring the quality and accuracy of the processing results. Then, through prompt word engineering design and optimization of prompt words, the large language model is guided to generate component configuration and security policy analysis results in a fixed format. The prompt words may include configuration file analysis targets and security policy analysis targets. The format of the analysis results output by the large language model can be shown in Table 2 below.

[0073] Table 2

[0074]

[0075] S105, formally describe the risks of the target big data platform based on the complete risk propagation chain.

[0076] In some embodiments, each node in the complete risk propagation chain can be mapped to a risk formal description model, and then the formalized entities and relationships in the model are integrated to obtain a risk factor set. According to the risk factor set, the risk of the target big data platform and the impact of the risk are formally described.

[0077] In one possible implementation, see Figure 5 ,The risk formal description model can be a diagram used to describe the ,relationship between one or more of the following risk elements: ,components, data processing processes, threats, vulnerabilities, risks, security policies and ,security events.

[0078] For example, see Figure 5,The edges connecting each graph in the risk formalization model can represent: the existence of vulnerabilities in components, that is, improper settings of components or security policies will lead to risks in the big data platform; threats will use vulnerabilities to increase risks, that is, there are certain types of threats that can successfully use vulnerabilities with a certain probability, increasing certain risks to a certain extent; risks exist in the data processing process, that is, threats can successfully use vulnerabilities to cause a certain probability and certain degree of impact on the data processing process; security policies reduce vulnerabilities, that is, there are certain security policies that can remove certain vulnerabilities on certain entities with a certain probability; security policies resist threats, that is, there are certain security policies that act on certain entities that can reduce the probability of certain threats successfully using certain vulnerabilities with a certain probability; threats evolve into security events, that is, when a certain threat successfully uses vulnerabilities to increase a certain risk and make the risk exceed the threshold, the threat can evolve into a security event; security time affects asset value, that is, security events can affect the confidentiality and integrity of assets. Availability, endanger asset value or affect the normal operation of businesses that rely on assets.

[0079] For example, the risk of a target big data platform in a standard security domain can be a collection of vulnerabilities within the time range of an attack event in the standard security domain, threats that exploit the vulnerability, and possible consequences. The overall risk of the target big data platform can be a collection of all risks in all security domains.

[0080] Exemplarily, the overall impact caused by the risk of the target big data platform may be the sum of the asset values ​​corresponding to all entities in all security domains whose states have been changed due to the threat.

[0081] Exemplarily, the assets of the target big data platform may include components and data processing processes, and the asset value may be equal to the product of the importance coefficient of the component or data processing process and the security requirement coefficient in the security domain in which it is located.

[0082] Optionally, the safety requirement factor and the importance factor can be given by the assessor.

[0083] In one example, see Figure 6 , the vulnerabilities in the risk formalization description model can be mapped through the configuration item nodes and security policy nodes in the complete risk propagation chain, and the components and security policies in the model can be mapped through the component nodes and security policy nodes respectively, the data processing flow in the model can be mapped through the data processing flow nodes, the risks in the model can be mapped through the risk nodes, and the threats in the model can be mapped through the entity nodes of the risk entities.

[0084] For example, see Figure 6 , there will be some kind of tail entity in the risk transmission chain, namely the vulnerability entity (see Figure 6 Entity 1 in the example and the relationship to the tail entity (see Figure 6 101 in ), so that some threat entities (see Figure 6 Entity 2 in the relationship can trigger the operation corresponding to the relationship to reach the tail entity with a certain probability. Therefore, the configuration item node and security policy node of the component connected to the tail entity (see Figure 6 The configuration item node 1, configuration item node 2 and security policy node 1) in the risk formal description model are mapped to the vulnerabilities in the risk formal description model.

[0085] For example, see Figure 6 , there will be some kind of head entity in the risk propagation chain, namely the threat entity (see Figure 6 Entity 1) and the relationship from the threat entity (see Figure 6 102 in ), so that the threat entity can trigger the operation corresponding to the relationship with a certain probability and reach certain vulnerable entities through path 102 (see Figure 6 Entities in 3). Therefore, threat entities and corresponding relationships can be mapped into the model as threats.

[0086] It should be understood that in the risk propagation chain, the complete risk propagation chain and the risk traceability diagram, an entity node can serve as both a tail entity and a head entity, can be affected by other entities as a vulnerability entity, and can also affect other entities as a threat entity.

[0087] For example, each security measure under a security domain can reduce the vulnerability of a certain type of system entity by performing certain processing on the entity (see Figure 6 Entity 1) The corresponding relationship R vuli (See Figure 6 The probability q of the operation corresponding to 101) being triggered vuli , or reduce the corresponding relationship R of the threat entity thri The probability q of the corresponding operation being triggered thri .

[0088] Since the risk traceability diagram is automatically generated based on the audit log, when the status of the target big data platform changes, the generated risk traceability diagram can also be dynamically updated without the need for human intervention for conversion; at the same time, the risk propagation chain can be obtained by querying the risk traceability diagram based on the attack event instead of manually querying the risk propagation path, which can improve the degree of automation; mapping the risk formal model through the risk propagation chain can further improve the objectivity of the analysis results compared to the traditional manual construction of the risk formal model, thereby improving the accuracy of the formal description results.

[0089] Furthermore, the risk formal description model provided by the present invention, which is composed of the basic risk elements, has strong applicability and can describe various big data platforms; the risk propagation chain and risk traceability diagram updated according to the dynamic changes of the target big data platform can adapt to the dynamic changes of the components and businesses of the target big data platform, and improve the accuracy and timeliness of the formal description results. Since the entire risk formal description process is automatically completed by the computer, the high-speed computing and analysis capabilities of the computer can be used to describe and predict the potential risks of the big data platform in real time, and at the same time, the labor cost can be significantly reduced.

[0090] Figure 7 The flowchart of a possible implementation method of a formal description method of a big data platform risk provided by an embodiment of the present invention is shown. As an example but not a limitation, the method can be a specific possible implementation method of the above description method, which can include steps S701-S705, and each step is described below.

[0091] S701, divide the target big data platform into standard security domains.

[0092] In one possible implementation, see Figure 8 , we can first identify all the security assessment indicators of the target big data platform, generate the initial security tree of the target big data platform based on these security assessment indicators, and then calculate the embedding vector of the initial security tree based on the embedding vector of each leaf node in the initial security tree. After that, we match the standard security tree of the target big data platform in the security standard template library according to the embedding vector of the initial security tree; finally, we divide the standard security domain of the target big data platform according to the standard security tree.

[0093] Exemplarily, the security assessment indicators of platform assets in the same initial security domain / standard security domain are the same.

[0094] In one example, the security standard template library may store multiple standard security domain templates constructed by security assessment indicators. Each row in the template library stores a standard code, standard name, standard indicator and standard indicator embedding vector corresponding to a standard security domain template.

[0095] For example, natural language processing technology can be used to identify a set of security evaluation criteria for a standard big data platform, and the indicator can be extracted as a standard security domain. For each set of big data platform security evaluation indicators input, a tree diagram is generated with the big data platform as the root node and the standard security domain as the leaf node. For this tree diagram, the embedding vectors of the merged leaf nodes are recursively calculated from the root node, the vector representation of the root node is updated, and the final embedding vector of the root node is used as the standard embedding vector of a standard security domain template.

[0096] In one example, similarly, when processing a target big data platform, the platform assets can be evaluated to generate a tree diagram (i.e., an initial security tree) with the target big data platform as the root node and each initial security domain as a leaf node. Then, starting from the root node, the embedding vector of the entire tree diagram can be recursively calculated.

[0097] In one example, the distance between the embedding vector of the initial security tree and the embedding vector of each standard security domain template in the security standard template library can be calculated, and the standard security domain template with the smallest distance can be selected as the standard security tree of the target big data platform.

[0098] In one example, the leaf nodes of the standard security tree may be used as standard security domains to divide the assets of the target big data platform to obtain the standard security domains of the target big data platform.

[0099] For example, a standard security domain may include multiple platform assets. Each platform asset has corresponding security requirements and security strategies. Platform assets in the same standard security domain have similar security requirements and strategies.

[0100] Exemplarily, the standard security domain is unique, the intersection between any two security domain sets is an empty set, and the union of all security domains constitutes the target big data platform.

[0101] S702, extract and generate audit logs within the standard security domain.

[0102] In one possible implementation, see Fig. 9 , you can use the log collection tool to collect and store logs from each component and subsystem in each standard security domain. Clean the stored log data to get the audit log.

[0103] In one example, the log sources can be integrated and marked according to the platform business process, and the logs can be imported into the log database for storage, while generating a log index as the original data for formalizing the risk description.

[0104] In one example, all log entries can be retrieved, and the format of the log data can be standardized using an ETL data cleaning tool to remove missing values, outliers, and invalid or duplicate data. The components and security domains to which the log entries belong can be marked based on the log sources for subsequent location of risk sources.

[0105] In a possible implementation, the configuration file of the component can be extracted based on the audit log, and the input and output of the data during the data processing process can be analyzed to locate the location of the attack event, the risk type, and the cause of the risk.

[0106] In an example, the data processing process can be divided into four types: data collection, data storage and synchronization, data processing, and data application according to the general architecture and business processes of the big data platform; the components of the target big data platform can be divided according to the data processing process and then the configuration files of the components can be extracted according to different file suffixes.

[0107] For example, in the data collection stage, you can generally deploy components such as Logstash, Kafka, Flume, ETL ETL tools, and SQL tools. In the data storage stage, you can generally deploy data synchronization components such as Zookeeper, database management software such as MySQL, HBase, Hive, Sqoop, and Cassandra, and file system components such as HDFS and GFS. In the data processing stage, you can generally deploy data batch processing or stream processing components such as Spark, Storm, and MapReduce. In the data application stage, you can generally deploy components such as Solr and Elasticsearch.

[0108] For example, the suffix of the component configuration file is generally ".properties", ".conf", ".xml", "yaml", "cfg", "sh". The component configuration file can be obtained by searching and extracting files with such suffixes in the corresponding component file directory in the system file explorer.

[0109] S703, construct a risk traceability diagram.

[0110] S704, generate a complete risk traceability chain based on the risk traceability diagram.

[0111] S705, formally describe the target big data platform based on the complete risk traceability chain.

[0112] Exemplarily, steps S701-S705 are the same as steps S101-S105 in the above method. For details, please refer to the description of steps S101-S105, which will not be repeated here.

[0113] As an example, during the execution of the above method, the electronic device can formalize the parameters generated or defined in the description process through the z language so that these parameters can be read, stored and further analyzed by the computer, and the ambiguity or ambiguity existing in natural language can be avoided.

[0114] Fig.10 The structure diagram of a formal description device for a big data platform risk provided by an embodiment of the present invention is shown. As an example but not a limitation, the device 1000 may include a global adaptive processing module 1010, a risk retrieval module 1020, a supplementary retrieval module 1030 and a formal description module 1040.

[0115] Exemplarily, the global adaptive processing module 1010 is used to analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, entity relationships and entity information of the target big data platform, and generate a risk traceability graph for the target big data platform, wherein the security assessment indicators of the platform assets in the same standard security domain are the same, and the risk traceability graph includes multiple entity nodes corresponding to the system entities one by one, and each entity node has at least one directed edge connected to other entity nodes, and the directed edge is used to describe the relationship between two system entities; the risk retrieval module 1020 is used to retrieve the risk traceability graph to obtain suspicious entities; the risk traceability graph is queried according to the information of the attack event to obtain the risk propagation chain, and the intersection of the risk-related entity and the suspicious entity is taken as the risk entity, wherein the risk-related entity is the entity in the risk propagation chain; the supplementary retrieval module 1030 is used to supplement the risk propagation chain according to the components and data processing processes related to the risk entity to obtain a complete risk propagation chain; the formal description module 1040 is used to formally describe the risks of the target big data platform according to the complete risk propagation chain.

[0116] Since the risk traceability diagram is automatically generated based on the audit log, when the status of the target big data platform changes, the generated risk traceability diagram can also be dynamically updated without the need for human intervention for conversion; at the same time, the risk propagation chain can be obtained by querying the risk traceability diagram based on the attack event instead of manually querying the risk propagation path, which can improve the degree of automation; mapping the risk formal model through the risk propagation chain can further improve the objectivity of the analysis results compared to the traditional manual construction of the risk formal model, thereby improving the accuracy of the formal description results.

[0117] Fig.11 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Fig.11 The electronic device 1100 shown may include: at least one processor 1110 ( Fig.11 Only one processor is shown in the figure), a memory 1120, and a computer program 1130 stored in the memory 1120 and executable on the at least one processor 1110, wherein the processor 1110 implements the steps of any of the above-mentioned method embodiments when executing the computer program 1130.

[0118] The electronic device 1100 may be a processing device such as a robot that can implement the above method. The embodiment of the present invention does not impose any limitation on the specific type of the electronic device.

[0119] Those skilled in the art will understand that Fig.11The electronic device 1100 is merely an example and does not constitute a limitation on the electronic device. The electronic device 1100 may include more or fewer components than shown in the figure, or may combine certain components, or may include different components. For example, the electronic device 1100 may also include an input and output interface.

[0120] The processor 1110 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASTC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0121] The memory 1120 may be an internal storage unit in some embodiments, such as a hard disk or a memory. The memory 1120 may also be an external storage device in other embodiments, such as a plug-in hard disk, a smart memory card (SmartMemory Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 1120 may also include both an internal storage unit and an external storage device. The memory 1120 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program, etc. The memory 1120 may also be used to temporarily store data that has been output or is to be output.

[0122] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0123] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0124] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0125] An embodiment of the present invention provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0127] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0128] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Claims

1. A formal description method for big data platform risks, characterized in that: include: Analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk tracing diagram of the target big data platform, wherein the security assessment indicators of platform assets in the same standard security domain are the same, and the risk tracing diagram includes a plurality of entity nodes corresponding to the system entities one by one, and each entity node has at least one directed edge connected to other entity nodes, and the directed edge is used to describe the relationship between two system entities; Retrieving the risk tracing graph to obtain suspicious entities; According to the information of the attack event, the risk tracing graph is queried to obtain a risk propagation chain, and the intersection of the risk-related entity and the suspicious entity is taken as the risk entity, wherein the risk-related entity is an entity in the risk propagation chain; Supplement the risk propagation chain according to the components and data processing processes related to the risk entity to obtain a complete risk propagation chain; The risks of the target big data platform are formally described based on the complete risk propagation chain.

2. The method according to claim 1, characterized in that Before analyzing the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities and entity information of the target big data platform and generate a risk tracing diagram of the target big data platform, the method further includes: Identify all security assessment indicators of the target big data platform, and extract an initial security tree of the target big data platform based on the security assessment indicators, wherein the initial security tree is a tree diagram with the initial security domain as a leaf node and the target big data platform as a root node; Calculating an embedding vector of the initial security tree according to the embedding vector of each leaf node in the initial security tree, wherein the embedding vector of the leaf node is a security assessment indicator of the initial security domain corresponding to the leaf node; Matching the standard security tree of the target big data platform in the security standard template library according to the embedding vector of the initial security tree; The standard security domain of the target big data platform is divided and obtained according to the standard security tree.

3. The method according to claim 1, characterized in that The suspicious entity is a system entity whose occurrence frequency in the risk tracing diagram is less than a preset frequency threshold.

4. The method according to claim 1, characterized in that: The entity information includes the time range of the system entity and the standard security domain where the system entity is located.

5. The method according to claim 4, characterized in that The information of the attack event includes the time range of the attack; The step of querying the risk traceability diagram according to the information of the attack event to obtain the risk propagation chain includes: Querying the risk tracing graph for system entities that perform suspicious behaviors within the attack time range, wherein the state of at least one system entity in the target big data platform changes or attempts to change after the system entity performs the suspicious behavior; The risk propagation chain is obtained by searching the risk tracing graph with the entity performing the suspicious behavior as the center and the attack time range as the boundary.

6. The method according to claim 1, characterized in that The step of supplementing the risk propagation chain according to the components and data processing process related to the risk entity to obtain a complete risk propagation chain includes: Add component nodes around the entity nodes corresponding to the risk entities, add configuration item nodes, security policy nodes and data processing process nodes around the component nodes, and add risk nodes around the data processing process nodes; The component nodes correspond to the components related to the risk entity one by one, the configuration item nodes and security policy nodes around the component nodes are used to represent the configuration items and security policies of the related components respectively, the data processing process nodes are used to represent the data processing process to which the related components belong, and the risk nodes are used to represent the type of risk that occurs when executing the data processing process to which the related components belong; Connecting the risk entity with the component node, the component node with the configuration item node, the component node with the security policy node, the component node with the data processing process node, and the data processing process node with the risk node to complete the supplementation of the risk entity; After completing the supplement of all risk entities, the complete risk propagation chain is obtained.

7. The method according to claim 6, characterized in that The formal description of the risk of the target big data platform according to the complete risk propagation chain includes: Mapping the risk formalization model according to the complete risk propagation chain to obtain a set of risk factors of the target big data platform, wherein the risk formalization description model is a diagram for describing the relationship between one or more of the following risk factors: components, data processing processes, threats, vulnerabilities, risks, security policies, and security events; A formal description is made of the risks of the target big data platform and the impacts of the risks based on the risk factor set.

8. A device for formally describing risks of a big data platform, characterized in that: include: A global adaptive processing module, which is used to analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk tracing diagram of the target big data platform, wherein the security assessment indicators of platform assets in the same standard security domain are the same, and the risk tracing diagram includes a plurality of entity nodes corresponding to the system entities one by one, and each entity node has at least one directed edge connected to other entity nodes, and the directed edge is used to describe the relationship between two system entities; A risk retrieval module, the risk retrieval module is used to retrieve the risk tracing graph to obtain suspicious entities; query the risk tracing graph according to the information of the attack event to obtain a risk propagation chain, and take the intersection of the risk-related entity and the suspicious entity as the risk entity, wherein the risk-related entity is an entity in the risk propagation chain; A supplementary retrieval module, the supplementary retrieval module is used to supplement the risk propagation chain according to the components and data processing processes related to the risk entity to obtain a complete risk propagation chain; A formal description module is used to formally describe the risks of the target big data platform according to the complete risk propagation chain.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the electronic device, the method according to any one of claims 1 to 7 is implemented.

Images