Formalized description method and device of big data platform risk

Abstract

The application discloses a kind of formalized description method and device of big data platform risk, the method includes: the audit log in each standard security domain of target big data platform is analyzed, to identify and extract the system entity of target big data platform, inter-entity relationship and entity information, generate the risk traceability diagram of target big data platform;Suspect entity is obtained by searching risk traceability diagram;According to the information of attack event, risk traceability diagram is obtained to obtain risk propagation chain, and the intersection of risk-related entity and suspect entity is used as risk entity;According to the component and data processing process related to risk entity, the complete risk propagation chain is obtained by supplementing risk propagation chain;According to complete risk propagation chain, the risk of target big data platform is formalized described.The application can improve the degree of automation and description accuracy.

Description

Technical Field

[0001] This invention belongs to the field of big data platform analysis technology, specifically relating to a formal description method and apparatus for big data platform risks. Background Technology

[0002] The emergence and development of big data platforms stemmed from the demand for large-scale data processing. Traditional single-machine databases suffer from drawbacks such as slow processing speed, high storage costs, and single points of failure when dealing with petabytes of data. Big data platforms, however, have achieved significant improvements in data storage capacity and processing efficiency through distributed storage and data processing. To date, big data platforms have developed into a professional and diverse ecosystem, supported by a wealth of software and tools within the big data ecosystem at every stage, including data acquisition, data transmission, data storage, and data processing.

[0003] However, big data platforms pose more security risks than general computer systems. A series of formal models for describing computer system risks have been proposed both domestically and internationally. Formal risk description refers to the accurate and clear description and representation of risk using mathematical and logical language. Based on the concept and characteristics of risk, it transforms risk into a set of formal logical calculus symbols or mathematical expressions. This provides a method that can accurately describe the sources, characteristics, impacts, and probabilities of risk, thereby helping enterprises better understand, quantify, and manage risks, and providing an analytical foundation for risk assessment.

[0004] Current formal risk description methods often rely on high-level abstract risk indicators, which are difficult to adapt to the actual operating conditions of big data platforms. This necessitates that operations and maintenance personnel collect operational data from the big data platform themselves and convert it into abstract risk indicators during implementation. Furthermore, current formal risk models are often manually constructed by security experts. Excessive human involvement makes the risk description results susceptible to subjective influences, leading to low accuracy. Summary of the Invention

[0005] This invention provides a formal description method and apparatus for risks on big data platforms, which can solve the problems of low automation and low accuracy of current formal risk description methods.

[0006] In a first aspect, embodiments of the present invention provide a formal description method for risks of big data platforms, the method comprising:

[0007] The audit logs of each standard security domain of the target big data platform are analyzed to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk tracing diagram of the target big data platform. Among them, the security assessment indicators of platform assets in the same standard security domain are the same. The risk tracing diagram includes multiple entity nodes that correspond one-to-one with system entities. Each entity node has at least one directed edge connected to other entity nodes. The directed edge is used to describe the relationship between two system entities.

[0008] Suspicious entities were identified by retrieving the risk source map;

[0009] Based on the information of the attack event, the risk propagation chain is obtained by querying the risk tracing graph, and the intersection of risk-related entities and suspicious entities is taken as the risk entity, where the risk-related entity is the entity in the risk propagation chain;

[0010] A complete risk propagation chain is obtained by supplementing the risk propagation chain with components and data processing procedures related to the risk entity;

[0011] The risks of the target big data platform are formally described based on a complete risk propagation chain.

[0012] Secondly, embodiments of the present invention provide a formal description apparatus for big data platform risks, including:

[0013] The global adaptive processing module is used to analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk tracing diagram of the target big data platform. Among them, the security assessment indicators of platform assets in the same standard security domain are the same. The risk tracing diagram includes multiple entity nodes that correspond one-to-one with system entities. Each entity node has at least one directed edge connected to other entity nodes. The directed edge is used to describe the relationship between two system entities.

[0014] The risk retrieval module is used to retrieve suspicious entities from the risk attribution graph; it queries the risk attribution graph based on the information of the attack event to obtain the risk propagation chain, and takes the intersection of risk-related entities and suspicious entities as the risk entity, where the risk-related entities are the entities in the risk propagation chain;

[0015] The supplementary retrieval module is used to supplement the risk propagation chain based on the components and data processing procedures related to the risk entity to obtain a complete risk propagation chain;

[0016] The formal description module is used to formally describe the risks of the target big data platform based on a complete risk propagation chain.

[0017] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory is used to store a computer program; the processor can be used to execute a calculator program (instructions) stored in the memory to implement the method of the first aspect described above.

[0018] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed, can implement the method described in the first aspect above.

[0019] The beneficial effects of this invention compared to existing technologies are as follows: Since the risk origination diagram is automatically generated based on audit logs, it can be dynamically updated when the status of the target big data platform changes, without the need for manual conversion; at the same time, the risk propagation chain is obtained by querying the risk origination diagram based on the attack event, rather than by manually querying the risk propagation path, which improves the degree of automation; and mapping the risk formal model through the risk propagation chain further improves the objectivity of the analysis results compared to the traditional method of manually constructing a risk formal model, thereby improving the accuracy of the formal description results. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the implementation of a formal description method for risks in a big data platform, provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram illustrating a scenario for generating a risk tracing map, provided by an embodiment of the present invention.

[0022] Figure 3 A schematic diagram illustrating a scenario for generating a complete risk propagation chain, provided by an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram illustrating a scenario for analyzing audit logs using a large language model, as provided in an embodiment of the present invention.

[0024] Figure 5 A schematic diagram of a formal risk description model provided in an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram illustrating a scenario for mapping a complete risk propagation chain to a formal risk description model, as provided in an embodiment of the present invention.

[0026] Figure 7 A flowchart illustrating one possible implementation of a formal description method for risks on a big data platform, provided by an embodiment of the present invention.

[0027] Figure 8This is a schematic diagram illustrating a scenario for defining a standard security domain for a target big data platform, provided by an embodiment of the present invention.

[0028] Figure 9 This is a schematic diagram illustrating a scenario for generating audit logs, provided by an embodiment of the present invention.

[0029] Figure 10 A schematic diagram of the structure of a formal description device for big data platform risks provided in an embodiment of the present invention;

[0030] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0032] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0033] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0034] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0035] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0037] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0038] The formal description method for big data platform risks provided in this embodiment of the invention can be applied to electronic devices such as mobile terminals, personal laptops, and supercomputers. This embodiment of the invention does not impose any restrictions on the specific type of electronic device.

[0039] Figure 1 The diagram shown is a flowchart illustrating the implementation of a formal description method for risks in a big data platform according to the present invention. As an example and not a limitation, this method can be applied to the aforementioned electronic device. The method may include steps S101-S105, which are described below.

[0040] S101, analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk tracing diagram of the target big data platform.

[0041] In one possible implementation, audit logs from each standard security domain within the target big data platform can be analyzed to identify system entities, inter-entity relationships, and entity information. Then, entity nodes corresponding to each entity are generated, these nodes are connected based on inter-entity relationships, and entity node information is supplemented based on the entity information to generate a risk tracing diagram of the target big data platform (see [link to relevant documentation]). Figure 2 ).

[0042] For example, a system entity can be the union of a set of system processes, a set of files, and a set of network links within the target big data platform.

[0043] In one example, an entity relationship can represent an operation performed by one system entity on another. Entity information can include the standard security domain in which the system entity operates and the timeframe in which the operation occurs.

[0044] A quintuple consisting of system entities, relationships between entities, and entity information (head, relation, tail, location, timetuple) can be used to describe events occurring within a target big data platform. Here, head is the head entity (i.e., the entity being operated on), tail is the tail entity (i.e., the entity being operated on), relation is the type of operation relationship between the head and tail entities (i.e., the type of relationship between entities), location is the standard security domain to which these two entities belong, and timetuple = (begin, end) is the time range of the operation, including the start and end times of the event.

[0045] For example, the set of operations between entities may include function calls such as start, exit, read, write, invoke, kill, create, edit, delete, rename, connect, and listen.

[0046] In one example, see Figure 2 The risk tracing graph can include multiple entity nodes that correspond one-to-one with system entities, and each entity node has at least one directed edge connected to other entity nodes.

[0047] For example, directed edges between entities can be used to describe the relationship between two system entities.

[0048] Specifically, the orientation of directed edges connecting head and tail entities can be determined according to causal order, such as the call relationships between processes, between processes and resources (i.e., files or network links), or the chronological order of events.

[0049] S102, retrieve the risk tracing map to obtain suspicious entities.

[0050] For example, since the number of normal event logs recorded by big data platforms is often far greater than the number of attack event logs, entities related to normal events will appear more frequently in the risk source map than entities related to attack events. Therefore, system entities that appear less frequently than a preset frequency threshold in the risk source map can be considered as suspicious entities, i.e., a candidate set of entities related to attack events.

[0051] Optionally, the preset frequency threshold may range from 5% to 20%, for example, the preset frequency threshold may be 10%.

[0052] S103: Based on the information of the attack event, query the risk source map to obtain the risk propagation chain, and take the intersection of the risk-related entities and the suspicious entities as the risk entities.

[0053] For example, information about an attack event may include the location of the attack and the time range of the attack.

[0054] In one possible implementation, attack event information can be used as a query condition to retrieve the risk propagation graph, resulting in a set of search results including entities that performed suspicious behavior within the attack timeframe. Then, for each entity performing suspicious behavior in the union of all search result sets, the system traces the risk propagation chain forward and backward along the directed edges of the risk propagation graph, using the corresponding entity node in the risk propagation graph as the center and the start and end times of the attack event as boundaries. (See [link to relevant documentation]). Figure 3 ).

[0055] For example, see Figure 3 In the risk transmission chain, there can be one-to-one, one-to-many, many-to-one, and many-to-many relationships.

[0056] In one example, after an entity performs suspicious behavior, the state of at least one system entity in the target big data platform will change or attempt to change, such as changing or attempting to change processes, files, network connections, etc.

[0057] In one possible implementation, all entities in the risk propagation chain can be selected to obtain a risk-related entity set, Risk_Related_Entities. Then, the intersection operation is performed between this set and a suspicious entity set, Suspicious_Entities. Finally, all entities in the intersection operation result are marked as risk entities.

[0058] Identifying potential risk points in a target big data platform based on the attack time range and frequency can effectively simplify the discovery, location, correlation, and tracking of risks in big data platforms.

[0059] For example, a risk entity can be defined as follows: for any risk-related entity belonging to the risk propagation chain, if the entity also belongs to the set of suspicious entities, then the entity is a risk entity. That is, Risk_Entities = Risk_Related_Entities ∧ Suspicious_Entities.

[0060] S104, a complete risk propagation chain is obtained by supplementing the risk propagation chain with the components and data processing procedures related to the risk entity.

[0061] In some embodiments, component nodes can be set around risk entities in the risk propagation chain, configuration item nodes, security policy nodes, and data processing process nodes can be added around component nodes, and then risk nodes can be added around data processing process nodes; risk entity nodes can be connected to component nodes, component nodes to configuration item nodes, component nodes to security policy nodes, component nodes to data processing process nodes, and finally data processing process nodes to risk nodes to obtain a complete risk propagation chain.

[0062] A complete risk propagation chain can provide assessors with a complete, comprehensive, and highly abstract view of the target big data platform's operational status and risk propagation, significantly reducing the workload of security operations personnel.

[0063] For example, the component nodes surrounding the entity node corresponding to the risk entity correspond one-to-one with the relevant components of the risk entity. The relevant components of the risk entity are the components that may be used in the event represented by the risk entity. Accordingly, the edges connecting the entity node and the component node can be used to represent the relationship between the risk entity and the components.

[0064] For example, configuration item nodes and security policy nodes surrounding a component node can be used to represent the configuration items and security policies of the component corresponding to that component node. Accordingly, edges connecting a component node and a configuration item node, and edges connecting a component node and a security policy node, can respectively represent the relationship between the component and the configuration item, and the component and the security policy, for example, indicating belonging to or including. Configuration item nodes and security policy nodes can be used to subsequently discover the causes of risks.

[0065] For example, the data processing nodes surrounding a component node can represent the data processing process to which the component node belongs, i.e., the component used in this data processing process. Accordingly, the edges connecting the component node and the data processing node can represent the relationship between the component and this data processing process.

[0066] For example, risk nodes surrounding a data processing node can represent the types of risks that may arise when performing such a data processing procedure. Accordingly, the edges connecting the data processing node and the risk node can represent the relationship between the data processing procedure and the risk. The data processing node and the risk node can be used as a basis for subsequent risk assessment.

[0067] In one possible implementation, the component's configuration files and security policies can be analyzed to obtain the component's configuration items, configuration values, and potential risk types. The configuration values ​​of each item are saved to the configuration item node, and the corresponding risk types are saved to the security policy node. Then, the relationship between the component and risk entities is extracted from the audit logs of the security domain in which the component resides. Simultaneously, the relationship between data processing procedures and risks, and the relationship between data processing procedures and components, are analyzed.

[0068] For example, data processing can be divided into four types: data acquisition, storage and synchronization, processing, and application. The types of risks that may arise in different data processing processes are shown in Table 1 below:

[0069] Table 1

[0070]

[0071] In one example, see Figure 4 Large language models can be used to analyze the configuration files and security policies of various components in a target big data platform, thereby solving the problems of traditional rule-based analysis methods when analyzing a wide variety of constantly updated component configuration files and security policies.

[0072] For details, see Figure 4 The configuration files and security policies of each component can be input into a large language model for processing. If the content is too long, it is segmented to ensure that the segmented content can be processed by the large language model, thereby ensuring the quality and accuracy of the processing results. Then, through prompt word engineering and optimization, the large language model is guided to generate component configuration and security policy analysis results in a fixed format. The prompt words can include configuration file analysis objectives and security policy analysis objectives. The format of the analysis results output by the large language model can be as shown in Table 2 below.

[0073] Table 2

[0074]

[0075] S105 provides a formal description of the risks of the target big data platform based on a complete risk propagation chain.

[0076] In some embodiments, each node in the complete risk propagation chain can be mapped to a formal risk description model, and then the formal entities and relationships in the model can be integrated to obtain a set of risk elements. A formal description is then performed based on the risk elements set to the target big data platform and the impact of those risks.

[0077] In one possible implementation, see Figure 5 A formal risk description model can be a diagram used to describe the relationship between one or more of the following risk elements: components, data processing, threats, vulnerabilities, risks, security policies, and security incidents.

[0078] For example, see Figure 5In the formal risk description model, the edges connecting the various graphs can represent: component vulnerabilities, meaning that improper component or security policy settings can lead to risks in the big data platform; threats exploiting vulnerabilities to increase risk, meaning that a certain type of threat can successfully exploit a vulnerability with a certain probability, increasing the risk to some extent; data processing risks, meaning that threats can successfully exploit vulnerabilities to affect the data processing process with a certain probability and degree; security policies reducing vulnerabilities, meaning that a certain security policy can remove a certain vulnerability on a certain entity with a certain probability; security policies defending against threats, meaning that a certain security policy acting on a certain entity can reduce the probability of a certain threat successfully exploiting a certain vulnerability; threats evolving into security events, meaning that when a threat successfully exploits a vulnerability to increase a certain risk, causing the risk to exceed a threshold, the threat can evolve into a security event; security events affecting asset value, meaning that security events can affect the confidentiality and integrity of assets; and availability, which jeopardizes asset value or affects the normal operation of businesses that depend on the asset.

[0079] For example, the risk of a target big data platform within a standard security domain can be the set of vulnerabilities within the timeframe of an attack event in that standard security domain, the threats that exploit those vulnerabilities, and the potential consequences. The overall risk of the target big data platform can be the set of all risks across all security domains.

[0080] For example, the overall impact of the risk to the target big data platform can be the sum of the asset values ​​of all entities within the entire security domain whose state has been altered due to the effects of the threat.

[0081] For example, the assets of a target big data platform may include components and data processing procedures, and the asset value may be equal to the product of the importance coefficient of the component or data processing procedure and the security requirement coefficient within its security domain.

[0082] Optionally, the safety requirement coefficient and importance coefficient can be provided by the assessor.

[0083] In one example, see Figure 6 Vulnerabilities in the formal description model can be mapped to configuration item nodes and security policy nodes in a complete risk propagation chain. Component nodes and security policy nodes can be mapped to components and security policies in the model, data processing flow nodes can be mapped to data processing flow in the model, risk nodes can be mapped to risks in the model, and entity nodes of risk entities can be mapped to threats in the model.

[0084] For example, see Figure 6 In the risk propagation chain, there will be a certain tail entity, namely a vulnerable entity (see...). Figure 6 Entity 1 in the middle) and the relationship connected to the tail entity (see Figure 6 101 in the middle), which makes certain threat entities (see 101) possible. Figure 6 Entity 2) in the context can trigger the corresponding operation to reach the tail entity with a certain probability. Therefore, the configuration item node and security policy node of the component connected to the tail entity (see [link to configuration item]) can be configured. Figure 6 Configuration item node 1, configuration item node 2, and security policy node 1 in the model are mapped to vulnerabilities in the formal risk description model.

[0085] For example, see Figure 6 In the risk propagation chain, there will be a head entity, namely the threat entity (see...). Figure 6 Entity 1 in the context of threats and the relationships derived from the threat entity (see Entity 1 in the context of threats) Figure 6 (102 in the text) allows a threatening entity to trigger the corresponding operation with a certain probability, thereby reaching certain vulnerable entities through path 102 (see 102 in the text). Figure 6 (Entity 3 in the model). Therefore, threat entities and their corresponding relationships can be mapped into the model as threats.

[0086] It should be understood that in the risk propagation chain, the complete risk propagation chain, and the risk sourcing diagram, an entity node can act as both a tail entity and a head entity, and can be affected by other entities as a vulnerable entity, as well as affect other entities as a threatening entity.

[0087] For example, various security measures within a security domain can reduce the vulnerability of certain system entities by performing certain processing on them (see [reference]). Figure 6 Entity 1) and its corresponding relation R vuli (see Figure 6 The probability q of the operation corresponding to 101) being triggered. vuli Or reduce the corresponding relationship R of the threatening entity. thri The probability q of the corresponding operation being triggered thri .

[0088] Since the risk attribution map is automatically generated based on audit logs, it can be dynamically updated as the status of the target big data platform changes, without the need for manual conversion. Furthermore, querying the risk attribution map based on attack events to obtain the risk propagation chain, rather than manually querying the path, improves automation. Mapping the risk formal model through the risk propagation chain further enhances the objectivity of the analysis results compared to traditional manual risk formal model construction, thereby improving the accuracy of the formal description results.

[0089] Furthermore, the formal risk description model provided by this invention, composed of its basic risk elements, has strong applicability and can describe various big data platforms. The risk propagation chain and risk sourcing diagram, updated according to the dynamic changes of the target big data platform, can adapt to the dynamic changes in the components and business operations of the target big data platform, improving the accuracy and timeliness of the formal description results. Since the entire formal risk description process is completed automatically by a computer, it can utilize the high-speed computing and analysis capabilities of computers to describe and predict the potential risks of big data platforms in real time, while significantly reducing labor costs.

[0090] Figure 7 The diagram shown is a possible implementation of a formal description method for big data platform risks provided by an embodiment of the present invention. As an example and not a limitation, this method can be a specific possible implementation of the above description method, which may include steps S701-S705, each of which will be described below.

[0091] S701 defines the standard security domains for the target big data platform.

[0092] In one possible implementation, see Figure 8 The process involves first identifying all security assessment metrics of the target big data platform, generating an initial security tree for the platform based on these metrics, and then calculating the embedding vector of the initial security tree based on the embedding vector of each leaf node. Next, the embedding vector of the initial security tree is used to match the standard security tree of the target big data platform in a security standard template library; finally, the standard security domain of the target big data platform is defined based on the standard security tree.

[0093] For example, the security assessment metrics for platform assets within the same initial security domain / standard security domain are the same.

[0094] In one example, the security standard template library can store multiple standard security domain templates constructed from security assessment metrics. Each line in the template library stores the standard code, standard name, standard metric, and standard metric embedding vector corresponding to a standard security domain template.

[0095] For example, natural language processing techniques can be used to identify a set of security evaluation criteria for a standard big data platform and extract these criteria as standard security domains. For each set of big data platform security evaluation criteria, a tree diagram is generated with the big data platform as the root node and the standard security domains as the leaf nodes. The embedding vectors of the leaf nodes are recursively calculated and merged starting from the root node, and the vector representation of the root node is updated. The final embedding vector of the root node is then used as the standard embedding vector of a standard security domain template.

[0096] In one example, similarly, when processing a target big data platform, the platform assets can be evaluated to generate a tree diagram (i.e., an initial security tree) with the target big data platform as the root node and each initial security domain as the leaf node. Then, starting from the root node, the embedding vector of the entire tree diagram can be recursively calculated.

[0097] In one example, the distance between the embedding vector of the initial security tree and the embedding vector of each standard security domain template in the security standard template library can be calculated, and the standard security domain template with the smallest distance can be selected as the standard security tree of the target big data platform.

[0098] In one example, the leaf nodes of the standard security tree can be used as standard security domains to divide the assets of the target big data platform into standard security domains.

[0099] For example, a standard security domain can include multiple platform assets. Each platform asset has corresponding security requirements and security strategies. Platform assets within the same standard security domain have similar security requirements and strategies.

[0100] For example, a standard security domain is unique, the intersection of any two sets of security domains is an empty set, and the union of all security domains constitutes the target big data platform.

[0101] S702, extract and generate audit logs within the standard security domain.

[0102] In one possible implementation, see Figure 9 Logs from various components and subsystems can be collected and stored synchronously using log collection tools within each standard security domain. The stored log data is then cleaned to obtain audit logs.

[0103] In one example, logs can be integrated and tagged with their source according to the platform's business processes, then imported into a log database for storage, while simultaneously generating a log index as the raw data for a formal description of risk.

[0104] In one example, all log entries can be retrieved, and the log data format can be standardized using an ETL data cleaning tool to remove missing values, outliers, and invalid or duplicate data. The log entries can be labeled with the component and security domain to which they belong based on their log source, which can be used to locate the source of risk later.

[0105] In one possible implementation, the component's configuration file can be extracted from the audit logs, and the input and output of data during data processing can be analyzed to locate the location of the attack, the type of risk, and the cause of the risk.

[0106] In one example, based on the general architecture and business processes of a big data platform, the data processing process can be divided into four types: data acquisition, data storage and synchronization, data processing, and data application. The components of the target big data platform can be divided according to the data processing process, and then the configuration files of the components can be extracted according to different file extensions.

[0107] For example, the data acquisition phase typically deploys components such as Logstash, Kafka, Flume, ETL tools, and SQL tools. The data storage phase typically deploys data synchronization components such as Zookeeper, database management software such as MySQL, HBase, Hive, Sqoop, and Cassandra, and file system components such as HDFS and GFS. The data processing phase typically deploys batch or stream processing components such as Spark, Storm, and MapReduce. The data application phase typically deploys components such as Solr and Elasticsearch.

[0108] For example, the file extensions of component configuration files are typically ".properties", ".conf", ".xml", "yaml", "cfg", or "sh". You can retrieve and extract files with these file extensions from the corresponding component file directory in the system file explorer to obtain the component's configuration file.

[0109] S703, Construct a risk source map.

[0110] S704, generating a complete risk traceability chain based on the risk traceability map.

[0111] S705 provides a formal description of the target big data platform based on a complete risk traceability chain.

[0112] For example, steps S701-S705 are the same as steps S101-S105 in the above method. For details, please refer to the description of steps S101-S105, which will not be repeated here.

[0113] As an example, during the execution of the above method, the electronic device can formally describe the parameters generated or defined in the process using the z language, so that these parameters can be read, stored and further analyzed by the computer, and the ambiguity or vagueness present in natural language can be avoided.

[0114] Figure 10 The diagram shown illustrates the structure of a formal description device for risks on a big data platform provided in an embodiment of the present invention. As an example and not a limitation, the device 1000 may include a global adaptive processing module 1010, a risk retrieval module 1020, a supplementary retrieval module 1030, and a formal description module 1040.

[0115] For example, the global adaptive processing module 1010 analyzes the audit logs in each standard security domain of the target big data platform to identify and extract system entities, relationships between entities, and entity information of the target big data platform, generating a risk tracing graph of the target big data platform. The platform assets within the same standard security domain have the same security assessment indicators. The risk tracing graph includes multiple entity nodes that correspond one-to-one with system entities. Each entity node has at least one directed edge connecting it to other entity nodes, which describes the relationship between two system entities. The risk retrieval module 1020 retrieves suspicious entities from the risk tracing graph. Based on the attack event information, it queries the risk tracing graph to obtain a risk propagation chain, and takes the intersection of risk-related entities and suspicious entities as the risk entity. The risk-related entity is an entity in the risk propagation chain. The supplementary retrieval module 1030 supplements the risk propagation chain based on components and data processing related to the risk entity to obtain a complete risk propagation chain. The formal description module 1040 formally describes the risks of the target big data platform based on the complete risk propagation chain.

[0116] Since the risk attribution map is automatically generated based on audit logs, it can be dynamically updated as the status of the target big data platform changes, without the need for manual conversion. Furthermore, querying the risk attribution map based on attack events to obtain the risk propagation chain, rather than manually querying the path, improves automation. Mapping the risk formal model through the risk propagation chain further enhances the objectivity of the analysis results compared to traditional manual risk formal model construction, thereby improving the accuracy of the formal description results.

[0117] Figure 11 The diagram shown is a structural schematic of an electronic device provided in an embodiment of the present invention. Figure 11 The illustrated electronic device 1100 may include: at least one processor 1110 ( Figure 11 The diagram shows only one processor, a memory 1120, and a computer program 1130 stored in the memory 1120 and executable on the at least one processor 1110, which, when executing the computer program 1130, implements the steps in any of the above-described method embodiments.

[0118] The electronic device 1100 may be a robot or other processing device capable of implementing the above methods. This embodiment of the invention does not impose any restrictions on the specific type of electronic device.

[0119] Those skilled in the art will understand that Figure 11This is merely an example of electronic device 1100 and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or combine certain components, or use different components. For example, the electronic device 1100 may also include input / output interfaces.

[0120] The processor 1110 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASTCs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0121] In some embodiments, the memory 1120 may be an internal storage unit, such as a hard disk or RAM. In other embodiments, the memory 1120 may be an external storage device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD), or a flash card. Furthermore, the memory 1120 may include both internal and external storage units. The memory 1120 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 1120 can also be used to temporarily store data that has been output or will be output.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0124] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0125] This invention provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0126] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0127] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0128] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

Claims

1. A formal description method for risks in big data platforms, characterized in that, include: The audit logs of each standard security domain of the target big data platform are analyzed to identify and extract the system entities, relationships between entities and entity information of the target big data platform, and generate a risk tracing graph of the target big data platform. The security assessment indicators of platform assets in the same standard security domain are the same. The risk tracing graph includes multiple entity nodes that correspond one-to-one with the system entities. Each entity node has at least one directed edge connected to other entity nodes. The directed edge is used to describe the relationship between two system entities. Suspicious entities were obtained by retrieving the risk tracing map; Based on the information of the attack event, the risk propagation chain is obtained by querying the risk tracing graph, and the intersection of the risk-related entities and the suspicious entities is taken as the risk entity, wherein the risk-related entities are the entities in the risk propagation chain; A complete risk propagation chain is obtained by supplementing the risk propagation chain with the components and data processing procedures related to the risk entity; The risks of the target big data platform are formally described based on the complete risk propagation chain. The step of supplementing the risk propagation chain with components and data processing procedures related to the risk entity to obtain a complete risk propagation chain includes: Add component nodes around the entity node corresponding to the risk entity, add configuration item nodes, security policy nodes and data processing process nodes around the component nodes, and add risk nodes around the data processing process nodes. The component nodes correspond one-to-one with the relevant components of the risk entity. The configuration item nodes and security policy nodes surrounding the component nodes are used to represent the configuration items and security policies of the relevant components, respectively. The data processing process nodes are used to represent the data processing process to which the relevant component belongs. The risk nodes are used to represent the type of risk that occurs when the data processing process to which the relevant component belongs is executed. Connect the risk entity to the component node, the component node to the configuration item node, the component node to the security policy node, the component node to the data processing process node, and the data processing process node to the risk node to complete the supplementation of the risk entity; The complete risk propagation chain is obtained after supplementing all risk entities; The formal description of the risks of the target big data platform based on the complete risk propagation chain includes: Based on the complete risk propagation chain mapping risk formal model, the risk element set of the target big data platform is obtained, wherein the risk formal description model is a diagram used to describe the relationship between one or more of the following risk elements: components, data processing process, threats, vulnerabilities, risks, security policies, and security incidents. The risks of the target big data platform and their impact are formally described based on the set of risk factors.

2. The method according to claim 1, characterized in that, Before analyzing the audit logs within each standard security domain of the target big data platform to identify and extract system entities, inter-entity relationships, and entity information of the target big data platform, and generating a risk tracing map of the target big data platform, the method further includes: Identify all security assessment indicators of the target big data platform, and extract the initial security tree of the target big data platform based on the security assessment indicators, wherein the initial security tree is a tree diagram with the initial security domain as the leaf node and the target big data platform as the root node; The embedding vector of the initial security tree is calculated based on the embedding vector of each leaf node in the initial security tree, wherein the embedding vector of the leaf node is the security evaluation index of the initial security domain corresponding to the leaf node. Match the target big data platform's standard security tree in the security standard template library based on the embedding vector of the initial security tree; The standard security domain of the target big data platform is obtained by dividing the target big data platform according to the standard security tree.

3. The method according to claim 1, characterized in that, The suspicious entity is a system entity that appears less frequently than a preset frequency threshold in the risk tracing map.

4. The method according to claim 1, characterized in that, The entity information includes the time range of the system entity and the standard security domain in which the system entity is located.

5. The method according to claim 4, characterized in that, The information about the attack event includes the attack time range; The step of querying the risk attribution graph based on attack event information to obtain the risk propagation chain includes: The risk tracing diagram queries system entities that perform suspicious behavior within the attack time range, wherein, after the system entity performs the suspicious behavior, the state of at least one system entity in the target big data platform will change or attempt to change. The risk propagation chain is obtained by searching the risk tracing graph with the entity performing the suspicious behavior as the center and the attack time range as the boundary.

6. A formal description device for risks of a big data platform, characterized in that, include: The global adaptive processing module is used to analyze the audit logs in each standard security domain of the target big data platform to identify and extract the system entities, relationships between entities, and entity information of the target big data platform, and generate a risk tracing graph of the target big data platform. The security assessment indicators of platform assets in the same standard security domain are the same. The risk tracing graph includes multiple entity nodes that correspond one-to-one with the system entities. Each entity node has at least one directed edge connected to other entity nodes. The directed edge is used to describe the relationship between two system entities. A risk retrieval module is used to retrieve suspicious entities from the risk attribution graph; query the risk attribution graph based on attack event information to obtain a risk propagation chain; and take the intersection of risk-related entities and the suspicious entities as the risk entity, wherein the risk-related entities are entities in the risk propagation chain; A supplementary retrieval module is used to supplement the risk propagation chain based on the components and data processing procedures related to the risk entity to obtain a complete risk propagation chain; A formal description module is used to formally describe the risks of the target big data platform based on the complete risk propagation chain. The supplementary retrieval module is specifically used for: Add component nodes around the entity node corresponding to the risk entity, add configuration item nodes, security policy nodes and data processing process nodes around the component nodes, and add risk nodes around the data processing process nodes. The component nodes correspond one-to-one with the relevant components of the risk entity. The configuration item nodes and security policy nodes surrounding the component nodes are used to represent the configuration items and security policies of the relevant components, respectively. The data processing process nodes are used to represent the data processing process to which the relevant component belongs. The risk nodes are used to represent the type of risk that occurs when the data processing process to which the relevant component belongs is executed. Connect the risk entity to the component node, the component node to the configuration item node, the component node to the security policy node, the component node to the data processing process node, and the data processing process node to the risk node to complete the supplementation of the risk entity; The complete risk propagation chain is obtained after supplementing all risk entities; The formal description module is specifically used for: Based on the complete risk propagation chain mapping risk formal model, the risk element set of the target big data platform is obtained, wherein the risk formal description model is a diagram used to describe the relationship between one or more of the following risk elements: components, data processing process, threats, vulnerabilities, risks, security policies, and security incidents. The risks of the target big data platform and their impact are formally described based on the set of risk factors.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by an electronic device, it implements the method as described in any one of claims 1-5.

Images