Method for monitoring security degree of big data platform

Abstract

The invention discloses a method for monitoring the security degree of a big data platform. The method is used for comprehensively and effectively monitoring the overall security degree of the big data platform. The scheme provided by the invention comprises the following steps: acquiring component information in a to-be-monitored big data platform, wherein the component information comprises port information of a plurality of functional components in the big data platform; performing security monitoring on ports of the plurality of functional components based on the component information to obtain port monitoring data corresponding to the plurality of functional components; determining the risk degree of a preset security domain corresponding to each functional component based on the port monitoring data; determining the safety degree of each functional component based on the risk degree of the preset safety domain; and determining the security degree of the big data platform based on the security degree of each functional component.

Description

Technical Field

[0001] The present application relates to the field of data security, and in particular to a method for monitoring the security level of a big data platform. Background Art

[0002] With the explosive growth of data, many industries have built big data platforms. In order to improve the data security in the big data platform, security monitoring of the big data platform is often carried out through security strategies.

[0003] Some security policies are set up for specific risk channels to monitor known risks in a targeted manner, but the risk monitoring effect on unknown channels is poor, and the overall security monitoring effect of the big data platform is not good, which leads to potential risks to the data security of the big data platform.

[0004] How to comprehensively and effectively monitor the overall security level of a big data platform is the technical problem that this application aims to solve. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method for monitoring the security level of a big data platform, so as to comprehensively and effectively monitor the overall security level of the big data platform.

[0006] In a first aspect, a method for monitoring the security level of a big data platform is provided, comprising:

[0007] Obtaining component information in a big data platform to be monitored, wherein the component information includes port information of multiple functional components in the big data platform;

[0008] Perform security monitoring on the ports of the multiple functional components based on the component information to obtain port monitoring data corresponding to the multiple functional components respectively;

[0009] Determine the risk level of the preset security domain corresponding to each functional component based on the port monitoring data;

[0010] Determining the security level of each functional component based on the risk level of the preset security domain;

[0011] The security level of the big data platform is determined based on the security level of each functional component.

[0012] In a second aspect, a device for monitoring the security level of a big data platform is provided, comprising:

[0013] An acquisition module is used to acquire component information in a big data platform to be monitored, wherein the component information includes port information of multiple functional components in the big data platform;

[0014] A monitoring module, which performs security monitoring on the ports of the plurality of functional components based on the component information to obtain port monitoring data respectively corresponding to the plurality of functional components;

[0015] A first determination module, which determines the risk level of the preset security domain corresponding to each functional component based on the port monitoring data;

[0016] A second determination module determines the security level of each functional component based on the risk level of the preset security domain;

[0017] The third determination module determines the security level of the big data platform based on the security level of each functional component.

[0018] According to a third aspect, an electronic device is provided. The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method according to the first aspect are implemented.

[0019] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method according to the first aspect are implemented.

[0020] In a fifth aspect, a computer program product is provided, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to execute part or all of the steps of the method of the first aspect.

[0021] In an embodiment of the present application, first, component information in a big data platform to be monitored is obtained, wherein the component information includes port information of multiple functional components in the big data platform. Then, security monitoring is performed on the ports of multiple functional components based on the component information to obtain port monitoring data corresponding to the multiple functional components. Next, the risk level of the preset security domain corresponding to each functional component is determined based on the port monitoring data. Subsequently, the security level of each functional component is determined based on the risk level of the preset security domain. Finally, the security level of the big data platform is determined based on the security level of each functional component. This solution determines security layer by layer based on the functional component ports, security domains, and overall big data platform of the big data platform. Security monitoring based on the ports of functional components can ensure that the monitoring data is true and valid. Then, the risk level is judged from the perspective of the security domain based on the monitoring data, and the rationality of the determined security level of the functional components is improved. Then, the overall security level of the big data platform is determined based on the security level of each functional component, thereby achieving comprehensive and effective monitoring of the overall security level of the big data platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 This is one of the flowcharts of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0024] Figure 2 This is a second flow chart of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0025] Figure 3 This is a flowchart of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0026] Figure 4 This is a fourth flow chart of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0027] Figure 5 This is a flowchart of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0028] Figure 6 This is the sixth flow chart of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0029] Figure 7 This is the seventh flow chart of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0030] Figure 8 This is a flowchart of a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0031] Figure 9a It is a schematic diagram of the structure of a consciousness cyber-physical system of a method for monitoring the security level of a big data platform using an embodiment of the present application;

[0032] Figure 9b It is a schematic diagram of the attack hypothesis matrix of the conscious cyber-physical system;

[0033] Fig.9c It is a 14-aspect matrix diagram of the attack hypothesis matrix of the consciousness cyber-physical system;

[0034] Fig.10 It is a structural diagram of a big data platform using a method for monitoring the security level of a big data platform according to an embodiment of the present application;

[0035] Fig.11It is a schematic diagram of the structure of a device for monitoring the security level of a big data platform according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. The numbering of the drawings in this application is only used to distinguish the various steps in the scheme, and is not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0037] In the field of data security, the data volume of big data platforms has shown explosive growth, and the types of data are increasing, which makes it difficult to effectively implement security monitoring of big data platforms. In addition, big data platforms have many business levels, and the data and conditions that need to be monitored are complex. Single security monitoring is discrete and isolated, making it difficult to achieve effective and comprehensive monitoring of big data platforms, and there are potential risks.

[0038] In order to solve the problems existing in the related art, the embodiment of the present application provides a method for monitoring the security level of a big data platform, such as Figure 1 As shown, the following steps are included:

[0039] S11: Obtain component information in the big data platform to be monitored, where the component information includes port information of multiple functional components in the big data platform.

[0040] The big data platform may be, for example, a cloud host platform.

[0041] In this step, component information is obtained for the big data platform to be monitored, and the component information is, for example, a platform component list, which includes information of multiple functional components included in the big data platform, specifically including port information of each functional component. In addition, it can also include connection relationships or data service relationships between multiple functional components.

[0042] In actual application scenarios, each functional component can be used to implement data collection, data storage HDFS (Hadoop Distributed File System), distributed computing, real-time computing, and big data applications hosted by the big data platform. Among them, the port information of different functional components may be of one or more types. For example, functional components can be divided into multiple functional types such as input components, processing components, and output components. Each component can include one or more port information based on the connection relationship, and the port information is used to monitor the port for transmitting data.

[0043] S12: Perform security monitoring on the ports of the multiple functional components based on the component information to obtain port monitoring data corresponding to the multiple functional components respectively.

[0044] In this step, security monitoring is performed on the ports of each functional component. During the monitoring process, targeted monitoring can be performed on the component functions based on the component information. For example, data source security monitoring is performed on the input component, processing effectiveness monitoring is performed on the processing component, and data destination security monitoring is performed on the output component. The monitoring rules for each functional component can be pre-set or automatically generated based on historical alarm logs and other information.

[0045] The data monitored by each functional component port can be stored based on the functional component information or the functional component port identifier. When judging the risk later, it can be quickly and effectively called based on the functional component information or the port identifier to improve the overall efficiency of risk monitoring.

[0046] S13: Determine the risk level of the preset security domain corresponding to each functional component based on the port monitoring data.

[0047] The preset security domain can be pre-set based on the actual functions of each functional component, and one functional component can correspond to one or more preset security domains. At least one security domain set for a functional component can be stored in the security domain table in correspondence with the corresponding functional component identifier.

[0048] Optionally, for any security domain, a risk assessment method and a corresponding risk degree may be set, and the risk degree may be expressed in the form of a risk level or a numerical value.

[0049] In this step, for the port monitoring data, the risk level is determined using the risk assessment rules of the preset security domain corresponding to the corresponding functional component, thereby obtaining the risk level of one or more security domains corresponding to the functional component. The risk level of the security domain can indicate the risk level of the corresponding functional component in terms of the security domain.

[0050] S14: Determine the security level of each functional component based on the risk level of the preset security domain.

[0051] In this step, the risk level of at least one security domain corresponding to any functional component is summarized to determine the security level of the functional component. The security level is used to represent the risk faced by the corresponding functional component in the corresponding at least one security domain.

[0052] The security level is negatively correlated with the risk level, that is, the higher the risk level of the security domain corresponding to the functional component, the lower the security level of the corresponding functional component. The above security level can be expressed in the form of security level or value.

[0053] In this step, the aggregation rules for determining the security level of the functional component based on the risk level of the security domain can be pre-set, and the aggregation rules corresponding to different functional components can be different. For example, for a functional component, the security level of the functional component is determined by the highest risk level in each security domain in the functional component. Alternatively, the average risk level of each security domain in the functional component can be counted, and then the security level of the functional component is determined using the average risk level.

[0054] After determining the security level of each functional component, the identifier of each functional component and the security level may be stored in correspondence, so that the corresponding security level can be quickly called based on the identifier of each functional component in subsequent steps.

[0055] S15: Determine the security level of the big data platform based on the security level of each functional component.

[0056] In this step, the overall security level of the big data platform is determined based on the security level of each functional component in the big data platform. The security level is used to represent the risks faced by each functional component of the big data platform. The above security level can be expressed in the form of security level or value.

[0057] In this step, the summary rules for determining the security level of the big data platform based on the security level of the functional components can be pre-set. For example, the lowest security level among the functional components can be determined as the security level of the big data platform. Alternatively, the average security level of each functional component can be calculated and the average security level can be used as the overall security level of the big data platform.

[0058] According to the scheme provided by the embodiment of the present application, first, the component information in the big data platform to be monitored is obtained, wherein the component information includes the port information of multiple functional components in the big data platform. Then, security monitoring is performed on the ports of the multiple functional components based on the component information to obtain the port monitoring data corresponding to the multiple functional components. Next, the risk level of the preset security domain corresponding to each functional component is determined based on the port monitoring data. Subsequently, the security level of each functional component is determined based on the risk level of the preset security domain. Finally, the security level of the big data platform is determined based on the security level of each functional component. This scheme determines security layer by layer based on the functional component ports, security domains, and the overall big data platform of the big data platform. Security monitoring based on the ports of the functional components can ensure that the monitoring data is true and valid. Then, the risk level is judged from the perspective of the security domain based on the monitoring data, and the rationality of the determined security level of the functional components is improved. Then, the overall security level of the big data platform is determined based on the security level of each functional component, thereby achieving comprehensive and effective monitoring of the overall security level of the big data platform.

[0059] Based on the solution provided in the above embodiment, optionally, the component information further includes a platform risk factor corresponding to each of the functional components, and the platform risk factor indicates the degree of influence of the corresponding functional component on the security risk caused by the big data platform;

[0060] Among them, Figure 2 As shown, in the above step S15, the security level of the big data platform is determined based on the security level of each functional component, including:

[0061] S21: Determine the risk impact of the first functional component on the big data platform based on the security level of the first functional component and the platform risk factor corresponding to the first functional component, wherein the first functional component is any one of the multiple functional components in the big data platform.

[0062] Among them, the platform risk factor can also be called the platform security impact factor, which is used to indicate the importance of the risk faced by the corresponding functional component to the risk faced by the big data platform as a whole. From the perspective of risk monitoring, the platform risk factor can be used to indicate the importance of risk monitoring of the corresponding functional component in the big data platform.

[0063] In actual applications, the platform risk factors corresponding to different functional components are affected by the functions themselves. For example, for the input module and processing module in the big data platform, the input module can be used to obtain data from the data source, and the processing module can be used to process the acquired data. Data input from the outside may bring input risks, so the platform risk factor of the input module can be greater than the platform risk factor of the processing module.

[0064] In some cases, the platform risk factor of the processing module may also be greater than the platform risk factor of the input module. The platform risk factor of any functional module may be related to the function actually performed by the functional module.

[0065] In this step, the degree of risk impact on the big data platform is comprehensively determined based on the security level of the first functional component and the platform risk factor. The security level of the first functional component is used to indicate whether the first functional component itself faces risks, and the platform risk factor of the first functional component is used to indicate the degree of impact of the risks faced by the first functional component on the overall big data platform. Based on the above security level and platform risk factor, the impact of the risks faced by the first functional component on the overall risk of the big data platform can be comprehensively determined.

[0066] Through this step, the risk impact of each functional component in the big data platform on the big data platform as a whole can be determined respectively. Among them, the greater the risk impact, the higher the risk faced by the big data platform caused by the functional component.

[0067] S22: Determine the security level of the big data platform according to the risk impact levels of the multiple functional components on the big data platform.

[0068] In this step, the risk impact of multiple functional components on the big data platform is summarized and the overall security level of the big data platform is comprehensively determined.

[0069] For example, the maximum value can be determined from the risk impact levels corresponding to multiple functional components, and then the security level of the big data platform can be determined based on the maximum value. That is, the item that poses the greatest threat to the security level of the big data platform can be determined from multiple functional components, and then the maximum risk faced by the big data platform can be evaluated to determine the overall security level of the big data platform based on the maximum risk. Alternatively, the overall security level of the big data platform can be comprehensively determined based on the average value of the risk impact levels corresponding to multiple functional components or other statistical data.

[0070] In this step, multiple functional components can be selected based on computing power to determine the security level of the big data platform. If the computing power is sufficient, the risk impact level corresponding to all functional components in the big data platform is used to determine the security level of the big data platform. If the computing power is limited, the risk impact levels corresponding to multiple functional components can be screened first, and a preset number of items with a larger risk impact level can be determined. Then, the risk impact levels corresponding to the preset number of functional components determined can be comprehensively counted to calculate the overall security level of the big data platform.

[0071] This solution jointly determines the security level of the big data platform by comprehensively considering the security levels of multiple functional components in the big data platform. It can improve the objective effectiveness of the determined security level of the big data platform and comprehensively determine the risks faced by the big data platform as a whole based on multiple functional components.

[0072] Based on the solution provided in the above embodiment, optionally, the component information also includes the identifier of the preset security domain corresponding to each of the functional components, and the component risk factor corresponding to the identifier of each of the preset security domains, and the component risk factor indicates the degree of influence of the corresponding preset security domain on the security risk caused by the functional component to which it belongs.

[0073] Among them, any functional component corresponds to at least one preset security domain, and the corresponding relationship can be stored in the security domain attribution table through the functional component identifier and the preset security domain identifier. Among them, the component risk factor corresponding to any preset security domain identifier is used to indicate the degree of influence of the preset security domain on the security risk of the functional component to which it belongs. The component risk factor can be pre-set according to actual needs or automatically generated according to historical alarm logs.

[0074] Among them, Figure 3 As shown, in the above step S14, the security level of each functional component is determined based on the risk level of the preset security domain, including:

[0075] S31: Determine the security level of at least one security domain corresponding to the first functional component based on the port monitoring data of the first functional component.

[0076] In this step, the security level of each security domain of the corresponding functional component is determined based on the port monitoring data. The security domain can correspond to the security category involved in the big data platform, such as transmission security, data security, system security, application security, network security, etc.

[0077] Through this step, the security level can be determined for each security domain of the functional component. The security level corresponding to the security domain and the functional component to which the security domain belongs can be associated and stored in a list for flexible calling in subsequent steps.

[0078] S32: Determine the risk impact of the first security domain on the first functional component based on the security level of the first security domain and the component risk factor corresponding to the first security domain, wherein the first security domain is any one of the at least one security domain corresponding to the first functional component.

[0079] In this step, the risk impact degree of each security domain in the first functional component is determined respectively. The risk impact degree is determined by the security level of the security domain itself and the component risk factor of the security domain, and can comprehensively represent the comprehensive risk caused by the security domain to the functional component. Among them, the greater the risk impact degree, the higher the risk faced by the first functional component caused by the security domain.

[0080] Through this step, the corresponding risk impact levels can be determined for multiple security domains in a functional component, and the risk impact levels corresponding to the security domains are associated with the security domain identifiers and stored in a list for flexible calling in subsequent steps.

[0081] S33: Determine the security level of the first functional component according to the risk impact level of the at least one security domain on the first functional component.

[0082] In this step, the risk impact degree of at least one security domain on the first functional component is summarized, and the security degree of the first functional component is comprehensively determined.

[0083] Optionally, a maximum value is determined from the risk impact levels corresponding to at least one security domain, and then the security level of the first functional component is determined based on the maximum value. That is, an item that poses the greatest threat to the security level of the first functional component is determined from the security domain, and then the maximum risk faced by the first security component is evaluated to determine the security level of the first functional component based on the maximum risk. Alternatively, the security level of the first functional component can also be comprehensively determined based on the average value of the risk impact levels corresponding to each security domain or other statistical data.

[0084] This solution determines the security level of the functional component by combining the security level of at least one security domain in the functional component, which can improve the objective effectiveness of the determined security level of the functional component, and further comprehensively determine the risks faced by the big data platform as a whole based on multiple functional components.

[0085] Based on the solution provided in the above embodiment, optionally, the component information also includes an identifier of a security indicator service item corresponding to a preset security domain.

[0086] Among them, the correspondence between security indicator service items and security domains can be stored in the security domain attribution table. The security indicator service items can be set based on the corresponding security domains and can be used for segmented monitoring of security domains, such as data encryption, sensitive information desensitization, data backup and recovery of data security domains, etc.

[0087] For example, for the data collection security domain, the corresponding security indicator service items may include data collection password complexity, password replacement strategy, data virus detection, etc. These security indicator service items can be used to achieve a comprehensive and effective security assessment of the corresponding security domain, thereby improving the accuracy of risk determination.

[0088] Among them, Figure 4 As shown, in the above step S31, determining the security level of at least one security domain corresponding to the first functional component based on the port monitoring data of the first functional component includes:

[0089] S41: Determine the security level of at least one security indicator service item corresponding to the first security domain based on the port monitoring data in the first functional component.

[0090] In this step, the security level of each security indicator service item of each security domain of the corresponding functional component is determined based on the port monitoring data. Through this step, the security level of each security indicator service item can be determined separately, and then the security level of the security domain, functional component and the big data platform as a whole can be determined layer by layer through the association relationship.

[0091] Optionally, the security level of the security indicator service item and the security domain to which it belongs can be associated and stored in a list for flexible calling in subsequent steps.

[0092] S42: Determine the security level of the first security domain according to the security level of at least one security indicator service item of the first security domain.

[0093] In this step, various security indicator service items in the first security domain are summarized to comprehensively determine the security level of the first security domain.

[0094] Optionally, a minimum value is determined from the security levels corresponding to at least one security indicator service item, and then the security level of the first security domain is determined based on the minimum value. That is, an item that poses the greatest threat to the security level of the first security domain is determined from at least one security indicator service item, and then the maximum risk faced by the first security domain is evaluated to determine the security level of the first security domain based on the maximum risk. Alternatively, the security level of the first security domain can also be comprehensively determined based on the average value of the security levels corresponding to the various security indicator service items or other statistical data.

[0095] This solution determines the security level of the security domain by combining the security level of at least one security indicator service item in the security domain. It can improve the objective effectiveness of the security level of the determined security domain, and then determine the risks faced by the big data platform as a whole layer by layer based on multiple security domains and multiple functional components.

[0096] Based on the solution provided in the above embodiment, optionally, Figure 5As shown, in the above step S21, determining the risk impact of the first functional component on the big data platform based on the security level of the first functional component and the platform risk factor corresponding to the first functional component includes:

[0097] S51: Based on the security level of the first functional component and the platform risk factor corresponding to the first functional component, determine the risk impact level of the first functional component on the big data platform through a platform risk impact level formula, wherein the platform risk impact level formula is:

[0098] The risk impact degree of the functional component on the big data platform = (1-security degree of the functional component) × platform risk factor.

[0099] For example, assuming that the first functional component is a data security collection component, assuming that the security level of the data security collection component is 36%, and the platform risk factor of the data security collection component is 80%, then the risk impact level of the data security collection component on the big data platform can be determined to be (1-36%)×80%=51.2% through the platform risk impact level formula of this step.

[0100] Through this step, the risk impact degree of each of the multiple functional components included in the big data platform can be determined, providing a data basis for subsequent comprehensive statistics on the overall security level of the big data platform.

[0101] Among them, in the above step S22, determining the security level of the big data platform according to the risk impact levels of the multiple functional components on the big data platform respectively includes:

[0102] S52: Determine the maximum risk impact of the multiple functional components on the big data platform as the security level of the big data platform.

[0103] In this step, the largest item is determined based on the risk impact of multiple functional components on the big data platform, and the largest item indicates that the risk faced by the corresponding functional component has the greatest impact on the risk faced by the big data platform as a whole. Then, the security level of the big data platform is determined based on the largest item.

[0104] Through the solution provided in the embodiment of the present application, the risk impact degree corresponding to the functional components can be effectively quantified through the platform risk impact degree formula, and the risks faced by multiple functional components can be effectively evaluated in numerical form, thereby more reasonably determining the overall security level of the big data platform.

[0105] Based on the solution provided in the above embodiment, optionally, in the above step S32, determining the risk impact of the first security domain on the first functional component based on the security level of the first security domain and the component risk factor corresponding to the first security domain includes:

[0106] Based on the security level of the first security domain and the component risk factor corresponding to the first security domain, the risk impact level of the first security domain on the first functional component is determined by a component risk impact level formula, wherein the component risk impact level formula is:

[0107] The risk impact of a security domain on its functional components = (1-security level of the security domain) × component risk factor.

[0108] For example, assuming that the security level of the first security domain is 50%, and the component risk factor of the first security domain is 50%, then the component risk impact level of the first security domain is (1-50%)×50%=25%.

[0109] Through this step, the risk impact degree of at least one security domain included in the functional component can be determined separately, providing a data basis for the subsequent comprehensive statistics of the overall security level of the big data platform.

[0110] Based on the solution provided in the above embodiment, optionally, Figure 6 As shown, in the above step S33, determining the security level of the first functional component according to the risk impact level of the at least one security domain on the first functional component respectively includes:

[0111] S61: Determine the maximum risk impact degree of a component from the risk impact degrees of the at least one security domain on the first functional component.

[0112] For example, assuming that the first functional component includes three security domains, and the risk impact levels are 25%, 20%, and 64% respectively, a maximum value of 64% is determined through this step, which indicates the maximum value of the risk faced by the first functional component caused by the corresponding security domain.

[0113] S62: Determine the safety level of the first functional component by using a functional component safety level formula, wherein the functional component safety level formula is:

[0114] Functional component safety level = 1-maximum component risk impact level.

[0115] Based on the maximum value of 64% of the risk impact degree of the above-mentioned component, the functional component safety degree is determined by the functional component safety degree formula of this step to be 1-64%=36%.

[0116] Through this step, the risk impact degree of the components corresponding to at least one security domain can be determined, and then the maximum risk faced by the functional components can be determined, so as to objectively determine the security level of the functional components and provide a data basis for the subsequent comprehensive statistics of the overall security level of the big data platform.

[0117] Based on the solution provided in the above embodiment, optionally, Figure 7 As shown, the component information also includes a security domain risk factor corresponding to each of the security indicator service items, and the security domain risk factor indicates the degree of influence of the corresponding security indicator service item on the security risk caused by the preset security domain to which it belongs;

[0118] Among them, in the above step S41, determining the security level of at least one security indicator service item corresponding to the first security domain based on the port monitoring data in the first functional component includes:

[0119] S71: Determine a security value coefficient corresponding to a first security indicator service item based on port monitoring data in a first functional component, wherein the first security indicator service item is any one of at least one security indicator service item corresponding to the first security domain.

[0120] In this step, the security value coefficient corresponding to the security indicator service item is determined according to the port monitoring data. The port monitoring data can be tested according to the detection rules corresponding to the security indicator service item to determine the security value coefficient. For example, if the port monitoring data indicates that the status of the system vulnerability monitoring is that there are 3 vulnerabilities, the security value coefficient of the security indicator service item of the system vulnerability monitoring can be 1, and if there is no risk, the security value coefficient can be 0.

[0121] S72: Based on the security domain risk factor and the security value coefficient corresponding to the first security indicator service item, determine the security level of the first security indicator service item through an indicator security level formula, wherein the indicator security level formula is:

[0122] The safety level of the safety indicator service item = (1-safety domain risk factor) × safety value coefficient.

[0123] For example, assuming that the security domain risk factor of the security indicator service item of the system vulnerability monitoring is 50%, then the security level of the system vulnerability monitoring is (1-50%)×1=50%.

[0124] Wherein, in the above step S42, determining the security level of the first security domain according to the security level of at least one security indicator service item of the first security domain includes:

[0125] S73: Determine the minimum security level of the at least one security indicator service item as the security level of the first security domain.

[0126] In this step, the security level of the first security domain is comprehensively determined based on at least one security indicator service item, and the smallest item is selected from the security levels of the security indicator service items. The smallest item indicates the security indicator service item facing the greatest risk, and also indicates the maximum risk level faced by the first security domain.

[0127] For example, assuming that the system security domain includes 4 security indicator service items, and the security levels are 100%, 100%, 100%, and 50%, respectively, then in this step, the smallest item is selected as the security level of the system security domain: min{100%, 100%, 100%, 50%}=50%.

[0128] Through the solution provided in the embodiment of the present application, the security level of the security domain is determined through security indicator service items based on port monitoring data, and the security level is expressed in numerical form through a preset formula, which can effectively improve the objectivity and effectiveness of the security level and provide a data basis for subsequent comprehensive statistical analysis of the overall security level of the big data platform.

[0129] Based on the solution provided in the above embodiment, optionally, in the above step S12, security monitoring is performed on the ports of the multiple functional components based on the component information to obtain port monitoring data corresponding to the multiple functional components, including:

[0130] Based on the component information, security detection is periodically performed on the ports of the multiple functional components according to preset detection rules to obtain port monitoring data corresponding to the multiple functional components, wherein the preset detection rules include uniqueness detection and / or compliance detection.

[0131] Among them, the detection rules can be flexibly set according to actual needs, and the detection rules of different functional components can be different. By periodically performing security detection, the timeliness of the acquired port monitoring data can be improved to improve the overall timeliness of risk monitoring. When the big data platform faces risks, it can effectively discover risks in real time, which is conducive to risk disposal in the first place.

[0132] Among them, uniqueness detection can be used to achieve data consistency and integrity verification, and compliance detection can be verified based on preset rules to determine whether the data is compliant.

[0133] The present invention is further described below in conjunction with the embodiments. Figure 8 .

[0134] The solution provided in the embodiment of the present application can be implemented by the following steps:

[0135] Step 1: Perform preprocessing in the big data platform to implement platform security assessment later. For example, configure the following items in the big data platform:

[0136] 1) Platform component list, which is used to store the identification and connection relationship of each functional component of the big data platform.

[0137] 2) Security domain table, used to store information such as security categories or detection rules corresponding to each security domain.

[0138] 3) Data security level list, which is used to store the security level of data and can implement risk assessment based on security level.

[0139] 4) Domain attribution table, used to store the relationship between functional components and security domains. A component may include one or more security domains, which can also store the relationship between security indicator service items and security domains.

[0140] 5) A security index table, which includes the security value coefficients of security index service items, and may also store the relationship between security index service items and security domains.

[0141] 6) Comprehensive security assessment table, which is used to store the risk assessment results at each level in the big data platform, and can be stored in the form of "identifier-value".

[0142] 7) Log manager, used to store and manage various logs.

[0143] Among them, the platform component table can be used to define the various parts that make up the big data platform, such as data acquisition, data storage HDFS, distributed computing, real-time computing, and big data applications carried by the big data platform. The security domain table can be used to define various security-related categories, such as transmission security, data security, system security, application security, and network security. The domain attribution table can be used to define the relationship between big data components and security domains, that is, the security measurement of big data components is calculated from the security of one or more security domains. The security indicator table can be used to monitor security domains in detail, such as data security domains including data encryption, sensitive information desensitization, and data backup and recovery.

[0144] Step 2: Import the key information of the information interfaces of each component of the big data platform (including input components, processing components, output components, etc.) into the above-mentioned configured platform component list, and store the key security information into the security domain table so that the required information can be flexibly called by subsequent processing steps.

[0145] Step 3: Perform security, uniqueness, compliance and other tests on the input, processing and output information of each component, and import the test results into the above-configured security indicator table. This step can be performed periodically based on preset rules to ensure the timeliness of data acquisition.

[0146] Step 4: Obtain the domain ID and component security impact factor from the domain ownership table according to the component ID, evaluate the factor, and import it into the above-configured comprehensive security evaluation table.

[0147] Step 5: Circularly process each set of domain identifiers and component security impact factors to provide a data basis for subsequent determination of the component's security level.

[0148] Step 6: Obtain all indicator service items and domain security impact factors from the indicator data collection service item table according to the security domain identifier, and calculate the security level of each indicator in the domain. Among them, the security level of a single indicator = (1-domain security impact factor) * security value coefficient. The security value coefficient can be obtained from the security indicator table to indicate whether the risk represented by the security indicator is detected from the component port monitoring data.

[0149] Step 7: Calculate the security level of a single security domain. The security level of a single domain is the minimum value of the security levels of all indicators corresponding to the domain, so as to determine the item facing the greatest risk.

[0150] Step 8: Calculate the component security impact of the domain according to the component security impact factor of each domain. The component security impact of a single domain = (1-security of a single domain) × component security impact factor.

[0151] After all domains and domain component security impact factor groups have been processed, proceed to step nine, otherwise repeat step five.

[0152] Step 9: Calculate the safety level of the component, where the safety level of the component = 1-the maximum value of the safety impact of the components in all domains.

[0153] Step 10: Calculate the platform security impact of the component based on the platform security impact factor of each component. The platform security impact of a single component = (1-security of a single component) × platform security impact factor.

[0154] If all components are processed, proceed to step 11, otherwise repeat step 3.

[0155] Step 11: Calculate the platform security level according to the platform security impact of each component. Platform security level = 1 - the maximum value of the platform security impact of all components.

[0156] Through the solution provided by the embodiment of the present application, the security is determined layer by layer based on the functional component ports, security domains, and overall big data platform of the big data platform. Security monitoring based on the ports of the functional components can ensure that the monitoring data is true and valid. Then, based on the monitoring data, the risk level is judged from the perspective of the security domain, and the rationality of the determined functional component security level is improved. Then, based on the security level of each functional component, the overall security level of the big data platform is determined, thereby achieving comprehensive and effective monitoring of the overall security level of the big data platform.

[0157] Based on the solution provided in the above embodiment, optionally, the preset detection rules include a multidimensional risk matrix, the dimensions of the multidimensional risk matrix include an attack target dimension and an attack surface dimension, and the value of any element in the multidimensional risk matrix represents the risk level caused to the big data platform by the attack target and attack surface corresponding to the location of the element.

[0158] In the embodiment of the present application, the threats faced by the entire big data platform are integrated into a multidimensional matrix (for example, a 3*3 matrix), and the matrix threats faced include, for example, attack surfaces and attack targets. The attack surface refers to the attacker's starting point and attack mode; the attack target refers to a part or link in the attacked system that the attacker hopes to have a major deviation. The attack surface and the attack target are not always the same.

[0159] Mobile Cyber-Physical Systems (MCPs) such as Figure 9a As shown in Figure 1. In a system attack, the attack surface and the attack target may be different. This misalignment between the attack surface and the attack target may occur in three aspects of MCPs, formed by the cross-attack assumptions of consciousness space, cyberspace, and physical space (real world). Figure 9b The 3x3 attack hypothesis matrix shown.

[0160] In order to improve the effectiveness of system security monitoring, MCPs can be decomposed into more detailed links in this solution. For example, MCPs can be simply decomposed into 14 aspects, including: Mm: motivation, Mv: value, Ms: semantics, Cd: data and data flow, Cm: metadata and pure data, Ca: application and business flow, Cc: computing node, Cs: storage node, Cn: network and network flow, Cp: Cyber ​​physical entity, Pc: controller, Ps: sensor, PS: spatial relationship, PT: temporal relationship.

[0161] The 14 aspects of MSPs are organized into a matrix, where different rows of the matrix represent different attack surfaces, and different columns of the matrix represent different attack targets, such as Fig.9c shown.

[0162] The following is a detailed description of the implementation process of this solution with examples. Fig.10 As shown in the figure, the security of the big data platform depends on the security impact of the three components of the platform: "data collection component", "data analysis and processing component", and "data output", and each component includes different devices.

[0163] Among them, the data collection component mainly includes two parts: collection and preprocessing. Collection includes file collection, database collection, data flow collection and other methods and collection tools. The password complexity, password change strategy and virus detection in the collection system are normal, so the security of these three items (security indicator service items) = 1-0% = 100%, the status of system vulnerability monitoring is that there are 3 vulnerabilities, and its security = 1-50% = 50%; the security of the system security domain = min{100%, 100%, 100%, 50%} = 50%, and the security impact of the system security domain on the component = (1-50%) × 50% = 25%.

[0164] The status of intrusion detection and network isolation in network security is normal, so the security of these two security indicator service items = 1-0% = 100%, the status of the firewall is abnormal, and its security = 1-40% = 60%; the security of the network security domain = min{100%, 100%, 60%} = 60%, the security impact of the network security domain on the component = (1-60%) × 50% = 20%.

[0165] The status of data encryption in data security is abnormal, and the security degree of this security indicator service item = 1-80% = 20%. The status of data desensitization and data backup is normal. The security degree of these two security indicator service items = 1-0% = 100. The security degree of the data security domain = min{20%, 100%, 100%} = 20%. The security impact of the network security domain on the component = (1-20%) × 80% = 64%.

[0166] Then, the security level of the data acquisition component is determined based on the security levels of multiple security domains = 1-max (25%, 20%, 64%) = 1-64% = 36%.

[0167] Furthermore, based on the security level of the functional components, the platform security impact of the data acquisition component (the degree of influence of a component on the overall security of the big data platform) is determined = (1-36%) × 80% = 51.2%.

[0168] Through the solution provided in the embodiments of the present application, the security level of the big data platform is determined by the security level of the components included, the security level of a component is determined by the security domains included, and the security level of a security domain is determined by the security indicator service items included, thereby achieving layer-by-layer determination of security level and improving the effectiveness of the overall security level of the big data platform that is ultimately determined.

[0169] In the scheme provided by the embodiment of the present application, the key component information is read sequentially, and security identification can be realized for the basic domain. All domain identifiers and component security impact factors are obtained from the component and security domain relationship table according to the component identifier, and then all indicator service items and domain security impact factor lists are obtained from the security indicator service item list according to the domain identifier, and the security of each indicator service item of the domain is calculated. Subsequently, the component impact of the domain is calculated according to the component security impact factor of each domain, and then the security of each component is calculated. Next, the platform impact of the component is calculated according to the platform security impact factor of each component, and finally the security of the big data platform is calculated, which can objectively and effectively determine the security level layer by layer, thereby improving the reasonable effectiveness of the security level of the final big data platform. This scheme is suitable for the needs of security management in the big data environment, and can facilitate security administrators to flexibly configure multi-level monitoring rules or associations, such as the relationship between the big data platform and the platform components, and the corresponding relationship between the security domain and the security indicator service items. Thus, an association model for security analysis between the various levels of the platform is established, which has high program scalability and flexibility.

[0170] In order to solve the problems existing in the prior art, the embodiment of the present application also provides a device 110 for monitoring the security level of a big data platform, such as Fig.11 As shown, including:

[0171] An acquisition module 111 is configured to acquire component information in a big data platform to be monitored, wherein the component information includes port information of multiple functional components in the big data platform;

[0172] A monitoring module 112 performs security monitoring on the ports of the plurality of functional components based on the component information to obtain port monitoring data corresponding to the plurality of functional components respectively;

[0173] A first determination module 113 determines the risk level of a preset security domain corresponding to each functional component based on the port monitoring data;

[0174] A second determination module 114 determines the security level of each functional component based on the risk level of the preset security domain;

[0175] The third determination module 115 determines the security level of the big data platform based on the security level of each functional component.

[0176] Through the device provided by the embodiment of the present application, first, the component information in the big data platform to be monitored is obtained, wherein the component information includes the port information of multiple functional components in the big data platform. Then, security monitoring is performed on the ports of the multiple functional components based on the component information to obtain the port monitoring data corresponding to the multiple functional components. Next, the risk level of the preset security domain corresponding to each functional component is determined based on the port monitoring data. Subsequently, the security level of each functional component is determined based on the risk level of the preset security domain. Finally, the security level of the big data platform is determined based on the security level of each functional component. This scheme determines the security layer by layer based on the functional component ports, security domains, and the overall big data platform of the big data platform. Security monitoring based on the ports of the functional components can ensure that the monitoring data is true and valid. Then, the risk level is judged from the perspective of the security domain based on the monitoring data, and the rationality of the determined security level of the functional components is improved. Then, the overall security level of the big data platform is determined based on the security level of each functional component, thereby achieving comprehensive and effective monitoring of the overall security level of the big data platform.

[0177] The key point of this application proposal is to propose a safety factor model based on the functional blocks of the big data platform. By evaluating the functional domain of the big data platform according to the security model, the overall safety factor of the big data platform is obtained, and the safety factor of the big data platform can be evaluated for different applications. Therefore, the entire model and the methods included are the key points and protection points of this proposal.

[0178] This application proposal creatively uses different functional partition functions to calculate the risk of the big data platform. Compared with the existing technology, the existing technology mainly calculates the risk based on the vulnerability of the technical module, and cannot focus on the risk of a certain functional block, and cannot associate the security risk coefficient of the specific application in the big data platform with the big data platform. Therefore, this technology solves the overall technical assessment risk of the entire big data platform.

[0179] Among them, the above modules in the device provided by the embodiment of the present application can also implement the method steps provided by the above method embodiment. Alternatively, the device provided by the embodiment of the present application can also include other modules in addition to the above modules to implement the method steps provided by the above method embodiment. And the device provided by the embodiment of the present application can achieve the technical effects that can be achieved by the above method embodiment.

[0180] Preferably, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned method embodiment for monitoring the security level of a big data platform are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0181] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned method embodiment for monitoring the security level of a big data platform is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0182] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program can be operated to enable a computer to execute part or all of the steps of the above-mentioned method embodiment for monitoring the security level of a big data platform, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0183] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0184] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0187] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0188] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0189] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0190] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0191] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0192] The above is only the embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for monitoring the security level of a big data platform, characterized in that: include: Obtaining component information in a big data platform to be monitored, wherein the component information includes port information of multiple functional components in the big data platform; Perform security monitoring on the ports of the multiple functional components based on the component information to obtain port monitoring data corresponding to the multiple functional components respectively; Determine the risk level of the preset security domain corresponding to each functional component based on the port monitoring data; Determining the security level of each functional component based on the risk level of the preset security domain; The security level of the big data platform is determined based on the security level of each functional component.

2. The method according to claim 1, characterized in that The component information also includes a platform risk factor corresponding to each of the functional components, wherein the platform risk factor indicates the degree of impact of the corresponding functional component on the security risk of the big data platform; The security level of the big data platform is determined based on the security level of each functional component, including: Determining the risk impact of the first functional component on the big data platform based on the security level of the first functional component and the platform risk factor corresponding to the first functional component, wherein the first functional component is any one of multiple functional components in the big data platform; The security level of the big data platform is determined according to the risk impact levels of the multiple functional components on the big data platform.

3. The method according to claim 2, characterized in that The component information also includes an identifier of a preset security domain corresponding to each of the functional components, and a component risk factor corresponding to the identifier of each of the preset security domains, wherein the component risk factor indicates the degree of influence of the corresponding preset security domain on the security risk of the functional component to which it belongs; Wherein, determining the security level of each functional component based on the risk level of the preset security domain includes: Determining, based on the port monitoring data of the first functional component, a security level of at least one security domain corresponding to the first functional component; Determining a risk impact degree of the first security domain on the first functional component based on a security degree of the first security domain and a component risk factor corresponding to the first security domain, wherein the first security domain is any one of at least one security domain corresponding to the first functional component; The security level of the first functional component is determined according to the risk impact level of the at least one security domain on the first functional component.

4. The method according to claim 3, characterized in that The component information also includes an identifier of a security indicator service item corresponding to a preset security domain; Wherein, determining the security level of at least one security domain corresponding to the first functional component based on the port monitoring data of the first functional component includes: Determine the security level of at least one security indicator service item corresponding to the first security domain based on the port monitoring data in the first functional component; The security level of the first security domain is determined according to the security level of at least one security indicator service item of the first security domain.

5. The method according to claim 4, characterized in that Determining the risk impact of the first functional component on the big data platform based on the security level of the first functional component and the platform risk factor corresponding to the first functional component includes: Based on the security level of the first functional component and the platform risk factor corresponding to the first functional component, the risk impact level of the first functional component on the big data platform is determined by a platform risk impact level formula, wherein the platform risk impact level formula is: The risk impact of the functional component on the big data platform = (1-security level of the functional component) × platform risk factor; Wherein, determining the security level of the big data platform according to the risk impact levels of the multiple functional components on the big data platform respectively includes: The maximum one of the risk impact degrees of the multiple functional components on the big data platform is determined as the security degree of the big data platform.

6. The method according to claim 5, characterized in that Determining the risk impact of the first security domain on the first functional component based on the security level of the first security domain and the component risk factor corresponding to the first security domain includes: Based on the security level of the first security domain and the component risk factor corresponding to the first security domain, the risk impact level of the first security domain on the first functional component is determined by a component risk impact level formula, wherein the component risk impact level formula is: The risk impact of a security domain on its functional components = (1-security level of the security domain) × component risk factor.

7. The method according to claim 6, characterized in that Determining the security level of the first functional component according to the risk impact level of the at least one security domain on the first functional component respectively includes: Determine a maximum component risk impact degree from the risk impact degrees of the at least one security domain on the first functional component respectively; The safety level of the first functional component is determined by a functional component safety level formula, wherein the functional component safety level formula is: Functional component safety level = 1-maximum component risk impact level.

8. The method according to claim 7, characterized in that The component information also includes the security domain risk factor corresponding to each of the security indicator service items, and the security domain risk factor indicates the degree of impact of the corresponding security indicator service item on the security risk caused by the preset security domain to which it belongs. Wherein, determining the security level of at least one security indicator service item corresponding to the first security domain based on the port monitoring data in the first functional component includes: Determining a security value coefficient corresponding to a first security indicator service item based on the port monitoring data in the first functional component, wherein the first security indicator service item is any one of at least one security indicator service item corresponding to the first security domain; Based on the security domain risk factor and the security value coefficient corresponding to the first security indicator service item, the security degree of the first security indicator service item is determined by an indicator security degree formula, wherein the indicator security degree formula is: Safety index service item safety level = (1-safety domain risk factor) × safety value coefficient; Wherein, determining the security level of the first security domain according to the security level of at least one security indicator service item of the first security domain includes: A minimum one of the security levels of the at least one security indicator service item is determined as the security level of the first security domain.

9. The method according to any one of claims 1 to 8, characterized in that: Performing security monitoring on the ports of the multiple functional components based on the component information to obtain port monitoring data corresponding to the multiple functional components respectively includes: Based on the component information, security detection is periodically performed on the ports of the multiple functional components according to preset detection rules to obtain port monitoring data corresponding to the multiple functional components, wherein the preset detection rules include uniqueness detection and / or compliance detection.

10. The method according to claim 9, characterized in that The preset detection rules include a multidimensional risk matrix, the dimensions of which include an attack target dimension and an attack surface dimension. The value of any element in the multidimensional risk matrix represents the risk level caused to the big data platform by the attack target and attack surface corresponding to the location of the element.

Images