Safety early warning comprehensive management and control system based on big data cloud platform
By integrating data collection, processing, risk assessment, early warning and emergency response modules on the big data cloud platform, using data mining and machine learning technology, the problem that traditional security early warning systems cannot effectively monitor complex security threats is solved, and more efficient Security threat detection and response capabilities.
Patent Information
- Application Number
- CN202411047426.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional security early warning systems are unable to effectively monitor and warn of complex and diverse security threats in big data cloud platforms, resulting in poor security protection and response capabilities.
A comprehensive security early warning management and control system based on the big data cloud platform was designed, including data acquisition module, data processing and analysis module, risk assessment module, early warning generation and processing module, emergency response module and visual display module, using data mining and machine learning Technology monitors and analyzes massive data in real time, identifies potential threats, generates early warning information, and conducts emergency responses.
It improves the accuracy and response speed of security threat detection, enhances security protection and response capabilities, and provides clear risk management suggestions and real-time monitoring through risk assessment and visual presentation to help system managers respond to security risks in a timely manner.
Smart Images

Figure CN119030749A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data management and control technology, and in particular to a security early warning integrated management and control system based on a big data cloud platform. Background Art
[0002] With the continuous development and popularization of big data technology, big data cloud platform has become an important tool for data processing and management in all walks of life. In the information age, enterprises and organizations are facing increasingly complex and diverse security threats, such as cyber attacks, data leaks, malware, etc., and traditional security early warning systems can no longer meet the needs of effective monitoring and early warning of these security threats. In order to better cope with these security challenges, a security early warning integrated management and control system based on big data cloud platform came into being. The system uses big data technology to monitor and analyze massive data in real time, identifies potential security threats through machine learning and artificial intelligence algorithms, and issues early warning notifications in a timely manner to help enterprises and organizations take timely measures to deal with security risks.
[0003] After searching, the Chinese patent application number CN2020110344947 discloses a security early warning integrated management and control system based on a big data cloud platform. The present invention discloses a security early warning integrated management and control system based on a big data cloud platform, including: multiple wireless monitoring terminals, NB wireless gateways, cloud servers and display terminals; the wireless monitoring terminals include MCU, monitoring sensors, and LoRa modules; wherein the MCU transmits the monitoring data of the monitoring sensor to the NB wireless gateway through the LoRa module, the NB wireless gateway transmits the monitoring data to the cloud server through the mobile Internet, the cloud server transmits the monitoring data to the display terminal, and the display terminal displays the monitoring data. The XXXX device in the above patent has the following shortcomings:
[0004] Although the above has formed a set of fully functional integrated management and control system, including: dust monitoring system, gas monitoring system, electrical and line monitoring system, pressure and vibration detection system, it does not detect security threats to the data in each subsystem and generate corresponding early warning information. Therefore, the safety protection and response capabilities are relatively poor. Summary of the invention
[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a security early warning comprehensive management and control system based on a big data cloud platform.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] The security early warning integrated management and control system based on the big data cloud platform includes a data acquisition module, a data processing and analysis module, a risk assessment module, an early warning generation processing module, an emergency response module and a visualization display module. The data acquisition module collects data from various data sources and transmits the data to the big data cloud platform. The data processing and analysis module processes and analyzes the collected data and extracts potential security threat information through data mining and machine learning technologies. The risk assessment module evaluates the security risks existing in the system according to the results of data processing and analysis, and generates a corresponding risk assessment report. The early warning generation processing module generates corresponding early warning information, and classifies, screens and analyzes it, and generates corresponding response measures. The emergency response module quickly responds to and processes security incidents according to the response measures generated by the early warning information. The visualization display module displays the data, analysis results, early warning information and other contents in the system in a visual manner.
[0008] Preferably, the data acquisition module has the following specific working steps:
[0009] A1: Determine the data source, including network traffic, log files, sensor data and other data sources, and configure the corresponding data collector according to the type and format of the data source;
[0010] A2: Extract the required log information, network traffic, sensor data, etc. from the data source, and transfer the extracted data to the big data cloud platform;
[0011] A3: Clean and pre-process the transmitted data, including removing duplicate data, format conversion, data transcoding, etc.
[0012] A4: Store the cleaned and preprocessed data in the database on the big data cloud platform.
[0013] Furthermore: the data processing and analysis module has the following specific working steps:
[0014] B1: Integrate data from different data sources in the big data cloud platform database to form a unified data set;
[0015] B2: Select features for security threat prediction and detection, including access frequency, login behavior, network traffic, device information, abnormal activities, log information, and user behavior;
[0016] B3: Establish data mining model and machine learning model, and train the two models according to the collected data to build a prediction model;
[0017] B4: Evaluate data mining models through cross validation and ROC curves, and evaluate machine learning models through confusion matrix, precision and recall.
[0018] Based on the above scheme: In step B3, the algorithm formula of the data mining model is as follows:
[0019] Randomly select a feature and a random cut value to partition the data, and recursively repeat the above process in the subspace until all data points are isolated or the maximum depth of the tree is reached, according to:
[0020]
[0021] Where x is a data point, T is an isolated tree, E(h(x)) is the expected value of the path length of the data point x on the isolated tree, c(n) is the average path length of the tree, and S(x,T) is the anomaly score. The smaller this value is, the more likely the data point x is to be an outlier.
[0022] A better solution among the above solutions is: in the step S3, the algorithm formula of the machine learning model is as follows:
[0023] a: Calculate the probability of each category appearing in the training data set:
[0024]
[0025] Among them, N k Describe the category C in the data set k The number of occurrences, N represents the total number of samples in the training set;
[0026] b: For each feature X i , calculate the conditional probability of the feature appearing in a given category:
[0027]
[0028] Among them, N i,k Indicates that in category C k Medium Feature X i Number of occurrences, N k Indicates category C k The total number of times all features appear in |V| represents the dimension of the feature space, and α is the smoothing parameter (usually 1);
[0029] c: Calculate the posterior probability that the sample belongs to a certain category under each category:
[0030]
[0031] Where n represents the number of features, X i Represents the i-th feature of the sample.
[0032] As a further solution of the present invention: the risk assessment module has the following specific working steps:
[0033] C1: Based on the data processing and analysis results, the identified security risks are classified according to risk level, impact scope and type;
[0034] C2: Conduct qualitative assessment of each type of risk, including the likelihood of risk, impact, and risk level indicators;
[0035] C3: Analyze the security impact of various risks on system functions, data integrity and availability;
[0036] C4: Generate a risk assessment report based on the risk assessment results, including security risks, risk levels, risk impact analysis and recommended risk response measures.
[0037] At the same time, in the C1 step, the risk level, impact scope and type of the security risk are scored, and the specific scoring algorithm formula is as follows:
[0038] Risk level score: Risk level score = potential loss score + impact score + probability score; the potential loss score, impact score and probability score are determined according to the scoring standards of low (1-3 points), medium (4-6 points) and high (7-10 points);
[0039] Impact scope score: Impact scope score = system component score + data scope score + business process score; among which, the system component score, data scope score and business process score are determined according to the primary (1-5 points) and secondary (5-10 points) scoring criteria;
[0040] Risk type score: Risk type score = Risk feature 1 score + Risk feature 2 score + ... + Risk feature n Score.
[0041] As a preferred embodiment of the present invention, the specific working steps of the early warning generation processing module are as follows:
[0042] D1: Determine the existence of security risks and abnormal situations based on the data processing and analysis module, and generate warning information, including warning type, time, location and related data;
[0043] D2: Classify the generated warning information and distinguish different types of warnings, such as intrusion detection, abnormal behavior, and vulnerability exploitation;
[0044] D3: Conduct in-depth analysis of the screened warning information, including the cause, impact scope, and potential threats of the warning event, and explore hidden information in the warning information to discover potential security threats or vulnerabilities;
[0045] D4: Generate corresponding response measures and suggestions based on the early warning analysis results, including repairing security vulnerabilities, blocking abnormal behaviors, and emergency response to intrusion incidents.
[0046] At the same time, the emergency response module has the following specific working steps:
[0047] E1: Receives warning information from the warning generation and processing module, and verifies and confirms the received warning information;
[0048] E2: Initiate the corresponding emergency response plan based on the early warning information and verification results, including mobilizing the emergency response team and initiating the emergency response process;
[0049] E3: Conduct in-depth analysis of security incidents, including the cause of the incident, scope of impact, and threat level, and implement corresponding security measures based on the security incident analysis results and pre-planned response measures, including repairing vulnerabilities, blocking attacks, and isolating affected systems;
[0050] E4: During the security incident handling process, timely notify relevant personnel and departments, coordinate cooperation among all parties, track and monitor the security incident handling process, monitor the development and handling of security incidents in real time, and adjust the response strategy in a timely manner.
[0051] As a more optimal solution of the present invention: the specific working steps of the visual display module are as follows:
[0052] F1: Choose appropriate visualization methods, such as line charts, bar charts, pie charts, and heat maps, to display data and information based on user needs and system functions;
[0053] F2: Present the processed and analyzed data in a visual form on the interface, including dynamic charts, real-time data updates, and map display functions;
[0054] F3: Provides data interaction function, allowing users to interact with visual charts through interactive operations, and can generate reports and charts for users to download;
[0055] F4: Realize real-time monitoring of the system security status and timely update the data and information on the visual display interface.
[0056] The beneficial effects of the present invention are:
[0057] 1. The security early warning integrated management and control system based on the big data cloud platform can analyze a large amount of security data through data mining and machine learning technology, discover potential patterns, abnormal behaviors and laws, thereby improving the accuracy of security threat detection. Through real-time analysis of big data, the system can promptly discover new security threats and generate corresponding early warning information to help early detection and response to potential security risks. Data mining and machine learning technology can automatically process a large amount of complex security data, identify abnormal behaviors and threats, reduce the workload of the security team, and improve the response speed of security incidents. By analyzing historical security data and the processing process of security incidents, the system can continuously optimize security strategies and early warning rules, and improve security protection and response capabilities.
[0058] 2. The risk assessment module of the security early warning integrated management and control system based on the big data cloud platform can comprehensively consider various risk factors and assess the security risks faced by the system by classifying the identified security risks according to risk level, impact scope and type, and conducting qualitative assessment of each type of risk. In addition to qualitative assessment of risks, the risk assessment module can also conduct quantitative assessment of indicators such as the possibility and impact of risks, and determine the risk level through specific scoring algorithms and formulas, which helps to objectively and quantitatively assess the severity of risks. The risk assessment module can analyze the security impact of various risks on system functions, data integrity and availability, help system managers better understand the potential impact of different types of risks on system operations, and provide a reference for the formulation of risk response measures. Based on the risk assessment results, the risk assessment module can generate a detailed risk assessment report, which includes security risks, risk levels, risk impact analysis and recommended risk response measures, and provide system managers with clear risk management recommendations and decision support.
[0059] 3. The security early warning integrated management and control system based on the big data cloud platform, the early warning generation processing module can quickly generate early warning information according to the judgment of the data processing and analysis module, including the early warning type, time, location and related data, to achieve real-time monitoring and early warning of security risks and abnormal situations. The early warning generation processing module classifies the generated early warning information and distinguishes different types of early warnings, such as intrusion detection, abnormal behavior, vulnerability exploitation, etc., which helps system managers to have a clearer understanding of the security threats faced by the system. The warning generation processing module conducts in-depth analysis of the screened early warning information, including analysis of the causes, impact scope, potential threats and other aspects of the early warning events, and can mine hidden information in the early warning information and discover potential security threats or vulnerabilities. According to the early warning analysis results, the early warning generation processing module can generate corresponding response measures and suggestions, including repairing security vulnerabilities, blocking abnormal behaviors, emergency response to intrusion incidents, etc., which helps system managers to take timely measures to deal with security threats.
[0060] 4. The security early warning integrated management and control system based on the big data cloud platform displays data and information in an appropriate visual way, such as line charts, bar charts, pie charts, heat maps, etc., so that complex data and analysis results are presented to users in an intuitive and easy-to-understand form, improving users' understanding and grasp of the system security status. It can also present the processed and analyzed data in a visual form in real time on the interface, including dynamic charts, real-time data updates and other functions, to help users keep abreast of the latest status and dynamic changes of the system, and provide data interaction functions, so that users can interact with visual charts through interactive operations, such as filtering, searching, zooming in and out, etc., to help users gain an in-depth understanding of the information and trends behind the data and improve the depth and breadth of data analysis.
[0061] 5. The data collection module of the security early warning integrated management and control system based on the big data cloud platform can collect data from multiple data sources, realize comprehensive data collection, and realize automatic data extraction and transmission by configuring data collectors and rules, improve the efficiency of data processing, collect data in real time, and transmit it to the big data cloud platform for processing and analysis, realize real-time monitoring and response to security incidents, and ensure the accuracy and completeness of the collected data and improve data quality through operations such as data cleaning and preprocessing. It can be flexibly configured and expanded as needed, support data sources of different types and sizes, and meet the changing needs of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 This is a module composition diagram of the security early warning comprehensive management and control system based on the big data cloud platform proposed by the present invention;
[0063] Figure 2 This is a flow chart of the data acquisition module in the security early warning integrated management and control system based on the big data cloud platform proposed by the present invention;
[0064] Figure 3 This is a flow chart of the data processing and analysis module in the security early warning integrated management and control system based on the big data cloud platform proposed by the present invention;
[0065] Figure 4 This is a flow chart of the risk assessment module in the security early warning integrated management and control system based on the big data cloud platform proposed by the present invention;
[0066] Figure 5 This is a flow chart of the early warning generation processing module in the security early warning comprehensive management and control system based on the big data cloud platform proposed by the present invention; DETAILED DESCRIPTION
[0067] The technical solution of the present invention is further described in detail below in conjunction with specific implementation methods.
[0068] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0069] Security early warning integrated management and control system based on big data cloud platform, such as Figure 1 As shown, it includes a data acquisition module, a data processing and analysis module, a risk assessment module, an early warning generation processing module, an emergency response module and a visualization display module. The data acquisition module collects data from various data sources and transmits the data to the big data cloud platform. The data processing and analysis module processes and analyzes the collected data, and extracts potential security threat information through data mining and machine learning technologies. The risk assessment module evaluates the security risks existing in the system according to the results of data processing and analysis, and generates a corresponding risk assessment report. The early warning generation processing module generates corresponding early warning information, and classifies, screens and analyzes it, and generates corresponding countermeasures. The emergency response module quickly responds to and processes security incidents according to the countermeasures generated by the early warning information. The visualization display module displays the data, analysis results, early warning information and other contents in the system in a visual manner.
[0070] In order to provide basic data support for subsequent data processing and analysis; Figure 2 As shown, the data acquisition module has the following specific working steps:
[0071] A1: Determine the data source, including network traffic, log files, sensor data and other data sources, and configure the corresponding data collector according to the type and format of the data source;
[0072] A2: Extract the required log information, network traffic, sensor data, etc. from the data source, and transfer the extracted data to the big data cloud platform;
[0073] A3: Clean and pre-process the transmitted data, including removing duplicate data, format conversion, data transcoding, etc.
[0074] A4: Store the cleaned and preprocessed data in the database on the big data cloud platform.
[0075] The data collection module can collect data from a variety of data sources, realizing comprehensive data collection. By configuring data collectors and rules, it can realize automatic data extraction and transmission, improve the efficiency of data processing, collect data in real time, and transmit it to the big data cloud platform for processing and analysis, so as to realize real-time monitoring and response to security incidents. Through operations such as data cleaning and preprocessing, the data collection module can ensure the accuracy and completeness of the collected data and improve data quality. It can be flexibly configured and expanded as needed, supporting data sources of different types and sizes to meet the changing needs of the system.
[0076] In order to provide effective security protection and risk management for the security early warning integrated management and control system; Figure 3 As shown, the data processing and analysis module has the following specific working steps:
[0077] B1: Integrate data from different data sources in the big data cloud platform database to form a unified data set;
[0078] B2: Select features for security threat prediction and detection, including access frequency, login behavior, network traffic, device information, abnormal activities, log information, and user behavior;
[0079] B3: Establish data mining model and machine learning model, and train the two models according to the collected data to build a prediction model;
[0080] B4: Evaluate data mining models through cross validation and ROC curves, and evaluate machine learning models through confusion matrix, precision and recall.
[0081] In step B3, the algorithm formula of the data mining model is as follows:
[0082] Randomly select a feature and a random cut value to partition the data, and recursively repeat the above process in the subspace until all data points are isolated or the maximum depth of the tree is reached, according to:
[0083]
[0084] Where x is a data point, T is an isolated tree, E(h(x)) is the expected value of the path length of the data point x on the isolated tree, c(n) is the average path length of the tree, and S(x,T) is the anomaly score. The smaller this value is, the more likely the data point x is an outlier.
[0085] The algorithm formula of the machine learning model is as follows:
[0086] a: Calculate the probability of each category appearing in the training data set:
[0087]
[0088] Among them, N k Describe the category C in the data set k The number of occurrences, N represents the total number of samples in the training set;
[0089] b: For each feature X i , calculate the conditional probability of the feature appearing in a given category:
[0090]
[0091] Among them, N i,k Indicates that in category C k Medium Feature X i Number of occurrences, N k Indicates category C k The total number of times all features appear in |V| represents the dimension of the feature space, and α is the smoothing parameter (usually 1);
[0092] c: Calculate the posterior probability that the sample belongs to a certain category under each category:
[0093]
[0094] Where n represents the number of features, X i Represents the i-th feature of the sample.
[0095] Data mining and machine learning technologies can be used to analyze large amounts of security data, discover potential patterns, abnormal behaviors, and regularities, thereby improving the accuracy of security threat detection. By analyzing big data in real time, the system can promptly discover new security threats and generate corresponding warning information to help detect and respond to potential security risks early. Data mining and machine learning technologies can automatically process large amounts of complex security data, identify abnormal behaviors and threats, reduce the workload of the security team, and improve the response speed to security incidents. By analyzing historical security data and the processing of security incidents, the system can continuously optimize security strategies and warning rules, and improve security protection and response capabilities.
[0096] In order to provide important reference for system security management and risk control; Figure 4 As shown, the risk assessment module has the following specific working steps:
[0097] C1: Based on the data processing and analysis results, the identified security risks are classified according to risk level, impact scope and type;
[0098] C2: Conduct qualitative assessment of each type of risk, including the likelihood of risk, impact, and risk level indicators;
[0099] C3: Analyze the security impact of various risks on system functions, data integrity and availability;
[0100] C4: Generate a risk assessment report based on the risk assessment results, including security risks, risk levels, risk impact analysis and recommended risk response measures.
[0101] In the C1 step, the risk level, impact scope and type of the security risk are scored, and the specific scoring algorithm formula is as follows:
[0102] Risk level score: Risk level score = potential loss score + impact score + probability score; the potential loss score, impact score and probability score are determined according to the scoring standards of low (1-3 points), medium (4-6 points) and high (7-10 points);
[0103] Impact scope score: Impact scope score = system component score + data scope score + business process score; among which, the system component score, data scope score and business process score are determined according to the primary (1-5 points) and secondary (5-10 points) scoring criteria;
[0104] Risk type score: Risk type score = Risk feature 1 score + Risk feature 2 score + ... + Risk feature n Score.
[0105] The risk assessment module classifies the identified security risks according to risk level, impact scope and type, and conducts qualitative assessment of each type of risk. It can comprehensively consider various risk factors and assess the security risks faced by the system. In addition to qualitative assessment of risks, the risk assessment module can also conduct quantitative assessment of indicators such as risk possibility and impact, and determine the risk level through specific scoring algorithms and formulas, which helps to objectively and quantitatively assess the severity of the risk. The risk assessment module can analyze the security impact of various risks on system functions, data integrity and availability, help system managers better understand the potential impact of different types of risks on system operations, and provide a reference for the formulation of risk response measures. Based on the risk assessment results, the risk assessment module can generate a detailed risk assessment report. The report content includes security risks, risk levels, risk impact analysis and recommended risk response measures, providing system managers with clear risk management recommendations and decision support.
[0106] To help system administrators detect and handle potential security threats in a timely manner; Figure 5 As shown, the specific working steps of the warning generation processing module are as follows:
[0107] D1: Determine the existence of security risks and abnormal situations based on the data processing and analysis module, and generate warning information, including warning type, time, location and related data;
[0108] D2: Classify the generated warning information and distinguish different types of warnings, such as intrusion detection, abnormal behavior, and vulnerability exploitation;
[0109] D3: Conduct in-depth analysis of the screened warning information, including the cause, impact scope, and potential threats of the warning event, and explore hidden information in the warning information to discover potential security threats or vulnerabilities;
[0110] D4: Generate corresponding response measures and suggestions based on the early warning analysis results, including repairing security vulnerabilities, blocking abnormal behaviors, and emergency response to intrusion incidents.
[0111] The warning generation and processing module can quickly generate warning information according to the judgment of the data processing and analysis module, including the warning type, time, location and related data, to achieve real-time monitoring and warning of security risks and abnormal situations. The warning generation and processing module classifies the generated warning information and distinguishes different types of warnings, such as intrusion detection, abnormal behavior, vulnerability exploitation, etc., which helps system managers to have a clearer understanding of the security threats facing the system. The warning generation and processing module conducts in-depth analysis of the screened warning information, including analysis of the causes, impact scope, potential threats and other aspects of the warning events. It can mine hidden information in the warning information and discover potential security threats or vulnerabilities. According to the warning analysis results, the warning generation and processing module can generate corresponding response measures and suggestions, including repairing security vulnerabilities, blocking abnormal behaviors, emergency response to intrusion incidents, etc., which helps system managers to take timely measures to deal with security threats.
[0112] In order to summarize the experience and lessons learned, and continuously optimize and improve the safety protection capability of the safety early warning integrated management and control system; the specific working steps of the emergency response module are as follows:
[0113] E1: Receives warning information from the warning generation and processing module, and verifies and confirms the received warning information;
[0114] E2: Initiate the corresponding emergency response plan based on the early warning information and verification results, including mobilizing the emergency response team and initiating the emergency response process;
[0115] E3: Conduct in-depth analysis of security incidents, including the cause of the incident, scope of impact, and threat level, and implement corresponding security measures based on the security incident analysis results and pre-planned response measures, including repairing vulnerabilities, blocking attacks, and isolating affected systems;
[0116] E4: During the security incident handling process, timely notify relevant personnel and departments, coordinate cooperation among all parties, track and monitor the security incident handling process, monitor the development and handling of security incidents in real time, and adjust the response strategy in a timely manner.
[0117] The emergency response module can quickly receive and verify early warning information, initiate emergency response plans, and quickly respond to and handle security incidents, which helps to respond to various security threats and abnormal situations in a timely manner. Through the unified coordination and management of the emergency response module, it can mobilize the emergency response team and coordinate cooperation among all parties to ensure that security incidents are effectively handled, improve response efficiency and collaboration effects, and conduct in-depth analysis of security incidents, including the cause of the incident, the scope of impact, the degree of threat, etc., which helps to fully understand the background and impact of security incidents and provide support for the formulation of effective response measures. According to the security incident analysis results and pre-formulated response measures, the emergency response module can automatically execute corresponding security measures, such as repairing vulnerabilities, blocking attacks, isolating affected systems, etc., reducing human intervention and processing time, and improving response efficiency. During the security incident handling process, the emergency response module can monitor the development and handling of security incidents in real time, adjust the response strategy in a timely manner, and ensure that security incidents are effectively controlled and resolved.
[0118] In order to help users better understand the security status of the system and discover potential problems, the specific working steps of the visualization module are as follows:
[0119] F1: Choose appropriate visualization methods, such as line charts, bar charts, pie charts, and heat maps, to display data and information based on user needs and system functions;
[0120] F2: Present the processed and analyzed data in a visual form on the interface, including dynamic charts, real-time data updates, and map display functions;
[0121] F3: Provides data interaction function, allowing users to interact with visual charts through interactive operations, and can generate reports and charts for users to download;
[0122] F4: Realize real-time monitoring of the system security status and timely update the data and information on the visual display interface.
[0123] By selecting appropriate visualization methods to display data and information, such as line graphs, bar graphs, pie charts, heat maps, etc., complex data and analysis results can be presented to users in an intuitive and easy-to-understand form, thereby improving users' understanding and grasp of the system security status. In addition, the processed and analyzed data can be presented in real time on the interface in a visual form, including dynamic charts, real-time data updates and other functions, to help users keep abreast of the latest status and dynamic changes of the system. Data interaction functions are provided, allowing users to interact with visual charts through interactive operations, such as filtering, searching, zooming in and out, etc., to help users gain an in-depth understanding of the information and trends behind the data and improve the depth and breadth of data analysis.
[0124] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A comprehensive safety early warning management and control system based on a big data cloud platform, including a data acquisition module, a data processing and analysis module, a risk assessment module, an early warning generation and processing module, an emergency response module and a visual display module, characterized in that: The data acquisition module collects data from various data sources and transmits the data to the big data cloud platform. The data processing and analysis module processes and analyzes the collected data and extracts potential security threat information through data mining and machine learning technologies. The risk assessment module assesses the security risks existing in the system based on the results of data processing and analysis and generates a corresponding risk assessment report. The early warning generation and processing module generates corresponding early warning information, and classifies, screens and analyzes it, and generates corresponding response measures. The emergency response module quickly responds to and processes security incidents based on the response measures generated by the early warning information. The visualization module displays the data, analysis results, early warning information and other contents in the system in a visual manner.
2. According to claim 1, the security early warning integrated management and control system based on the big data cloud platform is characterized in that: The data acquisition module has the following specific working steps: A1: Determine the data source, including network traffic, log files, sensor data and other data sources, and configure the corresponding data collector according to the type and format of the data source; A2: Extract the required log information, network traffic, sensor data, etc. from the data source, and transfer the extracted data to the big data cloud platform; A3: Clean and pre-process the transmitted data, including removing duplicate data, format conversion, data transcoding, etc. A4: Store the cleaned and preprocessed data in the database on the big data cloud platform.
3. The security early warning integrated management and control system based on the big data cloud platform according to claim 1 is characterized in that: The data processing and analysis module has the following specific working steps: B1: Integrate data from different data sources in the big data cloud platform database to form a unified data set; B2: Select features for security threat prediction and detection, including access frequency, login behavior, network traffic, device information, abnormal activities, log information, and user behavior; B3: Establish data mining model and machine learning model, and train the two models according to the collected data to build a prediction model; B4: Evaluate data mining models through cross validation and ROC curves, and evaluate machine learning models through confusion matrix, precision and recall.
4. The security early warning integrated management and control system based on the big data cloud platform according to claim 3 is characterized in that: In step B3, the algorithm formula of the data mining model is as follows: Randomly select a feature and a random cut value to partition the data, and recursively repeat the above process in the subspace until all data points are isolated or the maximum depth of the tree is reached, according to: Where x is a data point, T is an isolated tree, E(h(x)) is the expected value of the path length of the data point x on the isolated tree, c(n) is the average path length of the tree, and S(x,T) is the anomaly score. The smaller this value is, the more likely the data point x is to be an outlier.
5. The security early warning integrated management and control system based on the big data cloud platform according to claim 3 is characterized in that: In the S3 step, the algorithm formula of the machine learning model is as follows: a: Calculate the probability of each category appearing in the training data set: Among them, N k Describe the category C in the data set k The number of occurrences, N represents the total number of samples in the training set; b: For each feature X i , calculate the conditional probability of the feature appearing in a given category: Among them, N i,k Indicates that in category C k Medium Feature X i Number of occurrences, N k Indicates category C k The total number of times all features appear in |V| represents the dimension of the feature space, and α is the smoothing parameter (usually 1); c: Calculate the posterior probability that the sample belongs to a certain category under each category: Where n represents the number of features, X i Represents the i-th feature of the sample.
6. The security early warning integrated management and control system based on the big data cloud platform according to claim 1 is characterized in that: The specific working steps of the risk assessment module are as follows: C1: Based on the data processing and analysis results, the identified security risks are classified according to risk level, impact scope and type; C2: Conduct qualitative assessment of each type of risk, including the likelihood of risk, impact, and risk level indicators; C3: Analyze the security impact of various risks on system functions, data integrity and availability; C4: Generate a risk assessment report based on the risk assessment results, including security risks, risk levels, risk impact analysis and recommended risk response measures.
7. The security early warning integrated management and control system based on the big data cloud platform according to claim 6 is characterized in that: In the C1 step, the risk level, impact scope and type of the security risk are scored, and the specific scoring algorithm formula is as follows: Risk level score: Risk level score = potential loss score + impact score + probability score; the potential loss score, impact score and probability score are determined according to the scoring standards of low (1-3 points), medium (4-6 points) and high (7-10 points); Impact scope score: Impact scope score = system component score + data scope score + business process score; among which, the system component score, data scope score and business process score are determined according to the primary (1-5 points) and secondary (5-10 points) scoring criteria; Risk type score: Risk type score = Risk feature 1 score + Risk feature 2 score + ... + Risk feature n Score.
8. The security early warning integrated management and control system based on the big data cloud platform according to claim 1 is characterized in that: The specific working steps of the warning generation processing module are as follows: D1: Determine the existence of security risks and abnormal situations based on the data processing and analysis module, and generate warning information, including warning type, time, location and related data; D2: Classify the generated warning information and distinguish different types of warnings, such as intrusion detection, abnormal behavior, and vulnerability exploitation; D3: Conduct in-depth analysis of the screened warning information, including the cause, impact scope, and potential threats of the warning event, and explore hidden information in the warning information to discover potential security threats or vulnerabilities; D4: Generate corresponding response measures and suggestions based on the early warning analysis results, including repairing security vulnerabilities, blocking abnormal behaviors, and emergency response to intrusion incidents.
9. The security early warning integrated management and control system based on the big data cloud platform according to claim 1 is characterized in that: The specific working steps of the emergency response module are as follows: E1: Receives warning information from the warning generation and processing module, and verifies and confirms the received warning information; E2: Initiate the corresponding emergency response plan based on the early warning information and verification results, including mobilizing the emergency response team and initiating the emergency response process; E3: Conduct in-depth analysis of security incidents, including the cause of the incident, scope of impact, and threat level, and implement corresponding security measures based on the security incident analysis results and pre-planned response measures, including repairing vulnerabilities, blocking attacks, and isolating affected systems; E4: During the security incident handling process, timely notify relevant personnel and departments, coordinate cooperation among all parties, track and monitor the security incident handling process, monitor the development and handling of security incidents in real time, and adjust the response strategy in a timely manner.
10. The security early warning integrated management and control system based on the big data cloud platform according to claim 1 is characterized in that: The specific working steps of the visual display module are as follows: F1: Choose appropriate visualization methods, such as line charts, bar charts, pie charts, and heat maps, to display data and information based on user needs and system functions; F2: Present the processed and analyzed data in a visual form on the interface, including dynamic charts, real-time data updates, and map display functions; F3: Provides data interaction function, allowing users to interact with visual charts through interactive operations, and can generate reports and charts for users to download; F4: Realize real-time monitoring of the system security status and timely update the data and information on the visual display interface.
Citation Information
Patent Citations
Security early warning comprehensive management and control system based on big data cloud platform
CN113012413A
Numerical control system security situation awareness and analysis system, method, equipment and terminal
CN115996146A
Computer security management system and method based on artificial intelligence
CN117574361A
Industrial chain risk management system and method based on digital intelligence
CN117952423A
Communication information security risk early warning management and control method and system based on big data
CN117955712A
Cited By
Mine mining electromechanical equipment fault intelligent diagnosis system and diagnosis method
CN119849945A
Computer network big data security protection method and system
CN119892504A
Intelligent factory affair management system and method based on cloud platform
CN120181516A
Cloud platform safety monitoring system based on big data
CN120223395A
A cloud platform security monitoring system based on big data
CN120223395B
