A computer data security monitoring method and system based on big data technology
Through the abnormal traffic and intrusion monitoring of big data technology, computer data security monitoring methods solve the problem that traditional security measures cannot cope with complex network attacks, achieve stronger threat identification and response, and improve the dynamicity and processing efficiency of security policies.
Patent Information
- Application Number
- CN202410996106.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-24
AI Technical Summary
Traditional security protection measures cannot effectively deal with complex and changeable ubiquitous cyber attacks, and it is difficult to fully identify and respond to various cybersecurity threats, especially traffic anomalies and DDoS attacks.
Using computer data security monitoring methods based on big data technology, detecting instructions are generated through abnormal traffic monitoring and intrusion monitoring, security risk coefficient and security coefficient are calculated, and security strategies are dynamically adjusted, and abnormal behaviors are refined in time, and abnormal behaviors are located continuously. Monitor and classify outliers.
It provides a stronger first line of defense, reduces false alarms, quickly locates problems, improves diagnosis and resolution efficiency, promptly detects long-term abnormal behaviors, optimizes resource allocation, and improves processing efficiency.
Smart Images

Figure CN118520472B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data security monitoring, and in particular to a computer data security monitoring method and system based on big data technology. Background Art
[0002] With the increase in data value, big data has become a core resource in modern society. The explosive growth of data volume has brought huge value, but also triggered unprecedented security challenges. Data security refers to the security protection of technology and management established and adopted for data processing systems to protect computer hardware, software and data from being destroyed, changed and leaked due to accidental and malicious reasons. Computer data security can be understood as: by adopting various technologies and management measures to ensure the normal operation of network systems, thereby ensuring the availability, integrity and confidentiality of data.
[0003] As data grows, security threats are also evolving and escalating. Traditional security protection measures, such as firewalls and anti-virus software, can no longer cope with complex and ever-changing ubiquitous network attacks alone. Summary of the invention
[0004] The purpose of the present invention is to solve the problems in the background technology and to propose a computer data security monitoring method and system based on big data technology.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A computer data security monitoring method based on big data technology comprises the following steps:
[0007] S101, generating detection instructions through abnormal traffic monitoring and intrusion monitoring, and identifying abnormalities based on security detection instructions;
[0008] S102, setting the start time and end time of the average broadband deviation from the average value as the start time point and the end time point, and calculating the abnormal value;
[0009] S103, calculating the security risk factor and safety factor of computer data;
[0010] S104, monitoring the abnormal duration of the monitoring start time and the monitoring end time, determining the abnormal value existing in the computer data, and matching it with the abnormal threshold to obtain the abnormal level;
[0011] In S101, by acquiring abnormal traffic monitoring and intrusion detection in the network monitoring tool in the computer, a detection instruction is generated, and the detection instruction includes: a first detection instruction and a second detection instruction. The first detection instruction sets a normal traffic level based on historical data, defines a threshold value exceeding the level, and is acquired through bandwidth usage. The specific calculation formula of the bandwidth usage is as follows:
[0012]
[0013] Where S represents the bandwidth usage, S 实 Indicates the actual bandwidth usage, S 总 Indicates the total available bandwidth. Statistical methods are used to identify abnormal behaviors. The standard deviation formula is used to identify abnormal values that deviate from the average value. The specific calculation formula is as follows:
[0014]
[0015] Among them, x i Represents the i-th observation value, N represents the number of observations, μ represents the average value, the second detection instruction is configured in the network node, and the IDS is used for real-time monitoring to ensure that all traffic is detected, and the signature database of the IDS is regularly updated to identify new threats and attack patterns, abnormal traffic monitoring detects whether the average bandwidth in the network deviates from the average value, if there are abnormal values deviating from the average value in the detection area, the first detection instruction is generated, if there are no abnormal values deviating from the average value in the detection area, the first detection instruction is not generated, intrusion detection detects the existence of connection attempts and frequent login failures in the monitoring area, if there are connection attempts and frequent login failures in the detection area, the second detection instruction is generated, if there are no connection attempts and frequent login failures in the detection area, the second detection instruction is not generated;
[0016] In the S102, the start time and end time of the average broadband deviation from the average value are set as the start time and the end time. Within the monitoring time period, the targets starting and ending in the monitoring area are monitored and evaluated. The start abnormal value and the end abnormal value within the monitoring time period are marked as start and ND respectively, and the corresponding time period weight value within the monitoring time period is obtained and marked as TE. The abnormal value is calculated. The specific calculation method is as follows:
[0017] NM = (start + ND) × TE;
[0018] Among them, NM represents the monitoring evaluation value, according to the abnormal value, an abnormal value monitoring set is generated, according to the abnormal value monitoring set, a capture instruction is generated, and according to the capture instruction, the corresponding abnormal traffic in the network bandwidth is extracted.
[0019] Furthermore, the abnormal traffic monitoring detects whether the average bandwidth in the network deviates from the average value. If there is an abnormal value deviating from the average value in the detection area, a first detection instruction is generated; if there is no abnormal value deviating from the average value in the detection area, the first detection instruction is not generated. The intrusion detection detects the existence of connection attempts and frequent login failures in the monitoring area. If there are connection attempts and frequent login failures in the detection area, a second detection instruction is generated; if there are no connection attempts and frequent login failures in the detection area, the second detection instruction is not generated.
[0020] Furthermore, an abnormal value monitoring set is generated based on the abnormal value, a capture instruction is generated based on the abnormal value monitoring set, and the corresponding abnormal traffic in the network bandwidth is extracted based on the capture instruction.
[0021] Furthermore, in S103, the security risk factor of the computer data is calculated and the intrusion sr , Virus InfectionVI sr , Information Theft sr And data leakage Pd sr , perform normalization processing to obtain the safety risk coefficient, the specific calculation formula of the safety risk coefficient is as follows:
[0022] AF=w1·In sr +w2·VI sr +w3·It sr +w4·Pd sr ;
[0023] Among them, AF represents the security risk factor, w1, w2, w3, and w4 represent weights, which are used to evaluate the risk value of each risk factor. The weight reflects the impact of the numerical value contained in the production process on computer data security. The greater the impact, the greater the weight of the corresponding numerical value.
[0024] Furthermore, the safety factor of computer data is calculated. The safety factor is calculated by obtaining the historical data LY of abnormal traffic in the network broadband, counting the total number of abnormal traffic corresponding to different detection conditions and marking it as YZ, and obtaining the type weight YQ of abnormal traffic. The specific calculation formula of the safety factor is as follows:
[0025]
[0026] Among them, regulated represents the safety factor, YZ i represents the number of abnormal traffic of the ith i represents the corresponding type weight, LY represents the total traffic, n represents the number of abnormal traffic types, and T represents the adjustment coefficient. The specific calculation formula of the adjustment coefficient is as follows:
[0027]
[0028] Among them, the calculation of the adjustment coefficient reflects the impact of the weighted proportion of abnormal traffic in the total traffic on the safety factor.
[0029] Furthermore, in S104, the abnormal duration D between the monitoring start time and the monitoring end time is normal Monitor and calculate the monitoring results R for each day i , by presetting a threshold, it is determined whether the monitoring result is abnormal. The calculation formula of the abnormal value is as follows:
[0030]
[0031] Among them, A i represents the abnormal value of the test result on the i-th day, Th represents the preset abnormal threshold, and the factor of abnormal duration is incorporated into the daily abnormal value calculation, and the formula for further adjusting the abnormal value is:
[0032]
[0033] Among them, 365 is the number of days in a year, which is used as a normalization factor, and otherwise means that when the actual observation value is less than or equal to the threshold, the outlier value is 0.
[0034] Furthermore, the abnormal values are matched with the preset abnormal thresholds, and the abnormal values are classified into normal, slightly abnormal, moderately abnormal, and highly abnormal. The expressions are as follows:
[0035]
[0036] Among them, Level i represents the abnormal level of the i-th day, Th mild Indicates mild abnormality, Th moderate Indicates moderate abnormality.
[0037] The present invention also provides a computer data security monitoring system based on big data technology, comprising:
[0038] Detection instruction generation module: Generates detection instructions through abnormal traffic monitoring and intrusion monitoring, and identifies anomalies based on security detection instructions;
[0039] Abnormal analysis module: the start time and the end time of the deviation of the average broadband from the average value are set as the start time and the end time, and the abnormal value is calculated;
[0040] Risk calculation module: calculates the security risk factor and safety factor of computer data;
[0041] Abnormal value monitoring module: monitors the abnormal duration of the monitoring start time and the monitoring end time, determines the abnormal values in the computer data, and matches them with the abnormal threshold to obtain the abnormal level.
[0042] Compared with the prior art, the advantages of the present invention are:
[0043] The present invention integrates abnormal traffic monitoring and intrusion monitoring, and the system can fully identify and respond to various network security threats, which include not only traditional intrusion attempts, but also traffic anomalies, such as DDoS attacks or scanning behaviors, thereby providing a stronger first line of defense for network security. By analyzing the time point when the average bandwidth deviates from the average value in detail, the start and end time of the anomaly are accurately set. This refined time positioning not only helps to reduce false alarms, but also can quickly locate the specific time period when the problem occurs, improve the efficiency of problem diagnosis and resolution, and the risk calculation combines real-time data and historical data to dynamically calculate the security risk factor and safety factor, which enables the system to intelligently adjust the security strategy according to the current network environment and previous behavior patterns, and respond to potential security threats in the most appropriate way. By continuously monitoring the abnormal duration between the defined start and end times, the system can continuously track the abnormal state of the data. This helps to timely discover long-term abnormal behavior, thereby preventing long-term or chronic network problems and security vulnerabilities, matching the monitored abnormal values with the preset abnormal thresholds, and then hierarchically processing the anomalies. This hierarchical mechanism enables the system to take different countermeasures according to the severity of the anomaly, optimize resource allocation, and improve processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a flowchart of a computer data security monitoring method based on big data technology proposed in Example 1 of the present invention.
[0045] Figure 2 This is a structural block diagram of a computer data security monitoring system based on big data technology proposed in Example 2 of the present invention. DETAILED DESCRIPTION
[0046] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0047] Embodiment 1
[0048] Reference Figure 1 The present invention provides a computer data security monitoring method based on big data technology, comprising:
[0049] S101, generating detection instructions through abnormal traffic monitoring and intrusion monitoring, and identifying abnormalities based on security detection instructions;
[0050] S102, setting the start time and end time of the average broadband deviation from the average value as the start time point and the end time point, and calculating the abnormal value;
[0051] S103, calculating the security risk factor and safety factor of computer data;
[0052] S104: Monitor the abnormal duration of the monitoring start time and the monitoring end time, determine the abnormal value existing in the computer data, and match it with the abnormal threshold to obtain the abnormal level.
[0053] In this embodiment, it is specifically necessary to explain that S101 generates a detection instruction by acquiring abnormal traffic monitoring and intrusion detection in a network monitoring tool in a computer. The detection instruction includes: a first detection instruction and a second detection instruction. The first detection instruction sets a normal traffic level based on historical data, defines a threshold value exceeding the level, and obtains the bandwidth usage rate. The specific calculation formula of the bandwidth usage rate is as follows:
[0054]
[0055] Where S represents the bandwidth usage, S 实 Indicates the actual bandwidth usage, S 总 Indicates the total available bandwidth. Statistical methods are used to identify abnormal behaviors. The standard deviation formula is used to identify abnormal values that deviate from the average value. The specific calculation formula is as follows:
[0056]
[0057] Among them, x i Represents the i-th observation value, N represents the number of observations, μ represents the average value, the second detection instruction is configured in the IDS in the network node, and the IDS is used for real-time monitoring to ensure that all traffic is detected, and the signature database of the IDS is updated regularly to identify new threats and attack patterns. The abnormal traffic monitoring detects whether the average bandwidth in the network deviates from the average value. If there are abnormal values deviating from the average value in the detection area, the first detection instruction is generated. If there are no abnormal values deviating from the average value in the detection area, the first detection instruction is not generated. The intrusion detection detects the existence of connection attempts and frequent login failures in the monitoring area. If there are connection attempts and frequent login failures in the detection area, the second detection instruction is generated. If there are no connection attempts and frequent login failures in the detection area, the second detection instruction is not generated.
[0058] In this embodiment, what needs to be explained specifically is S102, the time points of the start time and the end time of the average broadband deviation from the average value are set as the start time point and the end time point, and the targets starting and ending in the monitoring area are monitored and evaluated within the monitoring time period, and the start abnormal value and the end abnormal value within the monitoring time period are marked as start and ND respectively, and the corresponding time period weight value within the monitoring time period is obtained and marked as TE, and the abnormal value is calculated. The specific calculation method is as follows:
[0059] NM = (start + ND) × TE;
[0060] Among them, NM represents the monitoring evaluation value, according to the abnormal value, an abnormal value monitoring set is generated, according to the abnormal value monitoring set, a capture instruction is generated, and according to the capture instruction, the corresponding abnormal traffic in the network bandwidth is extracted.
[0061] In this embodiment, it is specifically necessary to explain that S103 calculates the security risk coefficient of the computer data and calculates the intrusion risk coefficient of the computer data. sr , Virus InfectionVI sr , Information Theft sr And data leakage Pd sr , perform normalization processing to obtain the safety risk coefficient, the specific calculation formula of the safety risk coefficient is as follows:
[0062] AF=w1·In sr +w2·VI sr +w3·It sr +w4·Pd sr ;
[0063] Among them, AF represents the security risk factor, w1, w2, w3, and w4 represent weights, which are used to evaluate the risk value of each risk factor. The weight reflects the impact of the numerical value contained in the production process on the security of computer data. The greater the impact, the greater the weight of the corresponding numerical value. The safety factor of computer data is calculated. The safety factor is obtained by obtaining the historical data LY of abnormal traffic in the network broadband, counting the total number of abnormal traffic corresponding to different detection conditions and marking it as YZ, and obtaining the type weight YQ of abnormal traffic. The specific calculation formula of the safety factor is as follows:
[0064]
[0065] Among them, regulated represents the safety factor, YZ i represents the number of abnormal traffic of the ith i Indicates the corresponding type weight, LY indicates the total traffic, n indicates the number of abnormal traffic types, and T indicates the adjustment coefficient. The specific calculation formula of the adjustment coefficient is as follows:
[0066]
[0067] Among them, the calculation of the adjustment coefficient reflects the impact of the weighted proportion of abnormal traffic in the total traffic on the safety factor.
[0068] For example, suppose an organization wants to assess the security risk factor of its computer data. K (threat probability): According to a recent security report, the score is 0.6 (on a scale of 0 to 1, 1 means very likely to occur). P (threat impact): If the data is leaked, it may cause significant financial losses and reputation damage, with a score of 0.8. L (system vulnerability level): After a security audit, it was found that the system has multiple unpatched vulnerabilities, with a score of 0.7. C (threat complexity): Attackers need certain technical capabilities to exploit these vulnerabilities, with a score of 0.5. Substitute these values into the formula:
[0069] Safety risk factor = (0.6×0.8)+(0.7×0.5)=0.48+0.35=0.83.
[0070] This result indicates that the organization has a high data security risk and needs to take steps to reduce the risk.
[0071] In this embodiment, it is specifically necessary to explain that S104 is to calculate the abnormal duration D between the monitoring start time and the monitoring end time. normal Monitor the test results of each day. i , by presetting a threshold, it is determined whether the monitoring result is abnormal. The calculation formula of the abnormal value is as follows:
[0072]
[0073] Among them, A i represents the abnormal value of the test result on the i-th day, Th represents the preset abnormal threshold, and the factor of abnormal duration is incorporated into the daily abnormal value calculation, and the formula for further adjusting the abnormal value is:
[0074]
[0075] Among them, 365 is the number of days in a year, which is used as a normalization factor. Otherwise means that when the actual observation value is less than or equal to the threshold, the abnormal value is 0. The abnormal value is matched with the preset abnormal threshold and the abnormal value is graded into normal, slightly abnormal, moderately abnormal and highly abnormal. The expression is as follows:
[0076]
[0077] Among them, Level i represents the abnormal level of the i-th day, Th mildIndicates mild abnormality, Th moderate Indicates moderate abnormality.
[0078] If there is an abnormal signal, a repair is generated based on the abnormal signal and the repair instruction is sent to the console to remind patrol personnel to maintain and repair data security. The early warning signal is a work after the early warning indicator is determined and the warning level of each omen is calculated. The safety early warning module uses a set of signs similar to red, yellow and green lights of traffic control signals to send different signals to the established indicator system and system risk status. Managers judge the severity of the risk by observing and analyzing the traffic lights.
[0079] Example 2
[0080] Reference Figure 2 This embodiment also provides a computer data security monitoring system based on big data technology, including:
[0081] Detection instruction generation module: Generates detection instructions through abnormal traffic monitoring and intrusion monitoring, and identifies anomalies based on security detection instructions;
[0082] Abnormal analysis module: the start time and the end time of the deviation of the average broadband from the average value are set as the start time and the end time, and the abnormal value is calculated;
[0083] Risk calculation module: calculates the security risk factor and safety factor of computer data;
[0084] Abnormal value monitoring module: monitors the abnormal duration of the monitoring start time and the monitoring end time, determines the abnormal values in the computer data, and matches them with the abnormal threshold to obtain the abnormal level.
[0085] It should be noted that the formulas involved in the above-mentioned are all dimensionless and numerical calculations. They are a formula closest to the actual situation obtained by collecting a large amount of data and performing software simulation. The proportional coefficients in the formula and the various preset thresholds in the analysis process are set by technical personnel in this field according to actual conditions or obtained by simulating a large amount of data.
[0086] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0087] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A computer data security monitoring method based on big data technology, characterized in that: include: S101, generating a detection instruction through abnormal traffic monitoring and intrusion monitoring, and identifying anomalies based on the detection instruction; S102, setting the start time and end time of the average broadband deviation from the average value as the start time point and the end time point, and calculating the abnormal value; S103, calculating the security risk factor and safety factor of computer data. The safety factor is calculated by obtaining the historical data LY of abnormal traffic in the network broadband, counting the total number of abnormal traffic corresponding to different detection conditions and marking it as YZ, and obtaining the type weight YQ of the abnormal traffic. The specific calculation formula of the safety factor is as follows: Among them, regulated represents the safety factor, YZ i represents the number of abnormal traffic of the ith i represents the corresponding type weight, LY represents historical data, n represents the number of abnormal traffic types, and T represents the adjustment coefficient. The specific calculation formula of the adjustment coefficient is as follows: Among them, the calculation of the adjustment coefficient reflects the influence of the weighted abnormal flow ratio in the total flow on the safety factor; S104, monitoring the abnormal duration of the start time point and the end time point, determining the abnormal value of the computer data, and matching it with the abnormal threshold to obtain the abnormal level; In S101, by acquiring abnormal traffic monitoring and intrusion detection in the network monitoring tool in the computer, a detection instruction is generated, and the detection instruction includes: a first detection instruction and a second detection instruction. The first detection instruction sets a normal traffic level based on historical data and defines a threshold value exceeding the level. The threshold value is obtained through bandwidth usage. The specific calculation formula of bandwidth usage is as follows: Where S represents the bandwidth usage, S 实 Indicates the actual bandwidth usage, S 总 Indicates the total available bandwidth. Statistical methods are used to identify abnormal behaviors. The standard deviation formula is used to identify abnormal values that deviate from the average value. The specific calculation formula is as follows: Among them, x i represents the i-th observation value, N represents the number of observations, and μ represents the average value. The second detection instruction ensures that all traffic is detected by configuring the IDS in the network node, and monitors in real time through the IDS, and regularly updates the signature database of the IDS to identify new threats and attack patterns; In the S102, the start time and end time of the average broadband deviation from the average value are set as the start time and the end time. Within the monitoring time period, the targets starting and ending in the monitoring area are monitored and evaluated. The start abnormal value and the end abnormal value within the monitoring time period are marked as start and ND respectively, and the corresponding time period weight value within the monitoring time period is obtained and marked as TE. The abnormal value is calculated. The specific calculation method is as follows: NM = (start + ND) × TE; Among them, NM represents the monitoring assessment value.
2. According to claim 1, a computer data security monitoring method based on big data technology is characterized in that: The abnormal traffic monitoring detects whether the average bandwidth in the network deviates from the average value. If there are abnormal values deviating from the average value in the detection area, a first detection instruction is generated. If there are no abnormal values deviating from the average value in the detection area, the first detection instruction is not generated. The intrusion detection detects the existence of connection attempts and frequent login failures in the monitoring area. If there are connection attempts and frequent login failures in the detection area, a second detection instruction is generated. If there are no connection attempts and frequent login failures in the detection area, the second detection instruction is not generated.
3. According to claim 1, a computer data security monitoring method based on big data technology is characterized in that: According to the abnormal value, an abnormal value monitoring set is generated, a capture instruction is generated according to the abnormal value monitoring set, and the corresponding abnormal traffic in the network bandwidth is extracted according to the capture instruction.
4. The computer data security monitoring method based on big data technology according to claim 1 is characterized in that: In S103, the security risk factor of the computer data is calculated and the intrusion sr , Virus InfectionVI sr , Information Theft sr And data leakage Pd sr , perform normalization processing to obtain the safety risk coefficient, the specific calculation formula of the safety risk coefficient is as follows: AF=w1·In sr +w2·VI sr +w3·It sr +w4·Pd sr ; Among them, AF represents the security risk factor, w1, w2, w3, and w4 represent weights, which are used to evaluate the risk value of each risk factor. The weight reflects the impact of the numerical values contained in the production process on computer data security.
5. The computer data security monitoring method based on big data technology according to claim 1 is characterized in that: In S104, the abnormal duration D of the start time point and the end time point is calculated. normal Monitor and calculate the monitoring results R for each day i , by presetting a threshold, it is determined whether the monitoring result is abnormal. The calculation formula of the abnormal value is as follows: Among them, A i represents the abnormal value of the test result on the i-th day, Th represents the preset abnormal threshold, and the factor of abnormal duration is incorporated into the daily abnormal value calculation, and the formula for further adjusting the abnormal value is: Among them, 365 is the number of days in a year, which is used as a normalization factor, and otherwise means that when the actual observation value is less than or equal to the threshold, the outlier value is 0.
6. The computer data security monitoring method based on big data technology according to claim 5 is characterized in that: The abnormal value is matched with the preset abnormal threshold, and the abnormal value is classified into normal, slightly abnormal, moderately abnormal and highly abnormal. The expression is as follows: Among them, Level i represents the abnormal level of the i-th day, Th mild Indicates mild abnormality, Th moderate Indicates moderate abnormality.
7. A computer data security monitoring system based on big data technology, applied to a computer data security monitoring method based on big data technology as described in any one of claims 1 to 6, characterized in that: include: Detection instruction generation module: Generates detection instructions through abnormal traffic monitoring and intrusion monitoring, and identifies anomalies based on security detection instructions; Abnormal analysis module: the start time and the end time of the deviation of the average broadband from the average value are set as the start time and the end time, and the abnormal value is calculated; Risk calculation module: calculates the security risk factor and safety factor of computer data; Abnormal value monitoring module: monitors the abnormal duration of the monitoring start time and the monitoring end time, determines the abnormal values in the computer data, and matches them with the abnormal threshold to obtain the abnormal level.
