A cloud-computing-based distributed network security monitoring system

Abstract

The application discloses a kind of distributed network security monitoring systems based on cloud computing, comprising: resource monitoring module: including resource monitoring unit and data collection unit;Dynamic scheduling module: including data processing unit;Load balancing module;Automatic expansion and reduction module;Fault detection and automatic migration module.The application can identify the bottleneck and idle state of resource utilization by monitoring and calculating the resource usage of each virtual machine and server, thereby dynamically optimizing resource allocation.This approach can maximize the use efficiency of resources, avoid waste or insufficient resources, and ensure that the system can still operate stably under high load or sudden network events, avoiding performance degradation or security risks caused by improper resource allocation, and ensuring the safe operation of the network security monitoring system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer network security, and particularly relates to a distributed network security monitoring system based on cloud computing. BACKGROUND

[0002] With the rapid development of information technology, cloud computing technology has become the core of modern computing infrastructure. Cloud computing can provide flexible and efficient resource scheduling and allocation through virtualization technology, thereby supporting various complex application scenarios. Due to its efficient resource utilization and on-demand expansion capability, more and more enterprises and organizations choose to migrate their critical business and applications to the cloud environment. However, as the cloud computing environment continues to expand, network security problems have become increasingly prominent.

[0003] The existing distributed network security monitoring system usually relies on preset rules and static resource allocation strategies, which may lead to low resource utilization efficiency and even cause system bottlenecks and security vulnerabilities when facing sudden network events or load surges. Therefore, how to develop a network security monitoring system capable of real-time monitoring and optimizing resource allocation based on dynamic resource scheduling technology in the cloud computing environment has become the focus of attention for those skilled in the art. SUMMARY

[0004] This section aims to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, abstract and title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] In a first aspect, the present application provides a distributed network security monitoring system based on cloud computing, comprising:

[0007] A resource monitoring module: including a resource monitoring unit and a data collection unit, for real-time monitoring of resource usage of each virtual machine and server in the cloud computing environment, and periodically reporting resource status to the central monitoring node;

[0008] A dynamic scheduling module: including a data processing unit, in communication with the resource monitoring module, based on historical data and real-time resource status, calculating resource allocation efficiency through a dynamic resource scheduling algorithm, and adjusting resource allocation of virtual machines and servers in real time according to the calculated results and corresponding value range, to optimize resource utilization;

[0009] A load balancing module: for balancing network traffic and computing tasks, dynamically adjusting traffic allocation strategies to avoid overloading of individual servers and virtual machines;

[0010] Automatic expansion and reduction module: for automatically expanding and reducing resources when detecting that resource usage reaches a preset threshold to cope with changes in load;

[0011] Fault detection and automatic migration module: for monitoring the running state of each node in real time, and automatically migrating the affected tasks and traffic to healthy nodes when detecting faults and resource bottlenecks.

[0012] As a preferred scheme of the distributed network security monitoring system based on cloud computing, the dynamic resource scheduling algorithm is specifically calculated as follows:

[0013]

[0014] The resource scheduling optimization function is used to evaluate the matching degree between resource usage and task demand at time t. The resource usage optimization function is used to evaluate the resource allocation efficiency of each virtual machine and server at time t. The task demand distribution function is used to measure the resource demand of all tasks.

[0015] As a preferred scheme of the distributed network security monitoring system based on cloud computing, the resource optimization function The specific calculation formula is as follows:

[0016]

[0017] Where x i (t) represents the resource usage; μ i is the average resource usage; and σ i is the standard deviation of resource usage.

[0018] As a preferred scheme of the distributed network security monitoring system based on cloud computing, the task demand distribution function The specific calculation formula is as follows:

[0019]

[0020] Where α j and β j are distribution parameters.

[0021] As a preferred scheme of the distributed network security monitoring system based on cloud computing, the The corresponding value range is specifically as follows:

[0022] 0, which means that the resource utilization is extremely low between time t0 and t, and there is serious resource idling and scheduling failure;

[0023] The resource utilization efficiency is lower than the task demand, indicating that the system has a bottleneck in task processing;

[0024] The resource allocation and task demand are balanced, and the system is in an ideal state;

[0025] The resource utilization efficiency is higher than the task demand, indicating that the system is running efficiently, but there is a situation of resource surplus;

[0026] The system resource utilization efficiency is extremely high, the task demand is low, and the resource utilization is excessively optimized.

[0027] As a preferred scheme of the distributed network security monitoring system based on cloud computing, wherein: the The corresponding value range is specifically operated as follows:

[0028] Re-evaluate the current resource configuration, confirm whether the resource distribution is reasonable, adjust the resource allocation strategy to avoid idling, and check whether the system has hardware failure and network bottleneck, and determine whether the fault detection and automatic migration module needs to be started;

[0029] Enable the automatic expansion module to increase more computing resources and bandwidth for high-demand tasks to ensure that critical tasks have priority to obtain resources;

[0030] Maintain the current resource allocation and scheduling strategy, continue to monitor the system to ensure that the balanced state continues, and make slight adjustments to load balancing according to real-time changes to maintain the best state;

[0031] Start the automatic reduction module to moderately reduce unnecessary resources to avoid waste, redistribute resources, and allocate excess resources to other low-priority tasks;

[0032] Confirm whether there is a problem of insufficient task demand, if so, the priority of the task needs to be adjusted and non-urgent tasks are postponed, and the resource recycling strategy is further optimized to ensure reasonable use of resources and avoid long-term idling and waste.

[0033] As a preferred embodiment of the distributed network security monitoring system based on cloud computing described in this invention, the system further includes a global resource scheduling and optimization platform for centrally managing all resources in the cloud environment and making optimal scheduling decisions from a global perspective.

[0034] As a preferred embodiment of the cloud-based distributed network security monitoring system described in this invention, the global resource scheduling and optimization platform includes a visual interface, which facilitates system administrators to monitor resource usage in real time, manually intervene in scheduling strategies, and view historical performance data and trend analysis reports of the system.

[0035] Secondly, to further address the problems existing in network security monitoring, the present invention provides a distributed network security monitoring method based on cloud computing, specifically including:

[0036] S1. Deploy a resource monitoring module in the cloud computing environment to monitor the resource usage of each virtual machine and server in real time, such as CPU, memory, storage and network bandwidth. The resource monitoring module periodically reports the current resource status to the central control node.

[0037] S2. Calculate the corresponding information based on the information provided by the resource monitoring module. And according to The specific results and their corresponding value range are used to evaluate the current resource utilization status;

[0038] S3, according to The obtained resource utilization status is used to dynamically adjust resource allocation to ensure the rational distribution of resources;

[0039] S4. The system monitors resource usage in real time through a global resource scheduling and optimization platform, and allows manual intervention in scheduling strategies, as well as viewing historical performance data and trend analysis reports.

[0040] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, runs a distributed network security monitoring system based on cloud computing as described in the first aspect of the present invention.

[0041] The beneficial effects of this invention are:

[0042] By monitoring and calculating the resource usage of each virtual machine and server in real time, the system can identify resource bottlenecks and idle states, thereby dynamically optimizing resource allocation. This approach can maximize resource utilization efficiency, avoid resource waste or shortage, and ensure that the system can still operate stably under high load or sudden network events, avoiding performance degradation or security risks caused by improper resource allocation, thus guaranteeing the secure operation of the network security monitoring system. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0044] Fig. 1 This is a system block diagram of a distributed network security monitoring system based on cloud computing proposed in this invention;

[0045] Fig. 2 This is a flowchart of a distributed network security monitoring method based on cloud computing proposed in this invention;

[0046] Fig. 3 This is a flowchart for determining the resource utilization rate of the present invention;

[0047] Fig. 4 This is a comparison chart of resource utilization and task completion time in Embodiment 2 of the present invention;

[0048] Fig. 5 This is a comparison chart of CPU utilization fluctuation range and network latency changes in Embodiment 2 of the present invention. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0051] Second, the "one embodiment" or "an embodiment" referred to herein means a specific feature, structure, characteristic, or combination of features and characteristics described herein that can be included in at least one implementation of the application. The various appearances of "in one embodiment" or "in an embodiment" in the specification do not all refer to the same embodiment, although they can.

[0052] Embodiment 1

[0053] Referring to Figs. 1-3 The application provides a cloud computing-based distributed network security monitoring system, comprising:

[0054] A resource monitoring module, including a resource monitoring unit and a data collection unit, for real-time monitoring of resource usage of each virtual machine and server in the cloud computing environment, and periodically reporting resource status to the central monitoring node;

[0055] A dynamic scheduling module, including a data processing unit, in communication with the resource monitoring module, based on historical data and real-time resource status, calculating resource allocation efficiency through a dynamic resource scheduling algorithm, and adjusting resource allocation of virtual machines and servers in real time according to the calculated results and corresponding value range, to optimize resource utilization;

[0056] A load balancing module for balancing network traffic and computing tasks, dynamically adjusting traffic distribution strategies to avoid overloading of individual servers and virtual machines;

[0057] An automatic expansion and reduction module for automatically expanding and reducing resources when detecting that resource usage reaches a preset threshold, to cope with changes in load;

[0058] A fault detection and automatic migration module for real-time monitoring of the running state of each node, and automatically migrating affected tasks and traffic to healthy nodes when detecting faults and resource bottlenecks.

[0059] The dynamic resource scheduling algorithm is specifically calculated as follows:

[0060]

[0061] is a resource scheduling optimization function, used to evaluate the matching degree between resource usage and task demand at time t; is a resource usage optimization function, used to evaluate the resource allocation efficiency of each virtual machine and server at time t; is a task demand distribution function, used to measure the resource demand of all tasks.

[0062] Further, the resource optimization function The specific calculation formula is as follows:

[0063]

[0064] where x i (t) denotes the resource usage; μ i is the average resource usage; σ i is the standard deviation of the resource usage, denotes the summation of the resource utilization of all N virtual machines or servers, φ i (t) denotes the dynamic weight function of virtual machine i at time t.

[0065] Further, the task demand distribution function The specific calculation formula is as follows:

[0066]

[0067] where α j and β j are distribution parameters, denotes the demand distribution of each task j, which is modeled using the Beat distribution function, where Γ is the Gamma function.

[0068] Further, The corresponding value range is specifically as follows:

[0069] is 0, indicating that the resource utilization is extremely low between time t0 and t, and there is serious resource idling and scheduling failure;

[0070] The resource utilization efficiency is lower than the task demand, indicating that the system has a bottleneck in task processing;

[0071] The resource allocation and task demand are balanced, and the system is in an ideal state;

[0072] The resource utilization efficiency is higher than the task demand, indicating that the system is running efficiently, but there is a situation of resource surplus;

[0073] The system resource utilization efficiency is extremely high, the task demand is low, and the resource utilization is excessively optimized, and the calculation result of is used to judge the current resource utilization.

[0074] Further, The corresponding value range is specifically as follows:

[0075] Reassess the current resource configuration, confirm whether the resources are distributed reasonably, adjust the resource allocation strategy to avoid idleness, and check whether there are hardware failures or network bottlenecks in the system, and determine whether it is necessary to start the fault detection and automatic migration module.

[0076] Enable the automatic scaling module to add more computing resources and bandwidth for high-demand tasks, ensuring that critical tasks receive resources first.

[0077] Maintain the current resource allocation and scheduling strategy, continue to monitor the system to ensure the balance remains intact, and make slight adjustments to the load balancer based on real-time changes to maintain the optimal state.

[0078] Activate the automatic reduction module to appropriately reduce unnecessary resources to avoid waste, and reallocate resources, allocating excess resources to other low-priority tasks;

[0079] Confirm whether there is insufficient task demand. If so, adjust task priorities and postpone non-urgent tasks, and further optimize resource reclamation strategies to ensure reasonable resource use and avoid prolonged idle waste. The resource monitoring module should continuously monitor resource usage across all value ranges to ensure accurate data input; the dynamic scheduling module should... In such cases, resource scheduling needs to be adjusted more frequently to cope with system changes; At times, the adjustment frequency can be appropriately reduced; the load balancing module in When the value approaches 1, the strategy of this module should remain stable; if This necessitates a reassessment of the load strategy to ensure reasonable resource distribution; the automatic scaling up and down module should be used in... When this happens, the expansion mechanism is activated to increase resources; At that time, the reduction mechanism is activated to reduce unnecessary resources; the fault detection and automatic migration module mainly... It is activated in a timely manner to ensure that possible failures can be handled promptly in extreme situations.

[0080] Furthermore, it also includes a global resource scheduling and optimization platform, which is used to centrally manage all resources in the cloud environment and make optimal scheduling decisions from a global perspective. The resource scheduling and optimization platform makes it easier for staff to manage resources.

[0081] Furthermore, the global resource scheduling and optimization platform includes a visual interface, which allows system administrators to monitor resource usage in real time, manually intervene in scheduling strategies, and view historical performance data and trend analysis reports.

[0082] The embodiment also provides a distributed network security monitoring method based on cloud computing, and specifically comprises the following steps:

[0083] S1, deploying a resource monitoring module in a cloud computing environment, monitoring resource usage of each virtual machine and server in real time, such as CPU, memory, storage and network bandwidth, the resource monitoring module periodically reports the current resource state to a central control node;

[0084] S2, calculating corresponding according to the information provided by the resource monitoring module; and judging the current resource utilization state according to the specific result and the corresponding value range thereof;

[0085] S3, dynamically adjusting resource allocation according to obtained resource utilization state, to ensure reasonable distribution of resources;

[0086] S4, monitoring resource usage in real time through a global resource scheduling and optimization platform, manually intervening in the scheduling strategy, and viewing historical performance data and trend analysis reports of the system.

[0087] The embodiment also provides a computer device suitable for a distributed network security monitoring system based on cloud computing, comprising a memory and a processor; the memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions to run the distributed network security monitoring system based on cloud computing.

[0088] Embodiment 2

[0089] Referring to Figs. 4-5 , the second embodiment of the present application is different from the first embodiment in that, in order to verify the beneficial effects, experimental comparison data of the present application and prior art are provided.

[0090] In this embodiment, a cloud computing environment containing 10 servers and 50 virtual machines is built. In this environment, the processing capabilities of the servers are different, and the virtual machine loads simulate various situations in actual use, including tasks with high computing requirements (such as data encryption, deep learning model training) and tasks with low computing requirements (such as log monitoring, data backup). At the same time, we installed a resource monitoring module on each server and virtual machine, which can record the usage of CPU, memory and network bandwidth in real time and report the data to the central server.

[0091] We compared the performance of the existing static resource allocation strategy and the dynamic resource scheduling algorithm of the present invention. In the static resource allocation strategy, the resources of each server and virtual machine are pre-allocated and do not change with the tasks. In the method of the present invention, the system calculates the resource allocation efficiency of each node in real time and dynamically adjusts it according to the efficiency value range to optimize the resource utilization efficiency. We evaluated the performance of the two strategies by monitoring and recording the resource usage, task completion time, and system load changes of each node.

[0092]

[0093]

[0094] By comparing the data in the table, we can clearly see that the dynamic resource scheduling strategy of the present invention is significantly better than the traditional static resource allocation strategy in terms of resource utilization, task completion time, and system stability. In the low load case, although the static allocation strategy can guarantee basic system stability, the resource utilization is only 60%, while the dynamic scheduling strategy of the present invention improves this value to 85% and shortens the task completion time by 33%.

[0095] In the medium and high load cases, the advantages of the present invention are more obvious. The static allocation strategy cannot dynamically adjust resources according to actual needs, resulting in a significant decrease in resource utilization and a serious impairment of system stability under high load, with CPU utilization fluctuating in a range of ±30% and network latency increasing significantly. The present invention, through real-time monitoring and scheduling, not only maintains a high level of resource utilization (80% in medium load and 70% in high load), but also significantly reduces the impact of system load fluctuations on performance, significantly improving system stability.

[0096] Overall, the dynamic resource scheduling algorithm of the present invention can significantly improve the performance and stability of the system under various load conditions by optimizing resource utilization, especially in high load cases.

[0097] It should be noted that the above examples are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced equivalently without departing from the spirit and scope of the present invention, and all should be included in the scope of the claims of the present invention.

Claims

1. A cloud computing-based distributed network security monitoring system, characterized in that: Comprise: Resource monitoring module: including resource monitoring unit and data collection unit, for real-time monitoring of the resource usage of each virtual machine and server in the cloud computing environment, and periodically report resource status to the central monitoring node; Dynamic scheduling module: including data processing unit, in communication with the resource monitoring module, based on historical data and real-time resource status, through dynamic resource scheduling algorithm to calculate resource allocation efficiency, and according to the calculated results and corresponding value range real-time adjustment of virtual machine and server resource allocation, to optimize resource utilization; Load balancing module: for balancing distribution of network traffic and computing tasks, dynamically adjusting traffic allocation strategy to avoid single server and virtual machine overload; Automatic expansion and reduction module: for automatic expansion and reduction of resources when detecting resource usage reaches the preset threshold, to cope with the change of load; Fault detection and automatic migration module: for real-time monitoring of the running state of each node, and automatically migrating the affected tasks and traffic to the healthy node when detecting fault and resource bottleneck; The dynamic resource scheduling algorithm specific formula is as follows: is a resource scheduling optimization function for evaluating the matching degree between resource usage and task demand at a time ; is a resource usage optimization function for evaluating the resource allocation efficiency of each virtual machine and server at a time ; is a task demand distribution function for measuring the resource demand of all tasks; The The corresponding value ranges are as follows: , is 0, which means that the resource utilization is very low between time and , and there is serious resource idling and scheduling failure; , resource utilization efficiency is lower than the task demand, indicating that the system has bottleneck in task processing; , resource allocation and task requirements are balanced, the system is in an ideal state; , resource utilization efficiency is higher than the task demand, indicating that the system is running efficiently, but there is a surplus of resources; , system resource utilization efficiency is extremely high, the task demand is low and the resource utilization is excessively optimized; The The corresponding value domain is specifically operated as follows: Re-evaluate the current resource configuration, confirm whether the resource distribution is reasonable, adjust the resource allocation strategy to avoid idling, and check whether there is a hardware failure and network bottleneck in the system, and determine whether to start the fault detection and automatic migration module; , enabling an auto-scaling module to add more computing resources and bandwidth for high-demand tasks and ensuring that critical tasks are prioritized for resources; , maintain the current resource allocation and scheduling strategy, continue to monitor the system to ensure that the balance state continues, and make slight adjustments to load balancing according to real-time changes to maintain the best state; , start the automatic reduction module, reduce unnecessary resources moderately, avoid waste, reallocate resources, and allocate excess resources to other low-priority tasks; Confirm if there is a task demand problem, if yes, the priority of the task needs to be adjusted and non-urgent tasks are postponed, and the resource recovery strategy is further optimized to ensure reasonable use of resources and avoid long-term idle waste.

2. The cloud computing based distributed network security monitoring system according to claim 1, wherein: The resource usage optimization function The specific calculation formula is as follows: wherein represents the resource usage amount; is the average resource usage amount; is the standard deviation of the resource usage amount.

3. The cloud computing based distributed network security monitoring system as claimed in claim 2, wherein: The task demand distribution function The specific calculation formula is as follows: wherein and are distribution parameters.

4. The cloud-computing-based distributed network security monitoring system according to claim 3, characterized in that: Also includes global resource scheduling and optimization platform, for centralized management of all resources in the cloud environment, and makes optimal scheduling decision according to the global perspective.

5. The cloud computing based distributed network security monitoring system as claimed in claim 4, wherein: The global resource scheduling and optimization platform includes a visual interface, which facilitates system administrators to real-time monitor resource usage, manually intervene scheduling strategy, and view historical performance data and trend analysis report of the system.

6. A cloud computing-based distributed network security monitoring method, based on the cloud computing-based distributed network security monitoring system of any one of claims 1-5, characterized in that: Specifically comprising: S1, in the cloud computing environment, deploy resource monitoring module, real-time monitoring of each virtual machine and server CPU, memory, storage and network bandwidth and other resource usage, resource monitoring module periodically reports the current resource status to the central control node; S2, according to the resource monitoring module to provide information corresponding to the calculation , and according to The specific results and their corresponding value range of the current resource utilization state are judged; S3、According to The obtained resource utilization state dynamically adjusts the resource allocation, ensuring reasonable distribution of resources; S4, through the global resource scheduling and optimization platform to real-time monitoring of resource usage, and can manually intervene scheduling strategy, and view historical performance data and trend analysis report of the system. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the cloud computing based distributed network security monitoring system in claims 1-5.

Images