Distributed network security monitoring system based on cloud computing

Abstract

The invention discloses a distributed network security monitoring system based on cloud computing. The system comprises a resource monitoring module which comprises a resource monitoring unit and a data collection unit; the dynamic scheduling module comprises a data processing unit; a load balancing module; an automatic expansion and reduction module; and a fault detection and automatic migration module. By monitoring and calculating the resource use condition of each virtual machine and server, the system can identify the bottleneck and idle state of resource utilization, thereby dynamically optimizing the resource allocation, the use efficiency of resources can be improved to the greatest extent, the problem of resource waste or insufficiency is avoided, and the resource utilization efficiency is improved. Meanwhile, under the condition of high load or sudden network events, the system can still operate stably, performance reduction or potential safety hazards caused by improper resource allocation are avoided, and safe operation of the network security monitoring system is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of computer network security, and in particular to a distributed network security monitoring system based on cloud computing. Background Art

[0002] With the rapid development of information technology, cloud computing technology has become the core of modern computing infrastructure. Cloud computing can provide flexible and efficient resource scheduling and allocation through virtualization technology, thereby supporting various complex application scenarios. Due to its efficient resource utilization and on-demand expansion capabilities, more and more enterprises and organizations choose to migrate their key businesses and applications to the cloud environment. However, with the continuous expansion of the cloud computing environment, network security issues have become increasingly prominent.

[0003] Existing distributed network security monitoring systems usually rely on preset rules and static resource allocation strategies, which may lead to inefficient resource utilization and even cause system bottlenecks and security vulnerabilities when facing sudden network events or sudden load increases. Therefore, how to develop a network security monitoring system that can monitor and optimize resource allocation in real time based on dynamic resource scheduling technology in a cloud computing environment has become the focus of technical personnel in this field. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a distributed network security monitoring system based on cloud computing, comprising:

[0007] Resource monitoring module: including resource monitoring unit and data collection unit, used to monitor the resource usage of each virtual machine and server in the cloud computing environment in real time, and regularly report the resource status to the central monitoring node;

[0008] Dynamic scheduling module: including a data processing unit, communicating with the resource monitoring module, calculating resource allocation efficiency through a dynamic resource scheduling algorithm based on historical data and real-time resource status, and adjusting resource allocation of virtual machines and servers in real time according to the calculated results and corresponding value ranges to optimize resource utilization;

[0009] Load balancing module: used to evenly distribute network traffic and computing tasks, and dynamically adjust traffic distribution strategies to avoid overloading of a single server and virtual machine;

[0010] Automatic expansion and reduction module: used to automatically expand and reduce resources when it detects that resource usage reaches a preset threshold to cope with changes in load;

[0011] Fault detection and automatic migration module: used to monitor the operating status of each node in real time. When faults and resource bottlenecks are detected, the affected tasks and traffic are automatically migrated to healthy nodes.

[0012] As a preferred solution of the distributed network security monitoring system based on cloud computing described in the present invention, the specific calculation formula of the dynamic resource scheduling algorithm is as follows:

[0013]

[0014] is a resource scheduling optimization function used to evaluate the matching degree between resource usage and task requirements at time t; is a resource utilization optimization function used to evaluate the resource allocation efficiency of each virtual machine and server at time t; is the task requirement distribution function, which is used to measure the resource requirements of all tasks.

[0015] As a preferred solution of the distributed network security monitoring system based on cloud computing described in the present invention, wherein: the resource optimization function The specific calculation formula is as follows:

[0016]

[0017] where x i (t) represents the resource usage; μ i is the average resource usage; σ i is the standard deviation of resource usage.

[0018] As a preferred solution of the distributed network security monitoring system based on cloud computing described in the present invention, the task demand distribution function The specific calculation formula is as follows:

[0019]

[0020] where α j and β j is the distribution parameter.

[0021] As a preferred solution of the distributed network security monitoring system based on cloud computing described in the present invention, wherein: The corresponding value ranges are as follows:

[0022] If it is 0, it means that the resource utilization rate is extremely low between time t0 and t, and there are serious resource idleness and scheduling errors;

[0023] Resource utilization efficiency is lower than task requirements, indicating that the system has a bottleneck in task processing;

[0024] Resource allocation and task requirements are relatively balanced, and the system is in an ideal state;

[0025] Resource utilization efficiency is higher than task requirements, indicating that the system is operating efficiently, but there is excess resources;

[0026] The system resource utilization is extremely efficient, the task demand is low and the resource utilization is over-optimized.

[0027] As a preferred solution of the distributed network security monitoring system based on cloud computing described in the present invention, wherein: The specific operations for the corresponding value range are as follows:

[0028] Re-evaluate the current resource configuration, confirm whether the resources are reasonably distributed, adjust the resource allocation strategy to avoid idleness, and check whether the system has hardware failures and network bottlenecks, and determine whether the fault detection and automatic migration module needs to be started;

[0029] Enable the automatic expansion module to add more computing resources and bandwidth for high-demand tasks, ensuring that critical tasks get resources first;

[0030] Maintain the current resource allocation and scheduling strategy, continue to monitor the system to ensure that the balance state persists, and make slight adjustments to the load balancing according to real-time changes to maintain the optimal state;

[0031] Start the automatic reduction module to appropriately reduce unnecessary resources to avoid waste, reallocate resources, and allocate excess resources to other low-priority tasks;

[0032] Confirm whether there is a problem of insufficient task demand. If so, it is necessary to adjust the task priority and postpone non-urgent tasks, and further optimize the resource recovery strategy to ensure the rational use of resources and avoid long-term idle waste.

[0033] As a preferred solution of the distributed network security monitoring system based on cloud computing described in the present invention, it also includes a global resource scheduling and optimization platform for centrally managing all resources in the cloud environment and making optimal scheduling decisions based on a global perspective.

[0034] As a preferred solution of the distributed network security monitoring system based on cloud computing described in the present invention, the global resource scheduling and optimization platform includes a visual interface, which is convenient for system administrators to monitor resource usage in real time, manually intervene in scheduling strategies, and view the system's historical performance data and trend analysis reports.

[0035] In a second aspect, in order to further solve the problems existing in network security monitoring, the present invention provides a distributed network security monitoring method based on cloud computing, which specifically includes:

[0036] S1. Deploy a resource monitoring module in the cloud computing environment to monitor the CPU, memory, storage, network bandwidth and other resource usage of each virtual machine and server in real time. The resource monitoring module regularly reports the current resource status to the central control node;

[0037] S2. Calculate the corresponding information based on the information provided by the resource monitoring module And according to The specific results and their corresponding value ranges are used to judge the current resource utilization status;

[0038] S3. According to The resource utilization status obtained is used to dynamically adjust resource allocation to ensure the reasonable distribution of resources;

[0039] S4. Real-time monitoring of resource usage is performed through the global resource scheduling and optimization platform. Scheduling strategies can be manually intervened, and historical performance data and trend analysis reports of the system can be viewed.

[0040] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it runs a distributed network security monitoring system based on cloud computing as described in the first aspect of the present invention.

[0041] Beneficial effects of the present invention:

[0042] By real-time monitoring and calculating the resource usage of each virtual machine and server, the system can identify bottlenecks and idle states in resource utilization, thereby dynamically optimizing resource allocation. This approach can maximize resource utilization efficiency and avoid resource waste or shortage. At the same time, it ensures that the system can still operate stably under high load or sudden network events, avoiding performance degradation or security risks caused by improper resource allocation, thereby ensuring the safe operation of the network security monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0044] Figure 1 A system block diagram of a distributed network security monitoring system based on cloud computing proposed by the present invention;

[0045] Figure 2 A flow chart of a distributed network security monitoring method based on cloud computing proposed by the present invention;

[0046] Figure 3 It is a flow chart of resource utilization determination of the present invention;

[0047] Figure 4 A comparison chart of resource utilization and task completion time in Example 2 of the present invention;

[0048] Figure 5 This is a comparison chart of the CPU utilization fluctuation range and network delay changes in Example 2 of the present invention. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0051] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0052] Example 1

[0053] Reference Figure 1-3 The present invention provides a distributed network security monitoring system based on cloud computing, comprising:

[0054] Resource monitoring module: including resource monitoring unit and data collection unit, used to monitor the resource usage of each virtual machine and server in the cloud computing environment in real time, and regularly report the resource status to the central monitoring node;

[0055] Dynamic scheduling module: includes a data processing unit, which communicates with the resource monitoring module, calculates the resource allocation efficiency through a dynamic resource scheduling algorithm based on historical data and real-time resource status, and adjusts the resource allocation of virtual machines and servers in real time according to the calculated results and the corresponding value range to optimize resource utilization;

[0056] Load balancing module: used to evenly distribute network traffic and computing tasks, and dynamically adjust traffic distribution strategies to avoid overloading of a single server and virtual machine;

[0057] Automatic expansion and reduction module: used to automatically expand and reduce resources when it detects that resource usage reaches a preset threshold to cope with changes in load;

[0058] Fault detection and automatic migration module: used to monitor the operating status of each node in real time. When faults and resource bottlenecks are detected, the affected tasks and traffic are automatically migrated to healthy nodes.

[0059] The specific calculation formula of the dynamic resource scheduling algorithm is as follows:

[0060]

[0061] is a resource scheduling optimization function used to evaluate the matching degree between resource usage and task requirements at time t; is a resource utilization optimization function used to evaluate the resource allocation efficiency of each virtual machine and server at time t; is the task requirement distribution function, which is used to measure the resource requirements of all tasks.

[0062] Further, resource optimization function The specific calculation formula is as follows:

[0063]

[0064] where x i (t) represents the resource usage; μ i is the average resource usage; σ i is the standard deviation of resource usage, represents the sum of resource utilization of all N virtual machines or servers, φ i (t) represents the dynamic weight function of virtual machine i at time t.

[0065] Furthermore, the task demand distribution function The specific calculation formula is as follows:

[0066]

[0067] where α j and β j is the distribution parameter, represents the demand distribution of each task j, which is modeled using the Beat distribution function, where Γ is the Gamma function.

[0068] Furthermore, The corresponding value ranges are as follows:

[0069] If it is 0, it means that the resource utilization rate is extremely low between time t0 and t, and there are serious resource idleness and scheduling errors;

[0070] Resource utilization efficiency is lower than task requirements, indicating that the system has a bottleneck in task processing;

[0071] Resource allocation and task requirements are relatively balanced, and the system is in an ideal state;

[0072] Resource utilization efficiency is higher than task requirements, indicating that the system is operating efficiently, but there is excess resources;

[0073] The system resource utilization efficiency is extremely high, the task demand is low and the resource utilization is over-optimized. The current resource utilization is judged based on the calculation results.

[0074] Furthermore, The specific operations for the corresponding value range are as follows:

[0075] Re-evaluate the current resource configuration, confirm whether the resources are reasonably distributed, adjust the resource allocation strategy to avoid idleness, and check whether the system has hardware failures and network bottlenecks, and determine whether it is necessary to start the fault detection and automatic migration module;

[0076] Enable the automatic expansion module to add more computing resources and bandwidth for high-demand tasks, ensuring that critical tasks get resources first;

[0077] Maintain the current resource allocation and scheduling strategy, continue to monitor the system to ensure that the balance state persists, and make slight adjustments to the load balancing according to real-time changes to maintain the optimal state;

[0078] Start the automatic reduction module to appropriately reduce unnecessary resources to avoid waste, reallocate resources, and allocate excess resources to other low-priority tasks;

[0079] Confirm whether there is a problem of insufficient task demand. If so, it is necessary to adjust the task priority and postpone non-urgent tasks, and further optimize the resource recovery strategy to ensure the rational use of resources and avoid long-term idle waste. The resource monitoring module must continuously monitor resource usage in any value range to ensure accurate data input; the dynamic scheduling module In the case of , resource scheduling needs to be adjusted more frequently to cope with system changes; When the load balancing module is When it is close to 1, the strategy of this module should remain stable; if You need to re-evaluate the load strategy to ensure that resources are properly distributed; the automatic expansion and reduction modules are When , the expansion mechanism is started to increase resources; When the system is running, the reduction mechanism is started to reduce unnecessary resources; the fault detection and automatic migration module is mainly used in to ensure that possible failures can be handled in a timely manner in extreme cases.

[0080] Furthermore, it also includes a global resource scheduling and optimization platform, which is used to centrally manage all resources in the cloud environment and make optimal scheduling decisions based on a global perspective. The resource scheduling and optimization platform makes management more convenient for staff.

[0081] Furthermore, the global resource scheduling and optimization platform includes a visual interface that allows system administrators to monitor resource usage in real time, manually intervene in scheduling strategies, and view historical system performance data and trend analysis reports.

[0082] This embodiment also provides a distributed network security monitoring method based on cloud computing, which specifically includes:

[0083] S1. Deploy a resource monitoring module in the cloud computing environment to monitor the CPU, memory, storage, network bandwidth and other resource usage of each virtual machine and server in real time. The resource monitoring module regularly reports the current resource status to the central control node;

[0084] S2. Calculate the corresponding information based on the information provided by the resource monitoring module And according to The specific results and their corresponding value ranges are used to judge the current resource utilization status;

[0085] S3. According to The resource utilization status obtained is used to dynamically adjust resource allocation to ensure the reasonable distribution of resources;

[0086] S4. Real-time monitoring of resource usage is performed through the global resource scheduling and optimization platform. Scheduling strategies can be manually intervened, and historical performance data and trend analysis reports of the system can be viewed.

[0087] This embodiment also provides a computer device suitable for a distributed network security monitoring system based on cloud computing, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to run the above distributed network security monitoring system based on cloud computing.

[0088] Example 2

[0089] Reference Figure 4-Figure 5 , which is the second embodiment of the present invention. This embodiment is different from the first embodiment in that, in order to verify its beneficial effects, experimental comparison data between the present invention and the prior art are provided.

[0090] This example builds a cloud computing environment with 10 servers and 50 virtual machines. In this environment, the processing capabilities of the servers vary, and the virtual machine load simulates a variety of situations in actual use, including tasks with high computing requirements (such as data encryption and deep learning model training) and tasks with low computing requirements (such as log monitoring and data backup). At the same time, we install a resource monitoring module on each server and virtual machine, which can record the usage of CPU, memory, and network bandwidth in real time and report the data to the central server.

[0091] We used the existing static resource allocation strategy and the dynamic resource scheduling algorithm of the present invention for comparison. In the static resource allocation strategy, the resources of each server and virtual machine are pre-allocated and will not be adjusted as the task changes. In the method of the present invention, the system calculates the resource allocation efficiency of each node in real time and dynamically adjusts it according to the efficiency value range to optimize the resource utilization efficiency. We evaluated the performance of these two strategies by monitoring and recording the resource usage of each node, task completion time, and system load changes.

[0092]

[0093]

[0094] By comparing the data in the table, we can clearly see that the dynamic resource scheduling strategy of the present invention is significantly better than the traditional static resource allocation strategy in terms of resource utilization, task completion time and system stability. Under low load conditions, although the static allocation strategy can ensure basic system stability, the resource utilization rate is only 60%, while the dynamic scheduling strategy of the present invention increases this value to 85% and shortens the task completion time by 33%.

[0095] The advantages of the present invention are more obvious under medium and high load conditions. The static allocation strategy cannot dynamically adjust resources according to actual needs, resulting in a significant decrease in resource utilization, serious damage to the stability of the system under high load, CPU utilization fluctuation range as high as ±30%, and a significant increase in network latency. The present invention, through real-time monitoring and scheduling, not only maintains resource utilization at a high level (80% under medium load and 70% under high load), but also greatly reduces the impact of system load fluctuations on performance, significantly improving the stability of the system.

[0096] In general, the dynamic resource scheduling algorithm of the present invention can significantly improve the performance and stability of the system under various load conditions by optimizing resource utilization efficiency, especially under high load conditions, its advantages are particularly prominent.

[0097] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A distributed network security monitoring system based on cloud computing, characterized by: include: Resource monitoring module: including resource monitoring unit and data collection unit, used to monitor the resource usage of each virtual machine and server in the cloud computing environment in real time, and regularly report the resource status to the central monitoring node; Dynamic scheduling module: including a data processing unit, communicating with the resource monitoring module, calculating resource allocation efficiency through a dynamic resource scheduling algorithm based on historical data and real-time resource status, and adjusting resource allocation of virtual machines and servers in real time according to the calculated results and corresponding value ranges to optimize resource utilization; Load balancing module: used to evenly distribute network traffic and computing tasks, and dynamically adjust traffic distribution strategies to avoid overloading of a single server and virtual machine; Automatic expansion and reduction module: used to automatically expand and reduce resources when it detects that resource usage reaches a preset threshold to cope with changes in load; Fault detection and automatic migration module: used to monitor the operating status of each node in real time. When faults and resource bottlenecks are detected, the affected tasks and traffic are automatically migrated to healthy nodes.

2. A distributed network security monitoring system based on cloud computing according to claim 1, characterized in that: The specific calculation formula of the dynamic resource scheduling algorithm is as follows: is a resource scheduling optimization function used to evaluate the matching degree between resource usage and task requirements at time t; is a resource utilization optimization function used to evaluate the resource allocation efficiency of each virtual machine and server at time t; is the task requirement distribution function, which is used to measure the resource requirements of all tasks.

3. The distributed network security monitoring system based on cloud computing according to claim 2 is characterized in that: The resource optimization function The specific calculation formula is as follows: where x i (t) represents the resource usage; μ i is the average resource usage; σ i is the standard deviation of resource usage.

4. The distributed network security monitoring system based on cloud computing according to claim 3 is characterized in that: The task demand distribution function The specific calculation formula is as follows: where α j and β j is the distribution parameter.

5. The distributed network security monitoring system based on cloud computing according to claim 4 is characterized in that: Said The corresponding value ranges are as follows: If it is 0, it means that the resource utilization rate is extremely low between time t0 and t, and there are serious resource idleness and scheduling errors; Resource utilization efficiency is lower than task requirements, indicating that the system has a bottleneck in task processing; Resource allocation and task requirements are relatively balanced, and the system is in an ideal state; Resource utilization efficiency is higher than task requirements, indicating that the system is operating efficiently, but there is excess resources; The system resource utilization is extremely efficient, the task demand is low and the resource utilization is over-optimized.

6. The distributed network security monitoring system based on cloud computing according to claim 5 is characterized in that: Said The specific operations for the corresponding value range are as follows: Re-evaluate the current resource configuration, confirm whether the resources are reasonably distributed, adjust the resource allocation strategy to avoid idleness, and check whether the system has hardware failures and network bottlenecks, and determine whether the fault detection and automatic migration module needs to be started; Enable the automatic expansion module to add more computing resources and bandwidth for high-demand tasks, ensuring that critical tasks get resources first; Maintain the current resource allocation and scheduling strategy, continue to monitor the system to ensure that the balance state persists, and make slight adjustments to the load balancing according to real-time changes to maintain the optimal state; Start the automatic reduction module to appropriately reduce unnecessary resources to avoid waste, reallocate resources, and allocate excess resources to other low-priority tasks; Confirm whether there is a problem of insufficient task demand. If so, it is necessary to adjust the task priority and postpone non-urgent tasks, and further optimize the resource recovery strategy to ensure the rational use of resources and avoid long-term idle waste.

7. The distributed network security monitoring system based on cloud computing according to claim 6 is characterized in that: The system also includes a global resource scheduling and optimization platform, which is used to centrally manage all resources in the cloud environment and make optimal scheduling decisions based on a global perspective.

8. The distributed network security monitoring system based on cloud computing according to claim 7 is characterized in that: The global resource scheduling and optimization platform includes a visual interface that allows system administrators to monitor resource usage in real time, manually intervene in scheduling strategies, and view historical performance data and trend analysis reports of the system.

9. A distributed network security monitoring method based on cloud computing, based on a distributed network security monitoring system based on cloud computing according to any one of claims 1 to 8, characterized in that: Specifically include: S1. Deploy a resource monitoring module in the cloud computing environment to monitor the CPU, memory, storage, network bandwidth and other resource usage of each virtual machine and server in real time. The resource monitoring module regularly reports the current resource status to the central control node; S2. Calculate the corresponding information based on the information provided by the resource monitoring module And according to The specific results and their corresponding value ranges are used to judge the current resource utilization status; S3. According to The resource utilization status obtained is used to dynamically adjust resource allocation to ensure the reasonable distribution of resources; S4. Real-time monitoring of resource usage is performed through the global resource scheduling and optimization platform. Scheduling strategies can be manually intervened, and historical performance data and trend analysis reports of the system can be viewed.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, a distributed network security monitoring system based on cloud computing as described in claims 1-8 is implemented.

Images