Integrated system based on underlying resource virtualization

Abstract

The embodiment of the invention provides an integrated system based on underlying resource virtualization, and relates to the technical field of virtualization, a resource monitoring module collects various performance indexes of underlying hardware of the system through a sensor and a monitoring tool, and a data analysis module carries out deep processing and analysis; a machine learning algorithm is used for constructing a prediction model to predict a resource demand trend, historical data is learned and mined, a rule and a mode of the data are identified, and a strategy generation module is used for generating a resource allocation strategy according to an output result of a data analysis module by combining a preset optimization target and a constraint condition and using an intelligent optimization algorithm. And the resource allocation module allocates and adjusts the underlying resources of the system, and realizes intelligent allocation, dynamic optimization and efficient utilization of the underlying resources of the system by reasonably allocating the resources and ensuring real-time matching of allocation of the resources and requirements of services, so as to meet continuously changing service requirements and improve the performance, reliability and expandability of the system.

Description

Technical Field

[0001] The present application relates to the field of virtualization technology, and in particular to an integrated system based on underlying resource virtualization. Background Art

[0002] In today's era of rapid development of information technology, data has become a core element of the operation of enterprises and society. With the rise of cloud computing, big data analysis, artificial intelligence and other applications, the demand for data storage and processing has grown exponentially. As an important solution for massive data storage and management, cluster storage products are crucial to improving their performance and efficiency. In the field of cluster storage, the optimization of system underlying resources and virtualization technology have always been the focus of research. Key technologies cover the allocation, scheduling, management of storage resources, and the distribution, redundancy and protection of data. Traditional storage systems usually adopt a centralized resource management method, which is prone to performance bottlenecks and uneven resource allocation when facing large-scale, high-concurrency data access.

[0003] With the continuous evolution of technology, distributed storage architecture has gradually become the mainstream. Under this architecture, virtualization of the underlying system resources has become the key to achieving efficient storage and flexible resource management. By abstracting physical storage resources into logical resources and performing unified allocation and management, resource utilization and system scalability can be greatly improved.

[0004] In the prior art, the resource virtualization technology in cluster storage products mainly has the following common implementation methods: First, threshold-based resource allocation strategy: The system sets some resource usage thresholds, and when the resource usage reaches or exceeds these thresholds, the resource allocation or adjustment operation is triggered. For example, when the memory usage exceeds 80%, more memory resources are allocated to the system. The advantage of this method is that it is simple and intuitive, and easy to implement. But the disadvantage is also obvious. The setting of thresholds is often based on empirical values, lacks accurate prediction of business load changes, and easily leads to untimely or over-allocation of resources.

[0005] Second, priority-based resource allocation strategy: set priorities for different businesses or applications, and when resources are tight, prioritize resources for high-priority businesses. This method guarantees the resource requirements of key businesses to a certain extent, but may result in low-priority businesses not getting enough resources for a long time, affecting the overall business operation efficiency.

[0006] Third, rule-based resource allocation strategy: Determine resource allocation based on a series of pre-established rules, such as business type, data access mode, etc. Although this approach has a certain degree of flexibility, the formulation of rules often requires an in-depth understanding of the business and is difficult to adapt to complex and changing business scenarios.

[0007] However, the above resource virtualization technologies still face many challenges. For example, in terms of resource allocation, the traditional static allocation strategy cannot adapt to the dynamic changes of business load, resulting in resource waste or performance degradation; in terms of data distribution, the fixed data distribution mode cannot ensure balanced data storage and efficient access; in terms of resource management, there is a lack of effective monitoring and prediction mechanisms, which makes it impossible to discover and solve potential performance problems in a timely manner. Summary of the invention

[0008] The present application provides an integrated system based on underlying resource virtualization, which aims to solve the problems existing in resource virtualization technology in existing cluster storage products, realize intelligent allocation, dynamic optimization and efficient utilization of the underlying resources of the system, so as to meet the ever-changing business needs and improve the performance, reliability and scalability of the system.

[0009] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0010] In a first aspect, an embodiment of the present application provides an integrated system based on underlying resource virtualization, the integrated system comprising a resource monitoring module, a data analysis module, a policy generation module and a resource allocation module connected in sequence;

[0011] Resource monitoring module, which is used to collect various performance indicators of the underlying hardware of the system through deployed sensors and monitoring tools. The performance indicators include but are not limited to CPU utilization, memory usage, disk I / O performance, network bandwidth usage, data access volume, number of concurrent requests, and request response time;

[0012] The data analysis module is used to deeply process and analyze the various performance indicators collected by the resource monitoring module, use machine learning algorithms to build a prediction model to predict resource demand trends, learn and mine historical data, and identify data rules and patterns;

[0013] The strategy generation module is used to generate resource allocation strategies using intelligent optimization algorithms based on the output results of the data analysis module and in combination with pre-set optimization goals and constraints;

[0014] The resource allocation module is used to allocate and adjust the underlying resources of the system according to the resource allocation strategy. It interacts with the underlying hardware to rationally configure the CPU core, memory space, disk storage, and network bandwidth to ensure that resource allocation matches business needs in real time.

[0015] In one possible implementation, the resource allocation module is also tightly coupled with the hardware resource manager and operating system kernel at the bottom layer of the system, controls resources through the hardware access interface and system API, and feeds back the operation results to the policy generation module in real time when executing resource allocation operations, so as to dynamically adjust and optimize the resource allocation strategy.

[0016] In a possible implementation, the resource monitoring module further uses a periodic sampling method or an event-triggered detection method to collect various performance indicators.

[0017] In a possible implementation, the resource monitoring module also uses the performance interface and API provided by the operating system to obtain various performance indicators.

[0018] In a possible implementation, the data analysis module is also used to use data cleaning technology to remove noise and outliers from various performance indicators, and then use feature engineering technology to extract valuable features and patterns.

[0019] In a possible implementation, the policy generation module combines priorities of different services, availability of resources, and performance requirements of the system when generating a resource allocation policy.

[0020] In a possible implementation, the resource monitoring module is connected to the data analysis module via a data interface, the data analysis module is connected to the policy generation module via a high-speed internal communication bus, and the policy generation module is connected to the resource allocation module via a dedicated control channel.

[0021] In a possible implementation, the intelligent optimization algorithm includes a genetic algorithm, a simulated annealing algorithm, a particle swarm optimization algorithm, and an ant colony algorithm.

[0022] In a possible implementation, the resource allocation module is used to operate underlying resources and allocate resources in a middleware or agent manner.

[0023] In a possible implementation, the optimization objectives include maximizing resource utilization, minimizing system response time, and optimizing cost control, and the constraints include an upper limit on hardware resources, restrictions on business priorities, and regulatory compliance requirements.

[0024] Compared with the prior art, the embodiment of the present application provides an integrated system based on the virtualization of underlying resources. The resource monitoring module collects various performance indicators of the underlying hardware of the system through sensors and monitoring tools. The data analysis module performs in-depth processing and analysis, and uses machine learning algorithms to build a prediction model to predict resource demand trends. Historical data is learned and mined to identify data rules and patterns. The strategy generation module generates a resource allocation strategy based on the output results of the data analysis module and combines pre-set optimization goals and constraints with an intelligent optimization algorithm. The resource allocation module allocates and adjusts the underlying resources of the system. By reasonably configuring resources, it ensures that the allocation of resources matches the business needs in real time, and realizes the intelligent allocation, dynamic optimization and efficient utilization of the underlying resources of the system to meet the ever-changing business needs and improve the performance, reliability and scalability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A structural diagram of an integrated system based on underlying resource virtualization proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0029] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0030] In the prior art, the resource virtualization technology in cluster storage products is mainly implemented in the form of threshold-based resource allocation strategy, priority-based resource allocation strategy, and rule-based resource allocation strategy. However, these strategies have the following shortcomings:

[0031] First, the flexibility and adaptability are insufficient. Most resource allocation strategies in existing technologies are based on fixed rules, thresholds or priorities, and cannot be dynamically adjusted according to real-time business load and system status. This means that when business needs change, the system may not be able to respond in time, resulting in unreasonable resource allocation, affecting system performance and user experience.

[0032] Second, the lack of intelligent prediction capabilities, the existing resource management system is usually unable to accurately predict future resource needs. This makes it impossible for the system to prepare resources in advance when facing sudden business growth or load peaks, which is prone to performance bottlenecks and service interruptions.

[0033] Third, resource utilization is low. Due to the inability to achieve accurate resource allocation and dynamic adjustment, existing technologies often lead to some resources being idle while other resources being overused. This not only causes a waste of resources, but also increases the operating costs of the system.

[0034] Fourth, it is difficult to ensure balanced distribution of data. In terms of data storage, existing technologies make it difficult to achieve balanced distribution of data in a cluster, resulting in some storage nodes being overloaded while other nodes are idle, affecting the efficiency of data access and the overall performance of the system.

[0035] Fifth, management and maintenance are complex. Existing resource virtualization technologies usually require complex configuration and management operations, which increases the workload of system administrators and is prone to human errors.

[0036] Therefore, in order to solve the above-mentioned technical problems, the embodiment of the present application proposes an integrated system based on underlying resource virtualization to realize intelligent allocation, dynamic optimization and efficient utilization of the underlying resources of the system to meet the ever-changing business needs and improve the performance, reliability and scalability of the system, which is described in detail below.

[0037] Please refer to Figure 1 , Figure 1 A structural diagram of an integrated system based on underlying resource virtualization proposed in an embodiment of the present application is shown. The integrated system includes a resource monitoring module, a data analysis module, a policy generation module and a resource allocation module connected in sequence.

[0038] The resource monitoring module is used to collect various performance indicators of the system's underlying hardware through deployed sensors and monitoring tools. The performance indicators include but are not limited to CPU utilization, memory usage, disk I / O performance, network bandwidth occupancy, data access volume, number of concurrent requests, and request response time.

[0039] The resource monitoring module integrates a variety of high-precision sensors and advanced monitoring tools to achieve comprehensive and real-time monitoring of the system's underlying hardware and business application performance. Monitoring tools include but are not limited to performance counters, system call trackers, and network packet sniffers.

[0040] First, the resource monitoring module continuously collects the real-time operating parameters of the system hardware, including CPU utilization, memory usage, disk I / O speed, and network bandwidth usage, reflecting the utilization efficiency and potential bottlenecks of the underlying system resources. At the same time, it pays close attention to the load of business applications, such as data access volume, number of concurrent requests, and request response time. In-depth monitoring of the performance of business applications, including data access volume, number of concurrent requests processed per second, and average response time for each request. Using an efficient data transmission mechanism, ensure that the information collected from all corners of the system can be quickly and accurately sent to the data analysis module.

[0041] The resource monitoring module also uses a periodic sampling method or an event-triggered detection method to collect various performance indicators.

[0042] The resource monitoring module adopts a monitoring method based on periodic sampling. This method performs sampling at fixed time intervals, which can reduce the monitoring burden of the system to a certain extent, but may cause some instantaneous key data to be missed. In addition, an event-triggered monitoring method can be used. When a specific system event occurs (such as resource utilization exceeding a certain threshold, a sudden large number of data access requests, etc.), monitoring will be started and relevant data will be collected. This method can obtain key information in a targeted manner, but it may not respond to some gradually evolving changes in resource demand in a timely manner.

[0043] The resource monitoring module also uses the performance interface and API provided by the operating system to obtain various performance indicators.

[0044] In addition to obtaining system performance indicators directly through hardware sensors, the resource monitoring module can also use the performance interfaces and APIs provided by the operating system to obtain relevant data. For example, in Windows systems, the usage of resources such as CPU and memory can be obtained through the WMI (Windows Management Instrumentation) interface; in Linux systems, similar information can be obtained through the proc file system or sysfs file system. In addition, some third-party open source monitoring tools and frameworks, such as Nagios and Zabbix, can also be integrated into the system as a means of resource monitoring. These tools usually have rich functions and plug-ins to meet different monitoring needs, but may require additional configuration and integration work.

[0045] The data analysis module is used to perform in-depth processing and analysis on the various performance indicators collected by the resource monitoring module, use machine learning algorithms to build a prediction model to predict resource demand trends, learn and mine historical data, and identify data rules and patterns.

[0046] The data analysis module deeply analyzes and refines the huge data set transmitted by the resource monitoring module. First, the raw data is cleaned to remove erroneous records, outliers and noise to ensure the purity and reliability of the analysis basis. Through feature selection and transformation, the most valuable information fragments and patterns for the prediction model are extracted from the massive data. This step is crucial to revealing the internal correlation of the data. Then, the prediction model is constructed using cutting-edge machine learning algorithms such as deep learning neural networks, random forests, and support vector machines to learn the complex laws in historical data and form accurate predictions of future resource demand and system load. Finally, through deep learning and mining of historical data, the data analysis module can reveal hidden trends, periodicities and potential abnormal patterns.

[0047] The data analysis module is also used to remove noise and outliers from various performance indicators using data cleaning techniques, and then feature engineering techniques are used to extract valuable features and patterns.

[0048] The data analysis module uses data cleaning technology to remove noise and outliers to ensure the quality and reliability of the data. Then, through feature engineering technology, valuable features and patterns are extracted from the raw data.

[0049] In a possible implementation, the parameter selection of the machine learning algorithm of the data analysis module needs to be adjusted according to the specific data characteristics and business needs. For example, for the neural network model, it is necessary to determine the number of hidden layers, the number of neurons, the learning rate and other parameters. The learning rate is usually selected between 0.001 and 0.1, and is adjusted according to the size and complexity of the data. If the amount of data is large and complex, the learning rate can be appropriately reduced to avoid overfitting. On the contrary, if the data is relatively simple, the learning rate can be appropriately increased to speed up the training speed.

[0050] Neural network models based on deep learning, such as convolutional neural networks (CNN) or recurrent neural networks (RNN), can also be used as alternative prediction models. CNN performs well when processing data in the form of images or matrices. If the resource usage data is converted into a two-dimensional or multi-dimensional matrix form, deeper features and patterns may be mined. RNN is particularly suitable for processing sequence data and can better capture long-term dependencies for resource usage with time series characteristics.

[0051] In the data analysis module, in addition to machine learning algorithms and statistical analysis methods, analysis methods based on data mining technology are also feasible. Association rule mining in data mining can discover potential associations between different resource indicators, thereby providing a new perspective for resource demand prediction. For example, through mining, it is found that there is a certain correlation pattern between CPU utilization and disk I / O, and then resource demand is predicted based on this correlation. Cluster analysis can group business applications or system states with similar resource usage characteristics, so as to formulate targeted resource allocation strategies for different groups.

[0052] The strategy generation module is used to generate resource allocation strategies using intelligent optimization algorithms based on the output results of the data analysis module and in combination with pre-set optimization goals and constraints.

[0053] Optimization goals include maximizing resource utilization, minimizing system response time, and optimizing cost control. Constraints include the upper limit of hardware resources, restrictions on business priorities, and regulatory compliance requirements.

[0054] For the definition of constraints and optimization goals, this application considers cost minimization, energy minimization or quality of service (QoS) guarantee as the main optimization goals. Constraints may include physical limitations of resources (such as the maximum capacity of hardware), business priority requirements, and compliance requirements of laws and regulations.

[0055] In a possible implementation, the parameters of the optimization algorithm used by the strategy generation module, such as the population size, crossover probability, and mutation probability of the genetic algorithm, also need to be carefully selected. The population size is generally between 50 and 200, the crossover probability is generally between 0.6 and 0.9, and the mutation probability is between 0.01 and 0.1. The selection of these parameters needs to comprehensively consider the scale of the system, the diversity of resources, and the complexity of the optimization target.

[0056] Intelligent optimization algorithms include genetic algorithm, simulated annealing algorithm, particle swarm optimization algorithm and ant colony algorithm.

[0057] In terms of the resource allocation strategy generation mechanism, a genetic algorithm is used to simulate the process of biological evolution, cross, mutate and select possible strategies, and gradually optimize to obtain the optimal resource allocation strategy. This algorithm has strong capabilities in dealing with complex multi-objective optimization problems, but the computational complexity is relatively high. The ant colony algorithm can also be applied to the generation of resource allocation strategies. The ant colony algorithm simulates the behavior of ants in the process of searching for food and uses the accumulation and update of pheromones to guide the search for the optimal solution. In the resource allocation problem, the resource allocation plan can be regarded as the path of the ants, and the optimal allocation strategy can be found through continuous iteration and pheromone update.

[0058] The strategy generation module designs and outputs efficient resource scheduling strategies based on the advanced optimization algorithms used by the data analysis module. First, it receives the output from the data analysis module, which includes the prediction of future resource requirements, the identification of current system performance bottlenecks, and the indication of potential optimization space.

[0059] According to the specific needs of the organization or system, the policy generation module will set clear optimization goals, such as increasing resource utilization to the highest level, shortening the response time of user requests to the shortest, and achieving the optimal configuration within the cost control framework. When formulating policies, actual constraints must be strictly considered, such as the physical limitations of hardware resources, the priority ranking of business operations, and the compliance requirements of external policies and regulations, to ensure that the generated policies are feasible.

[0060] Using intelligent optimization algorithms, such as genetic algorithms, simulated annealing algorithms, and particle swarm optimization algorithms, the strategy generation module explores the solution space under the set multi-objective and constraint conditions to find the best balance point for resource allocation. After considering all factors, the module outputs one or more resource allocation strategies to maximize resource utilization and system efficiency while meeting the requirements of business continuity and cost-effectiveness.

[0061] The policy generation module combines the priorities of different services, the availability of resources, and the performance requirements of the system when generating resource allocation policies.

[0062] The policy generation module comprehensively considers various factors, such as the priority of different businesses, the availability of resources, the performance requirements of the system, etc., to ensure that the generated policies can not only meet business needs, but also make full use of system resources to maximize overall benefits.

[0063] The resource allocation module is used to allocate and adjust the underlying resources of the system according to the resource allocation strategy. It interacts with the underlying hardware to rationally configure the CPU core, memory space, disk storage, and network bandwidth to ensure that resource allocation matches business needs in real time.

[0064] The resource allocation module implements fine control of system resources through the underlying hardware interface based on the resource allocation strategy carefully calculated by the policy generation module. Following the fluctuation of business needs, it dynamically adjusts the allocation of CPU cores, memory allocation and recovery, disk I / O optimization, and reasonable allocation of network bandwidth to ensure that resources can instantly adapt to the changing application load.

[0065] By using the underlying driver and API interface, direct communication with the server hardware is achieved to ensure that the resource configuration instructions can be accurately executed by the hardware, thus improving the accuracy and efficiency of resource configuration. The optimization solutions output by the strategy generation module, such as on-demand capacity expansion, load balancing, resource reservation and other strategies, are converted into actual operation instructions to achieve the optimal configuration of resources and avoid resource waste and performance bottlenecks.

[0066] In the process of dynamically adjusting resources, the resource allocation module must also ensure smooth operation and system stability, avoid service interruption or performance degradation due to resource reallocation, and maintain the high availability of the system.

[0067] The resource allocation module is also tightly coupled with the hardware resource manager and operating system kernel at the bottom of the system. It controls resources through the hardware access interface and system API. When executing resource allocation operations, it feeds back the operation results to the policy generation module in real time to dynamically adjust and optimize the resource allocation strategy.

[0068] The resource allocation module is tightly coupled with the hardware resource manager and operating system kernel at the bottom of the system, and implements fine control of resources through direct hardware access interfaces and system APIs. When executing resource allocation operations, the operation results are fed back to the policy generation module in real time to dynamically adjust and optimize the policy.

[0069] The resource allocation module is used to operate the underlying resources and allocate resources using middleware or agents.

[0070] In terms of the implementation of the resource allocation module, not only can the underlying resources be directly operated, but also middleware or agents can be used to indirectly implement resource allocation. For example, by inserting a resource allocation agent layer between the operating system kernel and the application, the agent layer coordinates and allocates resources according to the generated policy. This approach can increase the flexibility and scalability of the system, but may introduce certain performance overhead.

[0071] In a possible implementation, the threshold and granularity of resource allocation of the resource allocation module are also important parameters. For example, the allocation granularity of the CPU core can be a single core or multiple cores, and the allocation granularity of the memory can be bytes, kilobytes, or megabytes. The setting of thresholds, such as the upper and lower limits of CPU utilization, the threshold of memory usage, etc., needs to be reasonably set according to the performance requirements of the system and the characteristics of the business load.

[0072] The resource monitoring module is connected to the data analysis module through a data interface, the data analysis module is connected to the strategy generation module through a high-speed internal communication bus, and the strategy generation module is connected to the resource allocation module through a dedicated control channel.

[0073] The resource monitoring module is responsible for collecting the usage of various resources in the system, such as network bandwidth, CPU load, storage space, etc. It is connected to the data analysis module through a high-speed, low-latency data interface to ensure that the data can be transmitted quickly. After the data analysis module receives the data from the resource monitoring module, it will immediately process and analyze it, and pass it to the strategy generation module through an efficient internal communication bus. The internal communication bus design facilitates fast and stable information exchange between modules. The strategy generation module runs a specific algorithm to formulate optimization or scheduling strategies based on the received analysis results. Once the strategy is generated, it is communicated to the resource allocation module through a dedicated control channel, which ensures the security and accuracy of the strategy communication and ensures that the strategy can be correctly executed, thereby realizing dynamic management and optimization of the system.

[0074] The implementation steps of an integrated system based on underlying resource virtualization proposed in the embodiment of the present application are as follows:

[0075] S1. System initialization: When the system starts, all components are ready, necessary settings and parameters are loaded, the internal communication framework is built, and key services are activated to lay the foundation for subsequent operations.

[0076] S2. Resource monitoring: The system monitoring component starts to operate and continuously collects system performance indicators and business load information, covering CPU usage, memory usage, disk read and write speed, network traffic, and application-level activity metrics.

[0077] S3. Data transmission: The collected raw data is immediately transferred to the data analysis module through an efficient data pipeline to ensure the timeliness of the data.

[0078] S4, Data Analysis: After receiving the data, this module performs data cleaning and preliminary processing, and uses machine learning algorithms and data mining techniques to conduct in-depth data analysis, aiming to predict future resource demand trends and business load patterns.

[0079] S5. Strategy generation: Based on the insights gained from the analysis, the strategy generation module uses optimization algorithms to produce resource allocation strategies, taking into account preset goals and constraints, and specifically provides allocation recommendations for the number of CPU cores, memory size, etc.

[0080] S6. Strategy transmission: The formulated strategy is transmitted to the resource allocation execution module through a secure and reliable channel to ensure the accurate delivery of the strategy instructions.

[0081] S7. Resource allocation: The resource allocation module interacts closely with the underlying system based on the received policies, implements actual configuration adjustments of hardware and software resources, and directly intervenes in the system resource configuration.

[0082] S8, Effect Monitoring: To verify the effectiveness of the strategy, this module continuously monitors the system performance after resource allocation, such as response time, performance indicators and resource utilization efficiency.

[0083] S9. Feedback adjustment: Based on monitoring feedback, the system evaluates the effectiveness of resource allocation strategies. If expectations are not met, the loop will be automatically triggered to re-analyze data and optimize strategies, forming a process of self-adjustment and continuous improvement.

[0084] After the system is initialized, the resource monitoring module relies on widely deployed sensors to continuously collect various data, covering system performance (such as CPU usage, memory usage) and business activities (such as request volume, data processing volume). The high frequency of data collection ensures the timeliness and comprehensiveness of the information. The collected data is sent to the data analysis module, which is not only supported by powerful computing power, but also applies machine learning algorithms, such as regression analysis to identify trends and cluster analysis to distinguish different behavior patterns. These analyses reveal the laws of resource utilization and the development trend of business loads. In addition, the application of prediction models enables the system to predict future resource demand and load pressure.

[0085] Based on the prediction results, the strategy generation module uses complex optimization algorithms to generate strategies according to the goals set by the system (such as performance standards, business priorities, cost control) and constraints. Genetic algorithms, simulated annealing algorithms, etc., simulate natural selection or physical processes to find the best, ensuring that the resource allocation strategy can not only meet current needs, but also foresee future changes. Based on the generated strategy, the resource allocation module directly interacts with the hardware layer to dynamically adjust resource configuration, including the number of CPU cores, memory size, etc. At the same time, it continuously monitors the implementation effect, and this feedback information is then passed back to the strategy generation module to form a closed-loop control mechanism. This means that the strategy can be continuously fine-tuned and optimized according to the actual operation situation to achieve the best system performance and resource utilization.

[0086] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0087] First, by real-time monitoring of system status and business load, and using advanced algorithms and models, we can achieve intelligent dynamic allocation of resources, ensure that each application can obtain the required resources, and improve the overall performance and response speed of the system.

[0088] Second, based on big data analysis and machine learning technology, it can accurately predict future resource needs, plan and prepare resources in advance, and effectively avoid resource shortages and performance bottlenecks.

[0089] Third, through refined resource management and optimization, we can maximize resource utilization, reduce system operating costs, and reduce resource waste.

[0090] Fourth, innovative data distribution algorithms are used to ensure balanced storage and efficient access to data in the cluster, improving data reliability and availability.

[0091] Fifth, provide a simple and intuitive management interface and automated configuration tools to reduce the complexity of system management and maintenance and reduce the occurrence of human errors.

[0092] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An integrated system based on underlying resource virtualization, characterized in that: The integrated system includes a resource monitoring module, a data analysis module, a strategy generation module and a resource allocation module connected in sequence; Resource monitoring module, which is used to collect various performance indicators of the underlying hardware of the system through deployed sensors and monitoring tools. The performance indicators include but are not limited to CPU utilization, memory usage, disk I / O performance, network bandwidth usage, data access volume, number of concurrent requests, and request response time; The data analysis module is used to deeply process and analyze the various performance indicators collected by the resource monitoring module, use machine learning algorithms to build a prediction model to predict resource demand trends, learn and mine historical data, and identify data rules and patterns; The strategy generation module is used to generate resource allocation strategies using intelligent optimization algorithms based on the output results of the data analysis module and in combination with pre-set optimization goals and constraints; The resource allocation module is used to allocate and adjust the underlying resources of the system according to the resource allocation strategy. It interacts with the underlying hardware to rationally configure the CPU core, memory space, disk storage, and network bandwidth to ensure that resource allocation matches business needs in real time.

2. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: The resource allocation module is also tightly coupled with the hardware resource manager and operating system kernel at the bottom of the system. It controls resources through the hardware access interface and system API. When executing resource allocation operations, it feeds back the operation results to the policy generation module in real time to dynamically adjust and optimize the resource allocation strategy.

3. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: The resource monitoring module also uses a periodic sampling method or an event-triggered detection method to collect various performance indicators.

4. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: The resource monitoring module also uses the performance interface and API provided by the operating system to obtain various performance indicators.

5. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: The data analysis module is also used to remove noise and outliers from various performance indicators using data cleaning techniques, and then feature engineering techniques are used to extract valuable features and patterns.

6. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: The policy generation module combines the priorities of different services, the availability of resources, and the performance requirements of the system when generating resource allocation policies.

7. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: The resource monitoring module is connected to the data analysis module through a data interface, the data analysis module is connected to the strategy generation module through a high-speed internal communication bus, and the strategy generation module is connected to the resource allocation module through a dedicated control channel.

8. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: Intelligent optimization algorithms include genetic algorithm, simulated annealing algorithm, particle swarm optimization algorithm and ant colony algorithm.

9. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: The resource allocation module is used to operate the underlying resources and allocate resources using middleware or agents.

10. The integrated system based on underlying resource virtualization according to claim 1, characterized in that: Optimization goals include maximizing resource utilization, minimizing system response time, and optimizing cost control. Constraints include the upper limit of hardware resources, restrictions on business priorities, and regulatory compliance requirements.

Images