Virtual machine data processing method and system and storage medium

Abstract

The invention relates to a virtual machine data processing method and system and a storage medium. A virtual machine is deployed, the resource use condition of the virtual machine is collected and stored, the load of the virtual machine is predicted by using a machine learning algorithm, a prediction result is obtained, resource allocation of virtual machine resources is dynamically adjusted based on the prediction result, and when the load of the virtual machine is changed or a fault occurs, the virtual machine is subjected to resource allocation. And performing live migration of the virtual machine, and generating a virtual machine performance report according to the resource use condition of the virtual machine. And the resource allocation of the virtual machine resources is dynamically adjusted according to the prediction result, so that the accuracy of the processing result and the processing efficiency are improved. And when the load of the virtual machine changes, the virtual machine thermal migration is carried out, so that load balancing is realized, and the problem of resource waste is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a virtual machine data processing method, system and storage medium. Background Art

[0002] With the rapid development of cloud computing and data centers, virtualization technology has become one of the key technologies to improve resource management efficiency, enhance system flexibility and reduce hardware costs. Virtualization technology enables multiple virtual machines (VMs) to run on the same physical hardware. Each virtual machine can independently simulate the operating system and application environment, which greatly promotes the effective allocation and utilization of resources. However, in related technologies, the resource scheduling of virtual machines often relies on static configuration or manual operation, which not only leads to low processing efficiency, but also affects the accuracy of processing results. Especially in the case of significant load fluctuations, virtual machines may encounter resource waste problems, which in turn has an adverse impact on the stability and overall performance of the system. Summary of the invention

[0003] The present invention provides a virtual machine data processing method, system and storage medium, aiming to solve at least one of the technical problems existing in the prior art.

[0004] The technical solution of the present invention is a virtual machine data processing method, comprising: Deploy virtual machines; Collecting and storing resource usage of the virtual machine; Using a machine learning algorithm to predict virtual machine loads, obtaining prediction results, and dynamically adjusting resource allocation of virtual machine resources based on the prediction results; When the virtual machine load changes or a failure occurs, hot migration of the virtual machine is performed; Generate a virtual machine performance report based on the resource usage of the virtual machine.

[0005] According to some embodiments of the present invention, the deploying a virtual machine includes: Creating a virtual machine configuration template, and deploying the virtual machine according to the virtual machine configuration template; According to the predefined operating system configuration, the operating system of the virtual machine is installed and the operating system is initially configured.

[0006] According to some embodiments of the present invention, the step of creating a virtual machine configuration template, deploying the virtual machine according to the virtual machine configuration template, and installing the operating system of the virtual machine and initializing the operating system according to a predefined operating system configuration includes: According to the requirements of different virtual machines, create corresponding virtual machine configuration templates to define the virtual machine resources; Deploy the virtual machine according to the virtual machine configuration template using Ansible or Terraform tools; Use one of cloud-init, PXE, and kickstart tools to install the operating system of the virtual machine and initialize the settings of the operating system according to the predefined operating system configuration.

[0007] According to some embodiments of the present invention, the collecting and storing the resource usage of the virtual machine includes: Use libvirt to collect the resource usage of the virtual machine; Use the Prometheus time series database to store the resource usage of the virtual machine.

[0008] According to some embodiments of the present invention, the virtual machine resources include the number of CPU cores, memory size, disk space, operating system type, and network configuration.

[0009] According to some embodiments of the present invention, the machine learning algorithms include regression analysis and time series analysis; The using machine learning algorithms to predict the virtual machine load includes: Combine the regression analysis and the time series analysis to predict the virtual machine load.

[0010] According to some embodiments of the present invention, when the virtual machine load changes or a failure occurs, perform virtual machine live migration, including: Use pacemaker and corosync to monitor the virtual machine, and start the virtual machine live migration when the failure occurs; When the virtual machine load changes, use the virsh command to perform the virtual machine live migration; When performing the virtual machine live migration across data centers, use distributed storage to make the data consistent.

[0011] According to some embodiments of the present invention, after generating a virtual machine performance report based on the resource usage of the virtual machine, the virtual machine data processing method further includes: Adjust the resource allocation of the virtual machine resources based on the adjustment suggestions in the virtual machine performance report.

[0012] The technical solution of the present invention further relates to a virtual machine data processing system for executing a virtual machine data processing method as described above, and the virtual machine data processing system includes: A virtual machine deployment module for deploying the virtual machine; A virtual machine data collection module, used to collect and store the resource usage of the virtual machine; the virtual machine deployment module is connected to the virtual machine data collection module; A resource tuning module, used for adjusting the resource allocation of the virtual machine resources; the virtual machine data acquisition module is connected to the resource tuning module; A virtual machine migration module, used for performing hot migration of the virtual machine; the resource tuning module is connected to the virtual machine migration module; The report generation module is used to generate the virtual machine performance report; the virtual machine migration module is connected to the report generation module.

[0013] The technical solution of the present invention also relates to an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the virtual machine data processing method as described above when executing the computer program.

[0014] The technical solution of the present invention also relates to a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the virtual machine data processing method as described above is implemented.

[0015] The beneficial effects of the present invention include: deploying virtual machines, collecting and storing the resource usage of virtual machines, using machine learning algorithms to predict the load of virtual machines, obtaining prediction results, dynamically adjusting the resource allocation of virtual machine resources based on the prediction results, performing hot migration of virtual machines when the load of virtual machines changes or failures occur, and generating a virtual machine performance report based on the resource usage of virtual machines. Dynamically adjusting the resource allocation of virtual machine resources based on the prediction results improves the accuracy of the processing results and the processing efficiency. Hot migration of virtual machines is performed when the load of virtual machines changes, achieving load balancing, which is conducive to solving the problem of resource waste.

[0016] In addition, additional aspects and advantages of the present invention will be given in part in the following description, and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is an optional flow chart of a virtual machine data processing method in an embodiment of the present invention.

[0018] Figure 2 This is an optional flow chart for deploying a virtual machine in an embodiment of the present invention.

[0019] Figure 3 It is an optional flow chart of deploying a virtual machine and installing and setting an operating system of the virtual machine in an embodiment of the present invention.

[0020] Figure 4It is a schematic diagram of a virtual machine data processing system in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0022] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, top, bottom, etc. used in the present invention are only relative to the relative positional relationship of the components of the present invention in the drawings.

[0023] In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art. The terms used in this specification are only for describing specific embodiments, not for limiting the present invention. The term "and / or" used herein includes any combination of one or more related listed items.

[0024] It should be understood that although the terms first, second, third, etc. may be used to describe various elements in the present invention, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of the present invention, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element.

[0025] Reference Figures 1 to 3 In some embodiments, the technical solution of the present invention is a virtual machine data processing method, including but not limited to steps S101 to S105, and each step is introduced in turn below.

[0026] Step S101: deploy a virtual machine.

[0027] Specifically, the virtual machine is a QEMU-KVM virtual machine. QEMU-KVM (Quick Emulator - Kernel-based Virtual Machine) is a widely used open source virtualization technology that combines QEMU and KVM kernel modules, provides powerful virtualization support, and can achieve full virtualization on the Linux platform.

[0028] It should be noted that QEMU (Quick Emulator) is an open source, general-purpose virtual machine emulator that can simulate virtual hardware resources such as processors, memory, and devices in user space to achieve simulated operation of the operating system. KVM (Kernel-based Virtual Machine) is a virtualization module in the Linux kernel that provides underlying acceleration support for virtual machines by calling the hardware virtualization instruction set of modern CPUs (such as Intel VT-x and AMD-V).

[0029] When QEMU is used alone, all virtualization operations are completed through software emulation, which consumes large system resources and has low operating efficiency, making it difficult to meet high-concurrency and high-performance business requirements. Although KVM has hardware acceleration capabilities, it is only a kernel module and cannot independently implement complete virtual machine life cycle management. It needs to be used in conjunction with the control program in the user space.

[0030] Through the deep integration of QEMU and KVM, QEMU is responsible for virtual machine management, device simulation and IO processing in user space, and KVM is responsible for providing CPU execution acceleration of virtual machines, thus forming a high-performance full virtualization system. Compared with the traditional solution of using only QEMU or KVM, the present invention has the following advantages based on QEMU-KVM: KVM provides direct support for CPU virtualization instructions, so that the operating efficiency of virtual machines is close to the performance of physical machines, which is significantly better than the pure software simulation method of QEMU. QEMU supports multiple architectures (such as x86, ARM, PPC, etc.), has good cross-platform capabilities, and is convenient for unified management in heterogeneous environments. QEMU has perfect virtual device simulation capabilities, can flexibly configure peripherals such as disks, networks, USB, etc., and is suitable for resource requirements in different scenarios. Combined with the KVM kernel module and the libvirt API interface, QEMU-KVM supports dynamic scheduling, online migration, snapshot backup and other functions of virtual machines, which is the technical basis for the present invention to realize virtual machine deployment and tuning. With the help of QEMU-KVM's hot migration mechanism, virtual machines can be migrated to other hosts without interrupting their operation, effectively supporting automatic fault recovery and load balancing.

[0031] In some embodiments, reference Figure 2 , deploying a virtual machine includes but is not limited to the following steps S201 to S202.

[0032] Step S201: Create a virtual machine configuration template, and deploy a virtual machine according to the virtual machine configuration template.

[0033] Step S202: installing the operating system of the virtual machine and initializing the operating system settings according to the predefined operating system configuration.

[0034] In some embodiments, reference Figure 3 , create a virtual machine configuration template, deploy the virtual machine according to the virtual machine configuration template; install the operating system of the virtual machine and initialize the operating system according to the predefined operating system configuration, including but not limited to the following steps S301 to S303.

[0035] Step S301: Create corresponding virtual machine configuration templates according to requirements of different virtual machines to define virtual machine resources.

[0036] Step S302: Use Ansible or Terraform to deploy a virtual machine according to the virtual machine configuration template.

[0037] Step S303: Use one of cloud-init, PXE and kickstart tools to install the operating system of the virtual machine and initialize the operating system according to the predefined operating system configuration.

[0038] In some embodiments, the virtual machine resources include the number of CPU cores, memory size, disk space, operating system type, and network configuration. Specifically, the network configuration is the network interface and IP address of the virtual machine.

[0039] Specifically, Ansible is an open source automated operation and maintenance tool developed based on Python language, mainly used to configure systems, deploy software and orchestrate more advanced IT tasks. Terraform is an open source Infrastructure as Code (IaC) tool developed by HashiCorp, which allows users to define, manage and automate the creation and deployment of cloud infrastructure in the form of code. Cloud-init is a widely used open source tool, mainly used to initialize the configuration of virtual machines or containers in cloud environments or virtualized environments. Preboot eXecution Environment (PXE) is a network-based diskless boot technology that allows computers to boot operating systems from remote servers over the network; it loads the files and operating system images required for startup through network interface cards (NICs) and network protocols (such as DHCP, TFTP, etc.), thereby achieving startup without local storage devices. Kickstart is an automated installation tool, mainly used to automate the installation and configuration of Linux operating systems; it uses predefined configuration files (Kickstart files) to specify various parameters and settings during the installation process, thereby achieving unattended installation. The central processing unit (CPU) is the computing and control core of the computer system and the final execution unit for information processing and program running. The Internet Protocol (IP) is the basic communication protocol used for data transmission between devices in a computer network.

[0040] It is understandable that, according to the needs of different virtual machines, corresponding standardized virtual machine configuration templates are created to define virtual machine resources, such as the number of CPU cores, memory size, disk space, operating system type, etc. The virtual machine configuration template includes the number of CPU cores, memory size, disk space, operating system type, network configuration, and disk type. The predefined operating system configuration includes user accounts, network configuration, time zone settings, etc.

[0041] Specifically, the target object may customize the virtual machine configuration template, or obtain the corresponding preset virtual machine configuration template according to the requirements of different virtual machines. The target object is a user.

[0042] In a possible implementation, a declarative configuration file is obtained, and the virtual machine resources and their properties are defined by the declarative configuration file using the Ansible or Terraform tool to deploy the virtual machine. Specifically, the declarative configuration allows the target object to describe the final desired resource state, and the Ansible or Terraform tool automatically creates and configures the virtual machine resources to ensure the consistency of the virtual machine deployment process.

[0043] Create a virtual machine configuration template that defines a virtual machine with 2 virtual CPUs, 4 GB of memory, disk, and network configuration. The example is as follows: resource "libvirt_domain" "test-vm" { name = "test-vm" memory = "4096" vcpu = 2 disk { volume_id = "${libvirt_volume.test-disk.id}" } network_interface { network_id = "${libvirt_network.test-network.id}" } } Specifically, Ansible or Terraform tools perform the creation and configuration of virtual machines through libvirt's API (such as the virsh command). An Application Programming Interface (API) is a definition and protocol specification that allows software applications or components to interact and exchange data.

[0044] Use Ansible to configure virtual machines through the virt module to create memory, CPU, disk, and network configurations. The following example shows: - name: Create and configure a VM hosts: localhost tasks: - name: Create virtual machine virt: name: test-vm memory: 4096 vcpu: 2 disk: size: 10 networks: - name: default os_variant: ubuntu20.04 Specifically, cloud-init is used to automate the operating system installation process when the virtual machine is started for the first time, completing user creation, network configuration, and time zone settings.

[0045] Create a user named ubuntu and grant administrator privileges. The example is as follows: #cloud-config users: - name: ubuntu sudo: ALL=(ALL) NOPASSWD:ALL shell: / bin / bash It is understandable that by calling tools such as Ansible and Kickstart, the deployment process of virtual machines is simplified, manual configuration errors are reduced, and the consistency and efficiency of virtual machine deployment are ensured.

[0046] Step S102: Collect and store the resource usage of the virtual machine.

[0047] In some embodiments, collecting and storing resource usage of a virtual machine includes: Use libvirt to collect virtual machine resource usage; Use the Prometheus time series database to store virtual machine resource usage.

[0048] Specifically, libvirt is an open source API, background program, and management tool for managing virtualization platforms. Prometheus is an open source system monitoring and alerting toolkit. The Prometheus time series database is a core component of the Prometheus monitoring system. It is a built-in database specifically for storing and querying time series data.

[0049] Specifically, the resource usage of the virtual machine, such as CPU, memory, disk I / O, etc., is obtained through the libvirt API. The resource usage of the virtual machine is captured regularly and stored in a time series format. The libvirt API is an open source API set provided by libvirt.

[0050] Use the libvirt-exporter plugin to export the resource usage of the virtual machine to Prometheus. libvirt-exporter is a Prometheus Exporter used to monitor the libvirt virtualization environment. The example is as follows: docker run -d -p 9272:9272 --name libvirt-exporter \ --mount type=bind,source= / var / run / libvirt,target= / var / run / libvirt \ quay.io / prometheuscommunity / libvirt-exporter In some embodiments, after collecting and storing the resource usage of the virtual machine, the virtual machine data processing method further includes: using Grafana to visualize the real-time performance data. Specifically, the real-time performance data is the real-time performance data of the resource usage of the virtual machine. Grafana is an open source data visualization and monitoring tool.

[0051] Specifically, Prometheus is used to query the resource usage of the virtual machine, and Grafana is used for visualization, helping administrators understand the resource usage of the virtual machine in real time.

[0052] Query the CPU usage of the virtual machine. The example is as follows: rate(process_cpu_seconds_total{job="libvirt"}[5m]) In some embodiments, a performance bottleneck analysis report is generated through Prometheus and Grafana to identify resource bottlenecks, such as excessive CPU load, excessive memory pressure, etc.

[0053] Step S103: Use a machine learning algorithm to predict the virtual machine load, obtain a prediction result, and dynamically adjust the resource allocation of the virtual machine resources based on the prediction result.

[0054] In one possible implementation, a machine learning algorithm is used to predict the virtual machine load and adjust resource allocation in advance. According to the real-time load of the virtual machine, the resource allocation of the virtual machine resources is dynamically increased or decreased, such as increasing the CPU core, memory size, storage, etc. Specifically, storage refers to disk size, storage type, etc.

[0055] In some embodiments, the machine learning algorithm includes regression analysis and time series analysis; using the machine learning algorithm to predict the virtual machine load includes: combining regression analysis and time series analysis to predict the virtual machine load.

[0056] Specifically, the regression analysis model and the time series analysis model are combined to predict the virtual machine load.

[0057] Regression analysis is a statistical method used to predict the relationship between a dependent variable (or response variable) and an independent variable (or explanatory variable). Regression analysis models are used to predict future load requirements based on the historical load data of a virtual machine. Independent variables represent the resource usage data of a virtual machine, such as CPU usage, memory usage, disk I / O, network bandwidth, etc., which are the input features of regression analysis. A regression model means that through a regression algorithm (such as linear regression, polynomial regression, etc.), the model establishes a relationship between input data (independent variables) and a target variable (response variable, such as future load requirements).

[0058] Implement linear regression in Python. Use historical VM load data (such as CPU and memory usage) to train a regression model and then predict future load requirements. An example is as follows: from sklearn.linear_model import LinearRegression import numpy as np # Assume that X is the historical resource usage data and y is the corresponding load data X = np.array([[30, 40], [35, 50], [40, 60]]) # Sample data: CPU, memory usage y = np.array([60, 70, 80]) # Load prediction target model = LinearRegression() model.fit(X, y) predictions = model.predict(np.array([[45, 65]])) # Predict future load print("Predicted load demand:", predictions) Time series analysis is a statistical method based on time-sequential data, which is widely used in forecasting, monitoring and trend analysis. Time series models are used to analyze the historical trend of virtual machine resource load and predict future load requirements.

[0059] Use the ARIMA model to model the historical load data of the virtual machine and predict future load demand. The ARIMA model is a statistical model used for time series analysis and forecasting. Its full name is the Autoregressive Integrated Moving Average Model, sometimes also called the Box-Jenkins model. The ARIMA model combines the three methods of autoregression (AR), difference (I), and moving average (MA), and is suitable for processing time series data with trend components or seasonal components. The following is an example: from statsmodels.tsa.arima.model import ARIMA import numpy as np # Assume this is historical load data (e.g., CPU load) data = [60, 65, 70, 75, 80, 85, 90, 95] # Create ARIMA model model = ARIMA(data, order=(5,1,0)) # ARIMA(5,1,0) is the model order model_fit = model.fit() # Perform future load prediction forecast = model_fit.forecast(steps=5) # Forecast the load for the next 5 time steps print("Forecasted load demand:", forecast) It is understandable that the regression analysis model predicts future load demand through historical resource data, helping the system to allocate resources in advance. The time series analysis model can analyze the periodicity and trend of resources to ensure that the system makes reasonable resource scheduling when the load fluctuates. Combining the regression analysis model and the time series analysis model improves the intelligence level of the system and makes virtual machine resource scheduling more efficient and stable.

[0060] In a possible implementation, scikit-learn is used to predict the virtual machine load, obtain a prediction result, and adjust the resource allocation of the virtual machine resources based on the prediction result.

[0061] Among them, Scikit-learn is a free and open source machine learning library for the Python programming language. Scikit-learn provides a variety of classification, regression, and clustering algorithms, including support vector machines, random forests, and gradient boosting. In addition, it also covers various aspects of the machine learning process, such as data preprocessing, feature engineering, model selection, and evaluation.

[0062] The following is an example of resource allocation: from sklearn.linear_model import LinearRegression model = LinearRegression() model.fit(X_train, y_train) # X_train is historical resource data, y_train is load data predictions = model.predict(X_test) # Predict future load demand It should be understood that when the CPU load of the virtual machine is too high, the number of CPU cores of the virtual machine can be increased through the virsh command, as shown in the following example: virsh setmem test-vm 4096M --live Step S104: When the virtual machine load changes or a failure occurs, hot migration of the virtual machine is performed.

[0063] Specifically, hot migration is the hot migration of QEMU-KVM. The hot migration of QEMU-KVM is to transfer the complete information of the virtual machine, such as its memory status, CPU status and connected devices, from one host machine to another host machine while the virtual machine is running, and restore the running status of the virtual machine on the target host machine, thereby realizing seamless migration of the virtual machine.

[0064] In some embodiments, when a virtual machine load changes or a failure occurs, hot migration of the virtual machine is performed, including: Use pacemaker and corosync to monitor virtual machines and initiate hot migration of virtual machines when a failure occurs. When the virtual machine load changes, use the virsh command to perform virtual machine hot migration; Use distributed storage to keep data consistent when hot migrating virtual machines across data centers.

[0065] It should be noted that the fault refers to the fault of the virtual machine itself, as well as the fault of the host machine, network, storage and other components that affect the normal operation of the virtual machine.

[0066] Specifically, pacemaker and corosync are two open source software components commonly used to build high availability (HA) clusters. They are usually used in combination to ensure that services in the cluster can automatically recover and continue to run when a node fails.

[0067] Through high availability frameworks such as corosync and pacemaker, the health of virtual machines and physical hosts is monitored in real time. When a failure is detected, the system will automatically migrate the virtual machine to a backup host. The following is an example: crm configure primitive VM_Migration ocf:pacemaker:RemoteNode \ op monitor interval=10s It should be understood that when the virtual machine load changes, hot migration of the virtual machine can balance the virtual machine load. When a failure occurs, starting hot migration of the virtual machine is conducive to ensuring uninterrupted service.

[0068] When the virtual machine load is high, use the virsh command to perform virtual machine hot migration. The hot migration function migrates the virtual machine to other hosts with lighter loads to avoid overload. The example is as follows: virsh migrate --live test-vm qemu+ssh: / / target_host / system In one possible implementation, distributed storage, such as Ceph, is used to ensure data consistency when migrating across data centers, and SDN technology is used to dynamically adjust the migration bandwidth to avoid bandwidth bottlenecks. Specifically, Ceph is a distributed storage system that supports object storage, block storage, and file storage services. Software Defined Networking (SDN) is an innovative network architecture method and technology that separates the network's control plane from the data forwarding plane to achieve flexible control of network traffic and dynamic allocation of resources.

[0069] It should be understood that automatically optimizing resources according to the virtual machine load is helpful to ensure maximum resource utilization.

[0070] Step S105: Generate a virtual machine performance report based on the resource usage of the virtual machine.

[0071] In one possible implementation, the resource monitoring data of the virtual machine, such as CPU usage, memory usage, disk I / O, network traffic, etc., is analyzed to automatically generate a virtual machine performance report on the health status of the virtual machine, performance bottlenecks, load changes, etc. The report includes detailed statistics on CPU load, memory usage, disk I / O, network bandwidth, etc. The report also points out the resources with bottlenecks on the virtual machine, such as excessive CPU load, high memory pressure, and disk I / O overload. The report also includes load trend prediction content. The steps of load trend prediction are based on historical data and machine learning models to predict load changes in the future period of time, obtain load trend prediction results, and prepare resource scheduling and adjustments for administrators in advance.

[0072] In a possible implementation, a comprehensive performance analysis report is generated based on historical data, trend predictions, and actual load conditions of resource usage of the virtual machine.

[0073] In some embodiments, after generating a virtual machine performance report based on the resource usage of the virtual machine, the virtual machine data processing method further includes pushing the virtual machine performance report to a system administrator or a relevant person in charge through email, message push, Web interface, etc. The full name of Web is World Wide Web, and its Chinese name is Wanweiwang.

[0074] The following are some examples of email push adjustment suggestions: echo "CPU load is too high on test-vm. Consider increasing CPU coresor migrating VM." | mail -s "VM Performance Report - test-vm" admin@example.com In a possible implementation, a visual display is performed in the form of a chart, a data table, etc., so that an administrator can more intuitively analyze the resource usage of the virtual machine.

[0075] Specifically, use Grafana to create a performance dashboard for the virtual machine, display key indicators such as CPU load, memory usage, etc., and provide historical trends. Generate virtual machine performance analysis charts in Grafana regularly and automatically push them to administrators.

[0076] In some embodiments, after generating a virtual machine performance report according to resource usage of the virtual machine, the virtual machine data processing method further includes: adjusting resource allocation of virtual machine resources based on adjustment suggestions in the virtual machine performance report.

[0077] In one possible implementation, Python or other data analysis tools are used to analyze load data based on machine learning frameworks such as scikit-learn to generate adjustment suggestions, including adding resources, adjusting resource configuration, or migrating virtual machines.

[0078] An example of a virtual machine performance report is as follows: { "vm_name": "test-vm", "cpu_usage": "85%", "memory_usage": "75%", "disk_io": "90MB / s", "network_bandwidth": "50Mbps", "performance_bottleneck": "CPU", "suggested_action": "Increase CPU cores to 4 or migrate to a lessloaded host" } In a possible implementation, when the CPU load of a virtual machine exceeds a preset threshold, the system will automatically recommend an adjustment suggestion of increasing the number of CPU cores or migrating the virtual machine to a host with a lower load.

[0079] It should be understood that the VM performance report is generated regularly to analyze the running status of the VM, provide tuning suggestions and optimization strategies, and help administrators improve the VM resource configuration and performance. The report generation cycle can be set according to the needs, and can support daily, weekly or monthly report generation.

[0080] It can be seen that the virtual machine is deployed, the resource usage of the virtual machine is collected and stored, the machine learning algorithm is used to predict the virtual machine load, and the prediction result is obtained. The resource allocation of the virtual machine resources is dynamically adjusted based on the prediction result. When the virtual machine load changes or a failure occurs, the virtual machine is hot migrated, and a virtual machine performance report is generated according to the resource usage of the virtual machine. The resource allocation of the virtual machine resources is dynamically adjusted according to the prediction result, which improves the accuracy and efficiency of the processing results. When the virtual machine load changes, the virtual machine is hot migrated to achieve load balancing, which is conducive to solving the problem of resource waste.

[0081] Reference Figure 4 In an embodiment of the present invention, a virtual machine data processing system is provided for executing the above-mentioned virtual machine data processing method. The virtual machine data processing system includes: A virtual machine deployment module, used to deploy virtual machines; The virtual machine data collection module is used to collect and store the resource usage of the virtual machine; the virtual machine deployment module is connected to the virtual machine data collection module; The resource tuning module is used to adjust the resource allocation of virtual machine resources; the virtual machine data collection module is connected to the resource tuning module; The virtual machine migration module is used to perform hot migration of virtual machines; the resource tuning module is connected to the virtual machine migration module; The report generation module is used to generate a virtual machine performance report; the virtual machine migration module is connected to the report generation module.

[0082] In some embodiments, the virtual machine data collection module is also used to visualize the real-time performance data.

[0083] An embodiment of the present invention further provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned virtual machine data processing method when executing the computer program. The electronic device can be any intelligent terminal including a computer.

[0084] A storage medium is also provided in an embodiment of the present invention. The storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned virtual machine data processing method is implemented.

[0085] It should be appreciated that the method steps in the embodiments of the present invention can be implemented or implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The method can use standard programming techniques. Each program can be implemented in a high-level process or object-oriented programming language to communicate with a computer system. However, if necessary, the program can be implemented in an assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can be run on a programmed ASIC for this purpose.

[0086] In addition, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0087] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention can also include the computer itself.

[0088] The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents physical and tangible objects, including specific visual depictions of physical and tangible objects produced on the display.

[0089] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of protection of the present invention. Within the scope of protection of the present invention, its technical scheme and / or implementation method may have various modifications and changes.

Claims

1. A virtual machine data processing method, characterized in that: include: Deploy virtual machines; Collecting and storing resource usage of the virtual machine; Using a machine learning algorithm to predict virtual machine loads, obtaining prediction results, and dynamically adjusting resource allocation of virtual machine resources based on the prediction results; When the virtual machine load changes or a failure occurs, hot migration of the virtual machine is performed; Generate a virtual machine performance report based on the resource usage of the virtual machine.

2. A virtual machine data processing method according to claim 1, characterized in that: The deploying a virtual machine includes: Creating a virtual machine configuration template, and deploying the virtual machine according to the virtual machine configuration template; According to the predefined operating system configuration, the operating system of the virtual machine is installed and the operating system is initially configured.

3. A virtual machine data processing method according to claim 2, characterized in that: The step of creating a virtual machine configuration template, deploying the virtual machine according to the virtual machine configuration template, and installing the operating system of the virtual machine and initializing the operating system according to a predefined operating system configuration includes: According to the requirements of different virtual machines, create corresponding virtual machine configuration templates to define the virtual machine resources; Use Ansible or Terraform tools to deploy the virtual machine according to the virtual machine configuration template; Use one of cloud-init, PXE and kickstart tools to install the operating system of the virtual machine and initialize the operating system according to the predefined operating system configuration.

4. A virtual machine data processing method according to claim 1, characterized in that: The collecting and storing the resource usage of the virtual machine includes: Using libvirt to collect the resource usage of the virtual machine; The Prometheus time series database is used to store the resource usage of the virtual machine.

5. A virtual machine data processing method according to claim 1, characterized in that: The virtual machine resources include the number of CPU cores, memory size, disk space, operating system type and network configuration.

6. A virtual machine data processing method according to claim 1, characterized in that: The machine learning algorithms include regression analysis and time series analysis; The method of using a machine learning algorithm to predict the virtual machine load includes: The virtual machine load is predicted by combining the regression analysis and the time series analysis.

7. A virtual machine data processing method according to claim 1, characterized in that: When the virtual machine load changes or a failure occurs, hot migration of the virtual machine is performed, including: Use pacemaker and corosync to monitor the virtual machine, and when the failure occurs, start the hot migration of the virtual machine; When the virtual machine load changes, use the virsh command to perform hot migration of the virtual machine; When hot migration of the virtual machine across data centers is performed, distributed storage is used to make data consistent.

8. A virtual machine data processing method according to claim 1, characterized in that: After generating a virtual machine performance report according to the resource usage of the virtual machine, the virtual machine data processing method further includes: The resource allocation of the virtual machine resources is adjusted based on the adjustment suggestion of the virtual machine performance report.

9. A virtual machine data processing system, used to execute a virtual machine data processing method according to any one of claims 1 to 8, characterized in that: The virtual machine data processing system comprises: A virtual machine deployment module, used to deploy the virtual machine; A virtual machine data collection module, used to collect and store the resource usage of the virtual machine; the virtual machine deployment module is connected to the virtual machine data collection module; A resource tuning module, used for adjusting the resource allocation of the virtual machine resources; the virtual machine data acquisition module is connected to the resource tuning module; A virtual machine migration module, used for performing hot migration of the virtual machine; the resource tuning module is connected to the virtual machine migration module; The report generation module is used to generate the virtual machine performance report; the virtual machine migration module is connected to the report generation module.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a virtual machine data processing method according to any one of claims 1 to 8 is implemented.

Images