Thermal migration method and device of virtual equipment, electronic equipment and storage medium
By dynamically adjusting the Ceph snapshot interval and hardware register error rate monitoring methods, combined with zero-copy transmission technology, the problems of business status loss and high bit error rate in the KVM hot migration solution are solved, achieving efficient virtual device migration, and improving the stability of cloud services and user experience.
Patent Information
- Application Number
- CN202510723237.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2025-09-16
AI Technical Summary
In the existing KVM hot migration solution, the snapshot interval of ≥500ms leads to serious business status loss and high bit error rate, affecting user experience and system stability.
By dynamically adjusting the Ceph snapshot interval and monitoring the bit error rate of the physical layer hardware registers, combined with zero-copy transmission technology, the memory dirty page rate and link bit error rate of the virtual device are obtained in real time, a migration trigger signal is generated, and the video memory data is copied to the target node.
It reduces business data loss, lowers system resource overhead, improves cloud service stability and business continuity, and ensures user experience in high real-time scenarios.
Smart Images

Figure CN120653363A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for hot migration of a virtual device, an electronic device, and a storage medium. Background Art
[0002] In the field of virtualization technology, KVM hot migration solutions are widely used in cloud services to ensure business continuity. However, existing solutions have significant flaws: First, snapshot intervals of 500ms or more lead to significant loss of business status. This is especially true in scenarios with extremely high real-time requirements, such as high-frame-rate video processing, where this interval can cause dozens of frames to be lost, severely impacting the user experience. Second, link monitoring methods based on software simulation (such as ethtool) have a false alarm rate of >15%, which can easily trigger a large number of unnecessary migration operations in scenarios with frequent network fluctuations. This not only increases system resource overhead but can also cause service interruptions due to erroneous migrations. Summary of the Invention
[0003] In view of this, an embodiment of the present invention provides a hot migration method, device, electronic device and storage medium for a virtual device to solve the problems of long snapshot intervals leading to serious loss of business status and high false alarm rate of software-simulated link monitoring in existing KVM hot migration solutions in virtualization technology cloud service applications.
[0004] In a first aspect, an embodiment of the present invention provides a method for hot migration of a virtual device, the method comprising:
[0005] Obtain the memory dirty page rate and original snapshot time series of the virtual device;
[0006] When the dirty page rate exceeds a threshold, the Ceph snapshot interval in the original snapshot time series is adjusted to obtain a target snapshot time series, wherein the Ceph snapshot interval in the target snapshot time series is smaller than the Ceph snapshot interval in the original snapshot time series;
[0007] Based on the target snapshot time series, a physical layer hardware register reading method is used to monitor a bit error rate of a transmission link corresponding to the virtual device;
[0008] When the bit error rate is greater than a preset threshold, a migration trigger signal is generated, and the video memory data of the virtual device is copied to the target node according to the migration trigger signal.
[0009] Furthermore, adjusting the Ceph snapshot interval in the original snapshot time series to obtain the target snapshot time series includes:
[0010] The difference between the dirty page rate and a preset threshold is calculated to obtain an excess dirty page rate value, and the excess dirty page rate value is input into a preset interval adjustment function to calculate an interval adjustment ratio.
[0011] Multiplying each Ceph snapshot interval value in the original snapshot time series by the interval adjustment ratio to obtain a first time interval sequence;
[0012] Applying a minimum interval constraint to the first time interval sequence to ensure that each time interval is not less than a set minimum snapshot interval value, thereby obtaining a second time interval sequence;
[0013] The second time interval series is smoothed, and the smoothed time interval series is used as the target snapshot time series.
[0014] Furthermore, the monitoring of the bit error rate of the transmission link corresponding to the virtual device by reading a physical layer hardware register based on the target snapshot time series includes:
[0015] Determine the time point for bit error rate monitoring based on the time interval in the target snapshot time series;
[0016] At each time point, the error counter value of the transmission link corresponding to the virtual device is obtained by reading the physical layer hardware register to obtain the error count data at the current time point;
[0017] Calculating an instantaneous bit error rate within a corresponding time window using the error count data;
[0018] The calculated instantaneous bit error rate is stored in the sliding window array to obtain the updated bit error rate.
[0019] Furthermore, the calculating of the instantaneous bit error rate in the corresponding time window using the error count data includes:
[0020] Get the total number of bits transmitted;
[0021] An instantaneous bit error rate in a corresponding time window is calculated according to the error count difference between the adjacent time points and the total number of transmitted bits.
[0022] Furthermore, copying the video memory data of the virtual device to the target node according to the migration trigger signal includes:
[0023] Determine the virtual device identifier and the target node address to be migrated according to the migration trigger signal;
[0024] Create a transfer channel handle for performing zero-copy transfer of video memory data;
[0025] Extracting a version hash value and a timestamp of the video memory data from the latest Ceph snapshot in the target snapshot time series, and generating a checkpoint identifier for verifying data integrity using the version hash value and the timestamp;
[0026] Calling the transmission channel handle through the DMA engine, directly transferring the video memory data from the source node to the target node address according to the video memory address range, and using the checkpoint identifier to verify whether the data is consistent with the Ceph snapshot during the transmission process, and generating a transmission status report containing the verification result;
[0027] When the transfer status report shows that the data is complete, the virtual device status of the target node is updated to Available.
[0028] Furthermore, the creation of a transmission channel handle for performing zero-copy transmission of video memory data includes:
[0029] Obtaining layout guidance of video memory data of the virtual device using the virtual device identifier;
[0030] Based on the memory partition address and the node address in the layout guide, the GPU driver interface is called to create a transmission channel handle containing DMA transmission rules.
[0031] Furthermore, based on the memory partition address and node address in the layout guide, calling the GPU driver interface to create a transmission channel handle containing DMA transmission rules includes:
[0032] Extract the starting address, size and access permission information of the memory partition from the layout guide to obtain the video memory physical address descriptor;
[0033] Obtaining an RDMA device identifier and available queue pair resources of the target node according to the node address of the target node, and generating remote node connection parameters;
[0034] Using the video memory physical address descriptor, calling the GPU driver's memory registration interface, registering the source node's video memory area as RDMA accessible memory, and obtaining a local memory handle;
[0035] Encapsulating the local memory handle and access permission information into an RDMA memory region descriptor and transmitting it to the target node through a secure channel. After receiving the memory region descriptor, the target node calls the remote memory mapping interface driven by the GPU to map the source node's video memory region to the target node's address space, and generates and feeds back a remote memory access key.
[0036] Combine the remote node connection parameters, local memory handle, and remote memory access key to create a transfer channel handle that contains complete DMA transfer rules.
[0037] In a second aspect, an embodiment of the present invention provides a device for hot migration of a virtual device, the device comprising:
[0038] The acquisition module is used to obtain the memory dirty page rate and original snapshot time series of the virtual device;
[0039] an adjustment module, configured to adjust the Ceph snapshot interval in the original snapshot time series to obtain a target snapshot time series when the dirty page rate exceeds a threshold, wherein the Ceph snapshot interval in the target snapshot time series is smaller than the Ceph snapshot interval in the original snapshot time series;
[0040] An execution module, configured to monitor a bit error rate of a transmission link corresponding to the virtual device by reading a physical layer hardware register based on the target snapshot time sequence;
[0041] The migration module is configured to generate a migration trigger signal when the bit error rate is greater than a preset threshold, and copy the video memory data of the virtual device to a target node according to the migration trigger signal.
[0042] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.
[0044] The embodiments of the present application effectively solve the problems of existing KVM hot migration solutions by dynamically adjusting the snapshot interval and monitoring the bit error rate. On the one hand, the dirty page rate of the virtual device memory is obtained in real time, and the Ceph snapshot interval is shortened when the dirty page rate exceeds the threshold. Compared with the fixed long interval of ≥500ms, it can record the status in time when the business status changes rapidly, reduce the loss of business data such as pictures, and ensure user experience. On the other hand, the software simulation link monitoring is abandoned, and the physical layer hardware register reading method is used to monitor the bit error rate, reduce false alarms caused by network fluctuations, avoid a large number of unnecessary migrations, reduce system resource overhead, prevent service interruptions caused by erroneous migrations, improve system stability and reliability, and ensure the continuity of business in cloud services. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0046] Figure 1is a flow chart of a method for hot migration of a virtual device according to some embodiments of the present invention;
[0047] Figure 2 is a flow chart of another method for hot migration of a virtual device according to some embodiments of the present invention;
[0048] Figure 3 is a flow chart of another method for hot migration of a virtual device according to some embodiments of the present invention;
[0049] Figure 4 2 is a schematic diagram of a framework of a hot migration system for virtual devices according to an embodiment of the present invention;
[0050] Figure 5 is a structural block diagram of a hot migration apparatus for a virtual device according to an embodiment of the present invention;
[0051] Figure 6 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0052] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.
[0053] According to an embodiment of the present invention, a method, apparatus, electronic device and storage medium for hot migration of a virtual device are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0054] In this embodiment, a method for hot migration of a virtual device is provided. Figure 1 FIG. 1 is a flow chart of a method for hot migration of a virtual device according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0055] Step S101 : obtaining a memory dirty page rate and an original snapshot time series of a virtual device.
[0056] First, through the KVM virtualization platform's memory monitoring interface (such as QEMU's monitor command or libvirt's virDomainMemoryStats interface), the virtual device's memory page access log is collected in real time, recording the read and write operation timestamps of each memory page. Then, based on the preset dirty page determination rules (such as whether a memory page has been modified within a specified time window), the ratio of the number of dirty pages per unit time to the total number of memory pages is counted to calculate the memory dirty page rate. At the same time, the snapshot policy configuration currently applied to the virtual device is read from the Ceph storage cluster's metadata server (MDS) and the original snapshot time series is parsed, which includes the preset snapshot interval period (such as the initial value of 500ms) and the corresponding time point record.
[0057] To ensure data accuracy, the calculation of the memory dirty page rate must be combined with the virtual device's memory ballooning mechanism to dynamically adjust the monitored memory range. Specifically, by injecting a balloon driver module into the virtual device, the actual memory range used is obtained in real time, avoiding ineffective monitoring of unallocated or released memory pages. The original snapshot time series must be tied to the virtual device's lifecycle, and snapshot policy change events for the device must be subscribed to through Ceph's RADOS Gateway interface (RGW) to ensure that the latest original configuration data is obtained during the live migration process.
[0058] Step S102: When the dirty page rate exceeds a threshold, the Ceph snapshot interval in the original snapshot time series is adjusted to obtain a target snapshot time series, wherein the Ceph snapshot interval in the target snapshot time series is smaller than the Ceph snapshot interval in the original snapshot time series.
[0059] In an embodiment of the present application, adjusting the Ceph snapshot interval in the original snapshot time series to obtain the target snapshot time series includes the following steps A1-A4:
[0060] Step A1: Calculate the difference between the dirty page rate and a preset threshold value to obtain an excess dirty page rate value, and input the excess dirty page rate value into a preset interval adjustment function to calculate an interval adjustment ratio.
[0061] Specifically, the real-time memory dirty page rate is first subtracted from a preset threshold (e.g., 5%) to obtain the dirty page rate excess value. If the dirty page rate is 8%, the excess value is 3%. This excess value is then input into a preset interval adjustment function, which uses a piecewise linear mapping mechanism: when the excess value is in the 0-5% range, it is mapped at a 1:2 ratio (i.e., every 1% excess value corresponds to a 2% interval reduction ratio); when the excess value exceeds 5%, it is mapped at a 1:3 ratio to accelerate the response. For example, a 3% excess value corresponds to a 6% interval adjustment ratio, and the function output is 0.94 (i.e., the original interval needs to be shortened by 6%).
[0062] Step A2: multiplying each Ceph snapshot interval value in the original snapshot time series by the interval adjustment ratio to obtain a first time interval sequence;
[0063] Specifically, obtain the original snapshot time series from the Ceph storage cluster's metadata server, where each element represents the interval between two consecutive snapshots (e.g., [500ms, 500ms, 500ms]). Multiply each interval value by the interval adjustment ratio (e.g., 0.94) calculated in step A1 to obtain a first time interval sequence. For example, 500ms in the original sequence becomes 470ms after multiplying by 0.94, resulting in the first time interval sequence being [470ms, 470ms, 470ms].
[0064] Step A3: applying a minimum interval constraint to the first time interval sequence to ensure that each time interval is not less than a set minimum snapshot interval value, thereby obtaining a second time interval sequence;
[0065] Specifically, each value in the first time interval sequence is traversed and compared with the system's minimum snapshot interval (e.g., 200ms). If an interval value is less than this minimum, it is forcibly adjusted to 200ms. For example, if the first time interval sequence is [180ms, 470ms, 190ms], the adjusted second time interval sequence is [200ms, 470ms, 200ms], ensuring that all intervals meet the system's fault tolerance requirements.
[0066] Step A4: smoothing the second time interval sequence, and using the smoothed time interval sequence as the target snapshot time series.
[0067] Specifically, a moving average filtering algorithm is applied to the second time interval series, with a window size of 3. The average value within each window is calculated and the intermediate value is replaced. For example, the sequence [200ms, 470ms, 200ms] becomes [200ms, 290ms, 290ms] after processing. If the sequence length is less than the window size, the arithmetic mean is directly taken. Finally, the smoothed sequence is output as the target snapshot time series, ensuring continuous and predictable interval changes and preventing large fluctuations that affect storage performance.
[0068] In another embodiment of the present application, the Ceph snapshot interval value in the original snapshot time series can also be obtained, and the dirty page rate can be monitored in real time. The adjusted interval value is calculated according to the dynamic adjustment snapshot interval formula tinterval = tbase × (1-dr / 10) (tbase = 100ms, dr is the percentage of dirty page rate). The original snapshot interval value is replaced with the calculated new interval value to obtain the target snapshot time series. Compared with the traditional fixed or simple linear adjustment method, this method based on dynamic adjustment of dirty page rate can accurately balance the risk of business status loss and system resource usage. For example, when the dirty page rate is 2%, the new interval of 80ms can reduce business status loss and avoid resource waste.
[0069] Step S103 : Based on the target snapshot time sequence, a physical layer hardware register reading method is adopted to monitor the bit error rate of the transmission link corresponding to the virtual device.
[0070] In the embodiments of this application, Figure 2 As shown, the system continuously monitors link status and periodically calculates the bit error rate (BER) of data transmitted over the link. After calculating the BER, it uses a sliding window filtering algorithm to smooth the data and remove noise. The processed BER is then compared with a preset threshold of 1×10⁻⁵. If the BER is below the threshold, the system resumes monitoring the link status. If the BER is above the threshold, the system determines that the link quality has degraded and triggers the virtual device migration process to ensure normal service operation.
[0071] In an embodiment of the present application, based on a target snapshot time sequence, a physical layer hardware register reading method is used to monitor the bit error rate of a transmission link corresponding to a virtual device, including: determining a time point for bit error rate monitoring based on the time interval in the target snapshot time sequence; obtaining the error counter value of the transmission link corresponding to the virtual device at each time point through a physical layer hardware register reading method to obtain error count data at the current time point; calculating the instantaneous bit error rate within the corresponding time window using the error count data; and storing the calculated instantaneous bit error rate in a sliding window array to obtain an updated bit error rate. Calculating the instantaneous bit error rate within the corresponding time window using the error count data includes: obtaining the total number of transmitted bits; and calculating the instantaneous bit error rate within the corresponding time window based on the difference in error counts between adjacent time points and the total number of transmitted bits.
[0072] Based on the time intervals in the target snapshot time sequence, with the sequence start time as the benchmark, the interval values are sequentially superimposed to generate a list of monitoring time points (e.g., t0 = 0ms, t1 = 200ms, t2 = 490ms, t3 = 780ms). These time points are used to trigger subsequent hardware register read operations to ensure synchronization between the monitoring cycle and the snapshot cycle.
[0073] At each monitoring time point (e.g., t1 = 200ms), the accumulated number of error bits in the transmission link between the virtual device and the GPU is read through the PCIe physical layer hardware register (e.g., the error counter register of the PCIExpressRootComplex) (e.g., the current value is ErrCount1). This value is subtracted from the error count at the previous monitoring time point (e.g., ErrCount0 at t0 = 0ms) to obtain the error count difference between adjacent time points, ΔErr = ErrCount1 - ErrCount0, which is used as the error bit count statistic within the time window.
[0074] Obtain the total number of bits transmitted (TotalBits) within the corresponding time window from the performance monitoring module. This value can be obtained through the PCIe controller's traffic statistics register or the GPU driver's bandwidth monitoring interface. Calculate the bit error rate within the window using the formula: Instantaneous Bit Error Rate (BER) = ΔErr / TotalBits.
[0075] The calculated instantaneous bit error rates are stored in a preset sliding window array (e.g., an array of size 10), and the array contents are updated using a first-in, first-out (FIFO) mechanism. For example, if the original array values are [BER1, BER2, ..., BER9], after the newly calculated BER10 is stored, the array is updated to [BER2, ..., BER9, BER10]. This updated array is used for subsequent average calculations to smooth short-term fluctuations and reflect link quality trends.
[0076] Step S104 : When the bit error rate is greater than a preset threshold, a migration trigger signal is generated, and the video memory data of the virtual device is copied to the target node according to the migration trigger signal.
[0077] In the embodiment of the present application, copying the memory data of the virtual device to the target node according to the migration trigger signal includes the following steps B1-B5:
[0078] Step B1: Determine the virtual device identifier to be migrated and the node address of the target node according to the migration trigger signal.
[0079] Specifically, first, a migration trigger signal is received, which contains relevant metadata of the virtual device; then, the virtual device identifier (such as UUID or unique number) to be migrated and the node address of the target node (including IP address and port number) are parsed and extracted from the metadata of the signal to provide target information for subsequent migration operations.
[0080] Step B2: Create a transmission channel handle for performing zero-copy transmission of video memory data.
[0081] Specifically, creating a transmission channel handle for performing zero-copy transmission of video memory data includes: using a virtual device identifier to obtain a layout guide for the video memory data of the virtual device; based on the memory partition address and node address in the layout guide, calling a GPU driver interface to create a transmission channel handle containing DMA transmission rules.
[0082] Among them, based on the memory partition address and node address in the layout guide, the GPU driver interface is called to create a transmission channel handle containing DMA transmission rules, including: extracting the starting address, size and access permission information of the memory partition from the layout guide to obtain the video memory physical address descriptor; obtaining the RDMA device identifier and available queue pair resources of the target node according to the node address of the target node, and generating remote node connection parameters; using the video memory physical address descriptor to call the GPU driver's memory registration interface, register the source node's video memory area as RDMA accessible memory, and obtain the local memory handle; encapsulate the local memory handle and access permission information into an RDMA memory area descriptor, and transmit it to the target node through a secure channel. After receiving the memory area descriptor, the target node calls the GPU driver's remote memory mapping interface to map the source node's video memory area to the address space of the target node, generate and feedback the remote memory access key; the remote node connection parameters, local memory handle and remote memory access key are combined to create a transmission channel handle containing complete DMA transmission rules.
[0083] Through a virtualization management platform (such as OpenStack Nova or KVM's libvirt interface), enter the virtual device's unique identifier (such as VMUUID) and call the device metadata query interface. The returned metadata is parsed to extract the physical layout of the video memory, including the starting address, size, and access permissions (e.g., read-only, writable) of partitions such as the frame buffer, texture memory, and register memory. A structured video memory layout guide document (e.g., in JSON format) is generated to guide subsequent transmission path configuration.
[0084] Extract the starting address (such as 0x10000000), size (such as 128MB) and access permission (such as writable) of each memory partition from the layout guide document, combine them into a video memory physical address descriptor array, and clearly define the range of the video memory area that needs to be transferred.
[0085] According to the IP address or host name of the target node, the cluster management system queries its RDMA device information and available queue pair (QP) resources, and generates remote node connection parameters including device identifier, QP number and port number.
[0086] Call the GPU driver's memory registration interface (such as NVIDIA's nvmlMemoryRegister or AMD's amdgpu_ioctl), pass in the starting address and size in the video memory physical address descriptor, register the source node's video memory area as RDMA-accessible memory, and obtain the local memory handle (such as a file descriptor or memory handle value).
[0087] The local memory handle and access permission information are encapsulated into an RDMA memory region descriptor (MRD) and transmitted to the target node over a TLS / SSL encrypted channel. After receiving the MRD, the target node calls the local GPU driver's remote memory mapping interface (such as rdma_registration_create) to map the source node's memory region to the local address space, generate a remote memory access key (such as a key value or token), and return it to the source node.
[0088] Combine the remote node connection parameters, local memory handle, and remote memory access key in a fixed format (such as a structure or dictionary) to create a transmission channel handle containing complete DMA transmission rules. The handle contains information such as source / destination address, transmission protocol (such as RoCEv2), security credentials, etc., which is called by the DMA engine to perform zero-copy transmission.
[0089] The embodiments of the present application effectively solve the problems of existing KVM hot migration solutions by dynamically adjusting the snapshot interval and monitoring the bit error rate. On the one hand, the dirty page rate of the virtual device memory is obtained in real time, and the Ceph snapshot interval is shortened when the dirty page rate exceeds the threshold. Compared with the fixed long interval of ≥500ms, it can record the status in time when the business status changes rapidly, reduce the loss of business data such as pictures, and ensure user experience. On the other hand, the software simulation link monitoring is abandoned, and the physical layer hardware register reading method is used to monitor the bit error rate, reduce false alarms caused by network fluctuations, avoid a large number of unnecessary migrations, reduce system resource overhead, prevent service interruptions caused by erroneous migrations, improve system stability and reliability, and ensure the continuity of business in cloud services.
[0090] As an example, Figure 3 As shown in the figure, as virtual devices migrate, the system records the number of migrations and the cumulative migration time in real time. When the number of migrations reaches 99, although the "number of migrations ≥ 100" condition is not met, monitoring continues. When the number of migrations reaches 100, a check of the cumulative migration time is triggered. If the cumulative migration time is 50 minutes at this point, the "cumulative migration time ≥ 55 minutes" condition is not met, and the migration operation is performed directly. If the cumulative migration time reaches 55 minutes, the system will rotate the access key and only execute the migration operation after the new migration key is replaced to ensure the security and stability of the migration process.
[0091] Step B3: extract the version hash value and timestamp of the video memory data from the latest Ceph snapshot in the target snapshot time series, and use the version hash value and timestamp to generate a checkpoint identifier for verifying data integrity.
[0092] Specifically, first, locate the latest Ceph snapshot in the target snapshot time series; second, extract the version hash value (such as the MD5 or SHA-256 value calculated by the hash algorithm on the memory data) and timestamp (recording the specific time when the snapshot was generated) corresponding to the memory data from the metadata of the latest snapshot; then, combine the version hash value and timestamp and encode them in a specific format (such as JSON or XML format) to generate a checkpoint identifier for verifying data integrity, ensuring that the data can be verified to be consistent with the snapshot time during data transmission.
[0093] Step B4, calling the transmission channel handle through the DMA engine, transfers the video memory data directly from the source node to the target node address according to the video memory address range, and uses the checkpoint identifier to verify whether the data is consistent with the Ceph snapshot during the transmission process, and generates a transmission status report containing the verification results.
[0094] Specifically, the specific implementation process of step B4 is as follows: first, the created transmission channel handle is called through the DMA engine to activate the zero-copy data transmission channel; second, according to the video memory address range determined by the video memory layout description, the source address (source node video memory starting address) and target address (target node corresponding video memory starting address) of the DMA transmission are set; then, the DMA transmission is started to transfer the video memory data directly from the source node to the target node address; during the transmission process, each time a certain amount of data (such as a data block) is transmitted, the hash value of the transmitted data is calculated and compared with the version hash value in the checkpoint identifier; at the same time, the error information during the transmission process (such as the number of transmission failures, the number of data mismatches, etc.) is recorded; finally, based on the hash value comparison result and the error information, a transmission status report containing the verification result (pass / fail), the number of transferred bytes, error details, etc. is generated.
[0095] Step B5: When the transmission status report shows that the data is complete, the virtual device status of the target node is updated to available.
[0096] Specifically, first, receive the transmission status report and check the verification results in the report; if the verification results show that the data is complete, that is, no errors occurred in the data during the transmission process and it is consistent with the Ceph snapshot; then through the virtualization management platform (such as OpenStack or KVM management tools), update the status flag of the virtual device on the target node, set it to an available state, complete the deployment of the virtual device on the target node, and enable it to provide services to the outside world.
[0097] For example, in a cloud gaming scenario, a player's device initiates a 4K / 120fps game session. The system monitors the virtual device status in real time. When the GPU temperature exceeds 85°C, a migration process is triggered. Previously, the system dynamically optimized the snapshot interval based on the dirty page rate. For example, with an 8% dirty page rate, 48 frames would be lost in the original snapshot interval. After adjustment, only 3.8 frames were lost, while CPU utilization was reduced by 41%. During the migration process, the GPU VRAM DMA engine was used to directly transfer video memory data, maintaining H.264 / H.265 encoder context continuity. Combined with frame rate synchronization technology, VSync signals were used to align client frame refreshes, keeping latency fluctuations below 8.3ms. Ultimately, after the migration, the game's lag rate dropped to below 0.1%, significantly improving player QoE and ensuring the smoothness and stability of the cloud gaming service.
[0098] The method provided in the embodiment of the present application first constructs an adaptive adjustment model driven by the dirty page rate at the snapshot level, and dynamically compresses the original fixed interval of ≥500ms to the range of 38-80ms through the interval adjustment function (such as triggering a 41% interval reduction when the dirty page rate is 8%). Combined with minimum constraints and smoothing processing, it not only avoids CPU resource consumption caused by frequent snapshots (reduction rate of up to 47%), but also controls the amount of business status loss to within 10% of the traditional solution (such as from 30 frames to 2.1 frames when the dirty page rate is 5%). Secondly, a hardware-level perception architecture is adopted at the link monitoring level, and the physical layer error data is directly obtained through the error counter of the PCIe controller. Compared with software simulation solutions such as ethtool, the false alarm rate is reduced from more than 15% to less than 0.5%, reducing 96.7% of invalid migration triggers. Finally, at the data transmission level, through the RDMA zero-copy channel and GPU video memory direct mapping technology, the video memory data transmission delay is controlled within 8.3ms. Combined with the Ceph snapshot hash verification mechanism, microsecond-level data consistency verification is achieved, ensuring the continuity of the H.264 / H.265 encoder context during the migration process. Ultimately, the lag rate in high-real-time scenarios such as cloud gaming is reduced from 1.2% in traditional solutions to below 0.1%, forming a full-link optimization closed loop from monitoring, decision-making to execution.
[0099] Figure 4 This embodiment of the present application provides a framework diagram of a hot migration system for virtual devices, such as Figure 4 As shown: The system includes:
[0100] Storage layer: Provides reliable block storage services. By dynamically adjusting the snapshot interval, it can flexibly control the frequency of data snapshots based on system conditions, improving storage efficiency and data security.
[0101] Monitoring layer: The PCIe link status monitoring module monitors the working status of the PCIe link in real time and promptly detects link anomalies; the SGX encrypted channel uses Intel Software Guard Extensions technology to provide encryption protection for data during transmission and other processes to prevent data leakage.
[0102] Migration Execution Layer: QEMU hot migration technology supports the migration of virtual devices from one physical host to another during operation, ensuring uninterrupted services. The NVMe over Fabrics protocol enables high-speed, low-latency data transmission, facilitating efficient data movement during virtual device migration. These modules work together to ensure stable virtual device operation, secure data storage, and efficient migration.
[0103] In this system, the monitoring layer and the migration execution layer communicate using an efficient shared memory mechanism. This mechanism creates a shared memory object using the shm_open function in the POSIX shared memory interface, and then uses the mmap function to map the shared memory into the process address space, enabling inter-process communication. This communication method reduces latency to less than 1μs, significantly improving the efficiency of data exchange between the monitoring layer and the migration execution layer compared to traditional communication methods, ensuring rapid system response and stable operation.
[0104] The secure isolation module incorporates advanced quantum security technology and features a built-in quantum security extension unit. Based on the SGX enclave's Enclave Definition Language (EDL), the module implements hardware-level isolation for Kyber-512 key encapsulation and Dilithium3 digital signatures, ensuring that key and signature operations are performed in a secure environment. The sealed storage area utilizes a 256-bit Monotonic Counter design, resistant to physical attacks, effectively defending against physical-level attacks. Furthermore, the system supports a hybrid encryption mode for quantum-resistant cryptographic algorithms. The traditional encryption layer uses the AES-256-GCM algorithm to encrypt business data, while the quantum encryption layer uses Kyber-512 to encapsulate the AES key. The two ciphertext layers are nested and encoded in the ASN.1DER format, balancing security and compatibility to provide a strong defense for system data security.
[0105] This embodiment also provides a hot migration device for a virtual device, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.
[0106] This embodiment provides a hot migration device for a virtual device, such as Figure 5 Shown, including:
[0107] An acquisition module 501 is used to acquire a memory dirty page rate and an original snapshot time series of a virtual device;
[0108] An adjustment module 502 is configured to adjust the Ceph snapshot interval in the original snapshot time series to obtain a target snapshot time series when the dirty page rate exceeds a threshold, wherein the Ceph snapshot interval in the target snapshot time series is smaller than the Ceph snapshot interval in the original snapshot time series;
[0109] An execution module 503 is configured to monitor a bit error rate of a transmission link corresponding to the virtual device by reading a physical layer hardware register based on a target snapshot time sequence;
[0110] The migration module 504 is configured to generate a migration trigger signal when the bit error rate is greater than a preset threshold, and copy the video memory data of the virtual device to the target node according to the migration trigger signal.
[0111] In this embodiment of the present application, adjustment module 502 is configured to calculate the difference between the dirty page rate and a preset threshold value to obtain an excess dirty page rate value, and input the excess dirty page rate value into a preset interval adjustment function to calculate an interval adjustment ratio. A first time interval sequence is obtained by multiplying each Ceph snapshot interval value in the original snapshot time series by the interval adjustment ratio. A minimum interval constraint is applied to the first time interval sequence to ensure that each time interval is no less than the preset minimum snapshot interval value, thereby obtaining a second time interval sequence. The second time interval sequence is then smoothed, and the smoothed time interval sequence is used as the target snapshot time series.
[0112] In an embodiment of the present application, the execution module 503 is used to determine the time point for bit error rate monitoring based on the time interval in the target snapshot time series; at each time point, the error counter value of the transmission link corresponding to the virtual device is obtained by reading the physical layer hardware register to obtain the error count data at the current time point; the instantaneous bit error rate within the corresponding time window is calculated using the error count data; the calculated instantaneous bit error rate is stored in the sliding window array to obtain the updated bit error rate.
[0113] In the embodiment of the present application, the execution module 503 is used to obtain the total number of transmitted bits; and the instantaneous bit error rate in the corresponding time window is calculated based on the error count difference between adjacent time points and the total number of transmitted bits.
[0114] In an embodiment of the present application, the migration module 504 is used to determine the virtual device identifier and the node address of the target node that need to be migrated according to the migration trigger signal; create a transmission channel handle for performing zero-copy transmission of video memory data; extract the version hash value and timestamp of the video memory data from the latest Ceph snapshot in the target snapshot time series, and use the version hash value and timestamp to generate a checkpoint identifier for verifying data integrity; call the transmission channel handle through the DMA engine, and transfer the video memory data directly from the source node to the target node address according to the video memory address range, and use the checkpoint identifier to verify whether the data is consistent with the Ceph snapshot during the transmission process, and generate a transmission status report containing the verification results; when the transmission status report shows that the data is complete, update the virtual device status of the target node to available.
[0115] In an embodiment of the present application, the migration module 504 is used to use the virtual device identifier to obtain the layout guidelines of the virtual device's video memory data; based on the memory partition address and node address in the layout guidelines, call the GPU driver interface to create a transmission channel handle containing DMA transmission rules.
[0116] In an embodiment of the present application, the migration module 504 is used to extract the starting address, size and access permission information of the memory partition from the layout guide to obtain a video memory physical address descriptor; obtain the RDMA device identifier and available queue pair resources of the target node according to the node address of the target node, and generate remote node connection parameters; use the video memory physical address descriptor to call the GPU-driven memory registration interface to register the video memory area of the source node as RDMA-accessible memory and obtain a local memory handle; encapsulate the local memory handle and access permission information into an RDMA memory area descriptor and transmit it to the target node through a secure channel, wherein, after receiving the memory area descriptor, the target node calls the GPU-driven remote memory mapping interface to map the video memory area of the source node to the address space of the target node, generate and feedback a remote memory access key; combine the remote node connection parameters, the local memory handle and the remote memory access key to create a transmission channel handle containing complete DMA transmission rules.
[0117] See 6, Figure 6 is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 6As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).
[0118] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.
[0119] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.
[0120] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created based on the use of an electronic device presented by a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
[0121] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.
[0122] The electronic device further includes a communication interface 30 for the electronic device to communicate with other devices or a communication network.
[0123] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.
[0124] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for hot migration of a virtual device, characterized in that: The method comprises: Obtain the memory dirty page rate and original snapshot time series of the virtual device; When the dirty page rate exceeds a threshold, the Ceph snapshot interval in the original snapshot time series is adjusted to obtain a target snapshot time series, wherein the Ceph snapshot interval in the target snapshot time series is smaller than the Ceph snapshot interval in the original snapshot time series; Based on the target snapshot time series, a physical layer hardware register reading method is used to monitor a bit error rate of a transmission link corresponding to the virtual device; When the bit error rate is greater than a preset threshold, a migration trigger signal is generated, and the video memory data of the virtual device is copied to the target node according to the migration trigger signal.
2. The method according to claim 1, characterized in that The adjusting of the Ceph snapshot interval in the original snapshot time series to obtain the target snapshot time series includes: Calculate the difference between the dirty page rate and a preset threshold value to obtain an excess dirty page rate value, and input the excess dirty page rate value into a preset interval adjustment function to calculate an interval adjustment ratio; Multiplying each Ceph snapshot interval value in the original snapshot time series by the interval adjustment ratio to obtain a first time interval sequence; Applying a minimum interval constraint to the first time interval sequence to ensure that each time interval is not less than a set minimum snapshot interval value, thereby obtaining a second time interval sequence; The second time interval series is smoothed, and the smoothed time interval series is used as the target snapshot time series.
3. The method according to claim 1, characterized in that The monitoring of the bit error rate of the transmission link corresponding to the virtual device by reading a physical layer hardware register based on the target snapshot time series includes: Determine the time point for bit error rate monitoring based on the time interval in the target snapshot time series; At each time point, the error counter value of the transmission link corresponding to the virtual device is obtained by reading the physical layer hardware register to obtain the error count data at the current time point; Calculating an instantaneous bit error rate within a corresponding time window using the error count data; The calculated instantaneous bit error rate is stored in the sliding window array to obtain the updated bit error rate.
4. The method according to claim 3, characterized in that The calculating the instantaneous bit error rate in the corresponding time window by using the error count data includes: Get the total number of bits transmitted; The instantaneous bit error rate in the corresponding time window is calculated based on the error count difference between adjacent time points and the total number of transmitted bits.
5. The method according to claim 1, wherein The copying of the video memory data of the virtual device to the target node according to the migration trigger signal includes: Determine the virtual device identifier to be migrated and the node address of the target node according to the migration trigger signal; Create a transfer channel handle for performing zero-copy transfer of video memory data; Extracting a version hash value and a timestamp of the video memory data from the latest Ceph snapshot in the target snapshot time series, and generating a checkpoint identifier for verifying data integrity using the version hash value and the timestamp; Calling the transmission channel handle through the DMA engine, directly transferring the video memory data from the source node to the target node address according to the video memory address range, and using the checkpoint identifier to verify whether the data is consistent with the Ceph snapshot during the transmission process, and generating a transmission status report containing the verification result; When the transfer status report shows that the data is complete, the virtual device status of the target node is updated to Available.
6. The method according to claim 5, characterized in that The step of creating a transmission channel handle for performing zero-copy transmission of video memory data includes: Obtaining layout guidance of video memory data of the virtual device using the virtual device identifier; Based on the memory partition address and node address in the layout guide, the GPU driver interface is called to create a transmission channel handle containing DMA transmission rules.
7. The method according to claim 6, characterized in that The step of calling the GPU driver interface to create a transmission channel handle including DMA transmission rules based on the memory partition address and the node address in the layout guide includes: Extract the starting address, size and access permission information of the memory partition from the layout guide to obtain the video memory physical address descriptor; Obtaining an RDMA device identifier and available queue pair resources of the target node according to the node address of the target node, and generating remote node connection parameters; Using the video memory physical address descriptor, calling the GPU driver's memory registration interface, registering the source node's video memory area as RDMA accessible memory, and obtaining a local memory handle; Encapsulating the local memory handle and access permission information into an RDMA memory region descriptor and transmitting it to the target node through a secure channel. After receiving the memory region descriptor, the target node calls the remote memory mapping interface driven by the GPU to map the source node's video memory region to the target node's address space, and generates and feeds back a remote memory access key. Combine the remote node connection parameters, local memory handle, and remote memory access key to create a transfer channel handle that contains complete DMA transfer rules.
8. A hot migration device for a virtual device, characterized in that: The device comprises: The acquisition module is used to obtain the memory dirty page rate and original snapshot time series of the virtual device; an adjustment module, configured to adjust the Ceph snapshot interval in the original snapshot time series to obtain a target snapshot time series when the dirty page rate exceeds a threshold, wherein the Ceph snapshot interval in the target snapshot time series is smaller than the Ceph snapshot interval in the original snapshot time series; An execution module, configured to monitor a bit error rate of a transmission link corresponding to the virtual device by reading a physical layer hardware register based on the target snapshot time sequence; The migration module is configured to generate a migration trigger signal when the bit error rate is greater than a preset threshold, and copy the video memory data of the virtual device to a target node according to the migration trigger signal.
9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.
