54 results about "Dirty page" patented technology

The invention relates to the technical field of databases, and provides a method and a device for quickly dropping write-intensive load data into a disk. The method comprises the following steps: acquiring an actual storage position of dirty page data, and determining a processing sequence of the dirty page data; according to the processing sequence, writing the dirty page data into a virtual storage area from a data buffer area, determining a virtual storage position of the dirty page data, and associating the virtual storage position with the actual storage position; a calculation unit of the virtual storage area obtains the actual storage position, and writes the dirty page data into the actual storage position from the virtual storage position; wherein a storage medium of the virtual storage area is an intelligent solid state disk; the problems that in the prior art, due to the fact that random write operation cannot be avoided, and page splitting or merging operation of a B + tree cannot be avoided, write blocking is serious, and the performance of a database system is sharply reduced are solved.

The invention discloses a multi-model cache management algorithm based on online learning, which comprises the following steps: S1, an SSD (Solid State Disk) controller receives an I / O (Input / Output) request sent by an upper-layer application and captures a request feature; s2, after a fixed number of pages are processed every time, the request features are aggregated to generate window-level indexes; s3, performing discretization processing, and mapping into a finite state space; s4, the Q table calculates the predicted value of each strategy in the basic strategy pool in the current state; s5, the online learning module dynamically selects a cache replacement strategy of the next period in the basic strategy pool through the predicted value; and S6, additionally setting a preferential expelling chain table in the cache, monitoring the busy degree of a rear-end flash memory chip in real time by an active write-back module, and asynchronously writing cold dirty pages in the preferential expelling chain table back to the flash memory. According to the method, the cache hit rate is improved through dynamic strategy switching; the active write-back module reduces dirty page replacement delay and prolongs the service life of the SSD; the total memory occupation of the Q table and the Xgboost model is small, and the method adapts to the resource limitation of the embedded controller.

The invention discloses a three-level cache replacement method based on dirty page perception and adaptive capacity, and belongs to the technical field of computer memory cache management. The method comprises the following steps of: constructing a short-time-efficiency queue, a medium-time-efficiency queue and a global queue in a CXL extended memory equipment side cache; executing object insertion, migration and metadata updating according to the hit condition of the access request in each queue; when the cache space is insufficient, the access count, the dirty page mark and the dirty page revival mark are combined to execute eviction judgment; and calculating a return visit ratio based on global queue identifier insertion times and return visit times, and dynamically adjusting short-time-efficiency and medium-time-efficiency queue capacities. According to the method, the problems of high dirty page write-back cost and inflexible cache space distribution in the CXL extended memory are solved, and the redundant write-back overhead is reduced, the utilization efficiency of the cache space is improved and the access delay is reduced through collaborative optimization of the access characteristics, the dirty page state and the elimination return visit behavior.

The invention discloses an online migration method, device and equipment of a virtual machine, and relates to the technical field of virtualization, the method comprises the following steps: in an online migration process of a source virtual machine, according to memory static page information of the source virtual machine and a target virtual machine, transmitting full memory data of the source virtual machine to the target virtual machine; according to the dirty marking frequency of each dirty memory page in the source virtual machine, obtaining a page type corresponding to each dirty memory page; according to the page type corresponding to each dirty memory page, determining a to-be-transmitted dirty page of the iteration transmission; if the current iteration transmission is not the last iteration transmission, transmitting the dirty page to be transmitted of the current iteration transmission to the target virtual machine; if the current iteration transmission is the last iteration transmission, pausing the source virtual machine, and transmitting the dirty page to be transmitted of the current iteration transmission to the target virtual machine; according to the method, the data volume needing to be transmitted in the online migration process of the virtual machine can be reduced, the migration speed is increased, and the influence on the system performance is reduced.

A computing system may access a source device configured to store device traffic in a source device queue. The computing system may launch, by one or more worker nodes of a control plane of the computing device, a destination device. The computing system may transmit a source device state to the destination device. The control plane worker nodes may configure the destination device from the source device state. The computing system may instruct the source device to pause storing the device traffic to the source device guest memory. The computing system may store dirty page information in the computing system memory. The computing system may transmit the dirty page information to the destination guest memory. The computing system may instruct the destination device to resume storing the device traffic to the destination guest memory.

Embodiments described herein are generally directed to an improved workload submission handling strategy for workloads received during live migration and targeting VFs of a hardware accelerator. In an example, while performing a live migration of a source VM running on a source host to a destination VM of a destination host, a new workload targeting a VF of a first HW accelerator of the source host is identified by a HW status manager of the source host by trapping the workload submission channel. Based on a nature of the new workload, the HW status manager determines whether to transfer the new workload to the destination host. Responsive to an affirmative determination, the HW status manager causes the new workload to be submitted to a VF of a second HW accelerator of the destination host by incorporating information regarding the new workload within a migration stream associated with the live migration.

The invention relates to a database fault recovery method and device, a storage medium and electronic equipment. The method comprises the steps that when a database normally runs, a WAL log is written into a disk, and meanwhile, a log record is pre-analyzed through a background asynchronous thread to extract core semantic information of the log record; constructing a cache structure sorted according to table space identifiers and data page numbers in the shared memory, and storing the core semantic information in the cache structure in a recovery-oriented compact format; when the check point is executed, the memory dirty page is flashed to a disk, and the cache initial position corresponding to the check point in the cache structure serves as metadata to be written into a control file; and after the database crashes and is restarted, acquiring a cache initial position from the control file, loading all core semantic information entries behind the cache initial position in the cache structure according to the cache initial position, positioning a target data page, and modifying the target data page. The technical problem that an existing database collapse recovery technology is low in recovery speed is solved.

The application discloses a file unshelling method and electronic equipment, and relates to the technical field of computer security, and comprises the following steps: inserting a probe at a target system call of a kernel, and injecting the probe into a target program; identifying a suspiciously shelled process through the target program; monitoring a memory permission change behavior performed by the suspiciously shelled process through the target program, and counting a permission change frequency of a memory region, an entropy value of the memory region and a call chain depth; when it is confirmed that the suspiciously shelled process is in an unshelling stage according to the permission change frequency, the entropy value and the call chain depth, writing data decrypted by the suspiciously shelled process into a target buffer through the target program; reading the data from the target buffer in a user mode, and recombining the data into a memory image according to a base address and a dirty page bitmap and writing the memory image back to a file. The method can avoid I / O bottlenecks caused by full memory dumping, reduce invalid data in the user mode, and reduce CPU occupation and memory bandwidth.

The invention provides a data processing method and device for a relational memory database. The method comprises the steps of obtaining two groups of check point files and initialization variables corresponding to a memory table space in a relational memory database; obtaining an active file group and a static file group according to the two groups of check point files and the initialization variable; according to the initialization variable, alternately writing dirty pages generated in the operation process of the relational memory database into the active file group and the static file group to obtain an updated active file group and an updated static file group; performing disk mapping processing according to the updated active file group and the updated static file group to obtain target disk data; and performing data recovery according to a data recovery request and the target disk data. According to the method, the security and stability of the system are improved, and the data recovery efficiency and accuracy can be improved when the system crashes.

This invention relates to the field of data flow control technology, and more particularly to a method, apparatus, device, and medium for flow control of a data buffer. Based on the number of first dirty pages at the previous time step, the number of second dirty pages at the current time step, the rate of change of the number of dirty pages at the previous time step, and the rate of change of the number of dirty pages at the current time step, the target flow control delay for the current time step is calculated. The flow of the data buffer is controlled according to the target flow control delay. Based on the amount of dirty page data and the rate of change of the number of dirty pages at adjacent time steps, a target flow control strategy for the database buffer is determined. Based on the target flow control strategy, the target flow control delay for the current time step is calculated. The flow of the data buffer is controlled according to the target flow control delay. By periodically calculating the buffer flow control delay, the data buffer automatically and dynamically adjusts the buffer allocation delay according to business pressure, thereby avoiding performance fluctuations caused by buffer exhaustion due to excessive business pressure.

A physical host includes a memory, a first buffer, a second buffer, a third buffer and a processor. The first buffer stores a log regarding dirty pages. The second buffer stores a dirty bitmap, where the dirty bitmap is written into the second buffer according to the log read from the first buffer. The third buffer stores the dirty bitmap. The processor obtains the current memory address to be migrated and a destination memory address, and marks a page table corresponding to the memory address to be migrated as dirty pages and writes the log marked as the dirty pages into the first buffer when the memory address to be migrated is written. The processor includes a memory copy engine for reading the dirty bitmap from the third buffer, and copying the content corresponding to the dirty pages to the destination memory according to the dirty bitmap.

The invention relates to a data migration method and device of a virtual machine, a storage medium and electronic equipment. The method comprises the steps that when dirty page data of the virtual machine is migrated, the current data size of the dirty page data is obtained, and the dirty page data are data which are inconsistent in a memory and a disk of the virtual machine; determining sending times of dirty page data according to the current data size and the limited bandwidth size; and under the condition that the sending frequency is multiple times, limiting the speed of a virtual central processing unit of the virtual machine so as to reduce the generation speed of the dirty page data until the migration of the dirty page data is completed. The technical problem that online migration of the virtual machine cannot be completed due to too fast generation of dirty pages is solved.

The invention provides a dirty page brushing method and a related product. The dirty page disk brushing method is applied to any node in a database cluster, and comprises the following steps: responding to a global check point process started by the database cluster; participating in the collection of redo sites by the database cluster; node-level dirty page disc brushing is carried out for multiple times; synchronizing dirty page disk brushing progress of the buffer area with other nodes; and performing one-time cluster-level dirty page disk brushing with other nodes. The method has the advantage that the dirty page brushing efficiency during the global check point period can be improved.

The present invention discloses a method and device for data management optimization, which includes: initializing the acceleration card storage, host cache page information, the host page cache and the acceleration card page management information according to the page attribute data structure, the capacity of the acceleration card storage and the capacity of the host page cache; receiving and sending an operation instruction to the accelerator, the accelerator processing the data page according to the type of the operation instruction and the page attribute data structure, marking the data page as a dirty page, forming dirty page data, and returning the processing result to the host; and simultaneously returning dirty page modification information corresponding to the operation instruction, caching the dirty page data in the host page cache, and directly reading the dirty page data from the host page cache when the dirty page data is written back, thereby reducing the communication overhead of data interaction and improving the data processing performance and reliability of the database system.

An input / output memory management unit (IOMMU) comprises address translation circuitry to control address translation in response to input / output memory system transactions issued by one or more input / output devices; at least one dirty page tracking base address register to store a base address of a dirty page buffer structure for tracking addresses of dirty pages of address space; and dirty page tracking circuitry to: detect whether an input / output memory system transaction associated with a given translation context for which dirty page tracking is enabled causes a transition of a given page of address space from clean to dirty, and in response to detecting the input / output memory system transaction associated with the given translation context for which dirty page tracking is enabled causing a transition of the given page from clean to dirty, issue a write memory access request requesting that a location within a selected dirty page buffer structure, identified using the base address indicated in a selected dirty page tracking base address register, is updated to identify a page address of the given page.

The invention discloses a data storage management method based on a FLASH in an MCU (Microprogrammed Control Unit). A storage space is divided into a node information area and a node data area, and empty / non-full / full / dirty four-state identification bits are configured for each page; new data are sequentially written into idle positions, original data are marked as a dirty state during data updating, and updated complete data are written into new positions of the storage area; and when the space is insufficient, triggering erasing balance, migrating valid data of dirty pages to empty pages, and then erasing the dirty pages to release the space. A single-identification-bit updating state and a whole-page erasing mechanism are adopted, the FLASH erasing frequency is remarkably reduced, the service life is prolonged, and meanwhile variable-length data storage and system starting consistency guarantee are supported.

The invention relates to the technical field of virtual machine migration, and discloses a virtual machine real-time migration method based on credential cloud, which comprises the following steps of: normally operating a source host to be migrated, preparing resources of a target host, dividing priorities of applications of the source host, and distinguishing high-priority applications from low-priority applications; copying all memories in the source host to a destination host by using pre-copying; iteratively copying all dirty pages generated by the high-priority applications to a destination host; when a dirty page with a high priority in the virtual machine reaches a shutdown condition, pausing the operation of the source host; after all the remaining dirty pages with high priority in the source virtual machine are completely copied to the target host, the target host is started, and the application with high priority of the target host can run normally; copying dirty pages of the low-priority applications to a destination host; according to the method, the priorities of the applications are graded, a certain strategy is adopted, the overall migration time is shortened, and the downtime of the virtual machine is further shortened.

The invention discloses a method for supporting simultaneous live migration of multiple virtual machines without packet loss by combining software and hardware in the field of virtual machine migration. The method comprises the following steps: creating virtual machine live migration auxiliary equipment D1 and a hardware dirty page reporting queue; creating a dirty page processing thread to record dirty page information; according to the live migration request, the vDPA driver registers to-be-migrated virtual machine information to the auxiliary device D1, and the auxiliary device D1 starts dirty page capturing and reporting; and after migration is finished, the vDPA driver cancels migrated virtual machine information and releases resources. According to the method, dirty page capturing and reporting are achieved through hardware, and performance loss of a pure software scheme is avoided; hardware only needs to configure a report queue, so that the hardware design complexity is reduced; independent management of dirty pages of multiple virtual machines is realized through a mapping table, and simultaneous thermal migration of the multiple virtual machines is supported; meanwhile, an exception handling mechanism is introduced, and reliability is improved. According to the method, the thermal migration performance, the hardware complexity and the resource consumption are considered, and the method is suitable for various vDPA virtual machine migration scenes.

The application discloses a WebAssembly application hot migration runtime model method for a serverless scene, relates to the technical field of cloud computing and edge computing, and designs a WASM runtime mechanism supporting seamless hot migration across architectures, including designing a unified compilation model to realize unified memory space layout of compilation products of different architectures; and proposing a software-implemented, lightweight dirty page tracking memory migration strategy, which can avoid the burden of process-level migration and maintain the lightweight characteristics of WASM.

A method and device for protecting a memory in response to use-after-free vulnerability, according to one embodiment, may comprise the steps of: releasing a physical page on the basis of a request for deallocation of heap memory transmitted from a user application and maintaining a virtual page in page metadata such that deallocation of the virtual page is delayed; performing simultaneous marking by simultaneously creating a free map in which information on one or more virtual pages maintained in the page metadata is recorded through a marker, and a mark map in which pointers found by scanning the memory are indicated; performing synchronous marking by modifying the mark map generated by the simultaneous marking by rescanning the memory on the basis of a dirty page generated during the simultaneous marking when the user application does not operate; searching for a virtual page requiring deallocation on the basis of the free map through a sweeper; and deallocating the searched virtual page when it is confirmed whether a pointer exists in the searched virtual page on the basis of the mark map.

The invention relates to the technical field of computers, in particular to a live migration method and device of virtual equipment, electronic equipment and a storage medium. According to the embodiment of the invention, by dynamically adjusting the snapshot interval and monitoring the bit error rate, the problem of the existing KVM thermal migration scheme is effectively solved. On one hand, the dirty page rate of the memory of the virtual device is obtained in real time, the Ceph snapshot interval is shortened when the dirty page rate exceeds the threshold value, and compared with a long interval which is fixed to be larger than or equal to 500 ms, the state can be recorded in time when the service state changes rapidly, loss of service data such as pictures is reduced, and user experience is guaranteed. On the other hand, software simulation link monitoring is abandoned, the bit error rate is monitored in a physical layer hardware register reading mode, misinformation generated by network fluctuation is reduced, a large amount of unnecessary migration is avoided, system resource overhead is reduced, service interruption caused by migrating is prevented, system stability and reliability are improved, and service continuity in cloud service is ensured.

The invention provides a virtual machine live migration method, electronic equipment, a storage medium and a program product, and the method comprises the steps: in response to a live migration request for a virtual machine, determining a minimum dirty page generation interval corresponding to the transmission speed of live migration; in each round of synchronous operation implemented according to the live migration request, a write memory management strategy is adopted to manage the write memory operation of the round, so that the dirty page generation interval corresponding to the write memory operation of the first part is allowed to be smaller than the minimum dirty page generation interval; the dirty page generation interval corresponding to the memory writing operation of the second part is limited to be not smaller than the minimum dirty page generation interval; and gradually strengthening the limitation of the write memory management strategy on dirty pages so as to gradually reduce the dirty pages generated by each round at the interval smaller than the minimum dirty page generation interval.

Embodiments of the present application provide a solid state disk based accelerated power-on method, a solid state disk, a computer device and a computer readable storage medium, the method comprising: when detecting that the power supply voltage of the solid state disk is abnormal, brushing the current first mapping table to the storage area of the solid state disk; when detecting that the power supply voltage returns to normal, reading the first mapping table in the storage area of the solid state disk, and restoring a temporary mapping table based on the first mapping table; and restoring the second mapping table when the voltage is abnormal according to the dirty page bitmap of the second mapping table before the voltage is abnormal and the temporary mapping table. In the case of detecting that the voltage of the external power supply returns to normal, the second mapping table can be directly restored by reading the first mapping table in the storage area of the SSD, avoiding the problem of long recovery time and long SSD power-on time caused by scanning user data to restore the second mapping table, and shortening the power-on time of the SSD.

The present application provides a dirty page sending method, an external device, and a computing device. In an embodiment, it is applied to an external device in a first computing device. The first computing device includes an address translation page table, and the address translation page table indicates the page table entries corresponding to the memory pages of the virtual machine. The page table entries indicate the physical addresses of the memory pages in the first computing device; the address translation page includes a first page table entry corresponding to the first memory page; the method includes: receiving first information, where the first information indicates the data to be modified corresponding to the virtual machine; based on the first information, determining to process the data to be modified in the first memory page; determining whether to register the first memory page as a dirty page in the first page table entry; if not, registering the first memory page as a dirty page in the first page table entry; and sending the information of the first memory page to the first computing device. There is no need for the CPU to poll the register, which increases the possibility of making full use of CPU resources.

The present application belongs to the technical field of computer information security, and particularly relates to a cloud host security baseline checking method and system based on virtual machine introspection, which comprises the following steps: accessing a target virtual machine through a host virtualization management program layer and dividing physical memory pages, obtaining physical memory dirty page states and system I / O load; according to the dirty page states of the physical memory pages in a historical observation window and a physical neighborhood set, obtaining an activity accumulation index and a spatial neighborhood linkage index at the current time, respectively; obtaining a checking trigger score according to the activity accumulation index, the spatial neighborhood linkage index and the system I / O load, and realizing security baseline checking based on the threshold relationship of the checking trigger score. The present application can identify normal business hotspots and abnormal tampering behaviors, and reduce the checking frequency when the system is under high load, thereby realizing low-overhead agentless security baseline checking.

The invention relates to the technical field of cloud computing containers, and provides a container thermal migration method based on memory snapshot difference compression, which comprises the following steps: in a normal operation stage of a container, initiating pre-migration preparation through a CRIU framework, continuously capturing dirty pages marked by a copy-on-write mechanism in a fixed period to form a difference page set, and storing the difference page set in a database; calculating the access popularity of each dirty page; dividing dirty pages of the difference page set into hot pages, warm pages and cold pages according to access popularity, and compressing the hot pages, the warm pages and the cold pages by adopting a self-adaptive compression strategy; constructing a three-level priority sending queue at a source node, and entering a network stack according to a priority sequence of a hot page, a warm page and a cold page; and when a convergence condition is met, suspending the container, only transmitting the last group of hot pages and necessary warm pages, and decompressing and rearranging memory pages at a target node according to a serial number to complete container recovery. According to the method, online container migration with high compression ratio, low CPU overhead, short downtime and high migration success rate is realized.

The embodiment of the invention discloses a data migration method and device, a to-be-migrated virtual machine, a medium and a product, and the method comprises the steps: determining the data block granularity of a disk bitmap of the to-be-migrated virtual machine based on input / output data generated in the operation process of the to-be-migrated virtual machine; after each round of disk dirty page transmission is completed, determining target transmission time of next round of memory dirty page transmission based on a first disk dirty page rate obtained by tracking a disk bitmap determined by the data block granularity; and scheduling the next round of memory dirty page transmission according to the target transmission time.

The present disclosure provides a dirty page migration method, a dirty page migration device, an electronic device, a medium and a product, and belongs to the technical field of virtual machines. The method comprises: obtaining task execution data of a source virtual machine, the task execution data being used to represent information of a target task currently executed by the source virtual machine; performing storage access prediction on the task execution data to obtain storage access prediction data of the source virtual machine; determining storage page transmission priorities according to the storage access prediction data, the storage page transmission priorities being used to represent transmission priorities of each storage page in the source virtual machine; and migrating a plurality of storage pages of the source virtual machine to a target virtual machine based on the storage page transmission priorities. The embodiments of the present disclosure can reduce repeated transmission of dirty data, reduce network overhead, and improve migration efficiency.

This application provides example methods for migrating a virtual machine and example apparatuses. One example physical host includes a source-end virtual machine and a physical function (PF) driver, where the source-end virtual machine is configured to send a first instruction to a source-end pass-through device via the PF driver, where the first instruction instructs the source-end pass-through device to record a storage address of dirty page data in the source-end pass-through device, and the dirty page data is data written by the source-end pass-through device to a memory corresponding to the source-end virtual machine. The source-end virtual machine is configured to obtain the storage address that is of the dirty page data and that is recorded by the source-end pass-through device, and send to-be-migrated data to a destination-end virtual machine.