521 results about "Data page" patented technology

In at least one embodiment, processing can include: receiving write operations; persistently recording, in a write cache or log, the write operations using page descriptors (PDESCs) of a PDESC pool and page buffers (PBs) of a PB pool; selecting, in accordance with criteria, write data pages stored in the PB pool for demotion to a physical large block (PLB) pool included in backend non-volatile storage, wherein each write data page selected denotes content written by a corresponding one of the write operations; responsive to the selecting, persistently storing the write data pages of the PB pool in the PLB pool; and updating PDESCs associated with the write data pages to reference corresponding storage locations in the PLB pool rather than in the PB pool.

The present application relates to a management method and management apparatus for a hybrid memory system, and a device and a medium. The method comprises: acquiring a write popularity value and an access popularity value of a currently accessed memory page; comparing the write popularity value with a specified write popularity threshold value, and generating write popularity page migration flag information; comparing the access popularity value with a specified access popularity threshold value, and generating access popularity page migration flag information; grouping hybrid memories in a hybrid memory system, and generating, on the basis of memory page information of all grouped memory banks, memory page management information corresponding to the hybrid memories; and on the basis of the write popularity page migration flag information, the access popularity page migration flag information, and current memory page management information corresponding to the currently accessed memory page, determining whether the current access memory page is a popular data page, and performing data migration according to a preset unpopular and popular data page migration strategy. The technical solution can improve the overall performance of a hybrid memory system having a horizontal architecture.

The invention discloses a data processing method, a storage medium, electronic equipment and a program product, and relates to the technical field of computers.The method comprises the steps that firstly, data access information is extracted from a solid state disk, and a currently accessed data block is determined; obtaining an access historical record table corresponding to the currently accessed data block, wherein the access historical record table comprises a data block access historical record table and a data page access historical record table; determining target pre-read data based on the weights of the plurality of data blocks and the weights of the plurality of data pages in the access historical record table; and if it is detected that the position of the target pre-read data is not in the pre-read position linked list, loading the target pre-read data into a cache of the solid state disk. Compared with the prior art, the method has the advantages that by maintaining the access history record table and recording the access frequency and weight of each data block and data page, the next data block and data page which are most likely to be accessed can be predicted more accurately, and then the pre-reading hit rate is increased.

The invention provides a data rereading method, device and equipment, which are applied to the technical field of data storage, and the method comprises the following steps: when data reading is performed on a flash memory, if an error rate exceeds an error correction range value, acquiring a first offset voltage parameter group from a preset rereading table, and rereading a target data page by using the first offset voltage parameter group. And if the target data page fails, performing voltage threshold distribution scanning processing on the target data page to obtain a second offset parameter set of the target data page. And comparing the second offset parameter set with the first offset parameter set to determine the voltage offset direction. And according to the voltage offset direction, searching a target reread offset voltage item in the selectable offset voltage items, determining a second offset voltage parameter group, reading the target data page again by using the reread offset voltage item, and performing reread meter parameter updating, the problems that in the current data rereading process, the flash memory performance is reduced, the historical experience dependency degree is high, and the process of calculating the optimal voltage threshold value is limited are solved.

The embodiment of the invention discloses a data writing method and device, computer equipment and a storage medium, and belongs to the technical field of databases. In the embodiment of the invention, when any thread in the computer equipment does not obtain the available memory page in the plurality of idle memory pages corresponding to the target data table, N idle data pages can be expanded for the thread, the idle memory pages are expanded based on the N idle data pages, and then data are written in the expanded idle memory pages. Therefore, any thread which cannot obtain the available memory page can automatically trigger the data table expansion process to timely expand N data pages for the thread for use. Moreover, data pages do not need to be expanded for other threads, so that the time required for expanding the data pages for each thread is short, the thread blocking time can be shortened, and the data writing performance is improved.

The invention discloses a data recovery method and a related device, the method can be applied to a server running a database system, and the method comprises the steps that in response to modification of a first data page, a first log is generated, modification of the first data page is recorded in the first log, and a first identifier indicates the version of the first data page; under the condition that the database system fails, the version of a second data page is determined, the second data page is the data page of the data to be recovered, and the page identifier of the second data page is the same as the page identifier of the first data page; and under the condition that the version of the first log is greater than that of the second data page and the version indicated by the first identifier is the same as that of the second data page, performing data recovery on the second data page by using the first log. Through the first identifier, data recovery on the second data page by using a log with a wrong version is avoided, and the accuracy of data recovery is ensured.

The invention belongs to the technical field of storage media, and particularly relates to a storage medium abrasion balancing system based on cold and hot data perception, which comprises a cold and hot data perception module used for monitoring the access frequency and the update time of data in real time so as to judge the cold and hot degree of the data, and establishing a heat table, recording a heat value of each data block or data page; and the data feature prediction module is used for predicting features of a future heat change trend and a data volume increase trend of the data by utilizing a machine learning algorithm according to historical data access and update records. According to the method, the hot and cold data sensing module is used for monitoring data access and updating conditions in real time to calculate a hot value, and the storage medium management module is combined to preferentially store hot data into blocks with less erasing times and store cold data into blocks with more erasing times. Therefore, the erasing times of the blocks of the storage medium tend to be balanced, and excessive abrasion of individual blocks is avoided.

The invention discloses a row-level data recovery method and device for a damaged file of a GoldenDB database, and the method comprises the following steps: S1, scanning a to-be-recovered GoldenDB data file, and recognizing and caching all data pages; s2, on the basis of metadata of the data pages, cluster index leaf node pages containing user data are screened out from the cached data pages; s3, for each screened leaf node page, analyzing an internal record organization structure of the leaf node page, and performing row-level data recovery; the row-level data recovery comprises the steps of traversing user records according to a virtual record pointer, and performing cross check and recovery on a traversal path in combination with a page directory slot so as to extract and mark the user records one by one; and S4, outputting all the recovered row-level data.

The embodiment of the invention provides a data migration method, an access processing method, a data migration system and electronic equipment. The data migration method is applied to a switch, the switch communicates with a plurality of memory devices in a memory pool based on an interconnection protocol, and each memory device is provided with a plurality of memory areas. When data migration needs to be carried out on a first memory area in a memory pool, a switch sequentially migrates a plurality of data pages obtained by dividing data in the first memory area into available second memory areas according to pages, and in the migration process, a page bitmap determined for the first memory region is utilized to record migration states of the plurality of data pages. By adopting the scheme, the problem of large-area fault of the server caused by the memory problem can be effectively solved, the page-level granularity is adopted to migrate the data in the first memory region page by page in the migration process, the access of the server to the first memory region is continuously processed in the migration process, and the influence on the performance of the server is reduced.

The invention provides a data processing method and device, equipment and a storage medium, and relates to the field of data processing.The method comprises the steps that a processor determines a data processing operation for a target data page in a user area of a target flash memory; and when the block mapping table does not exist in the memory or the mapping relation corresponding to the target data block where the target data page is located does not exist in the block mapping table, if the data processing operation is a data reading operation, reading the target data from the target data page, when the bit flipping number determined based on the error correction code corresponding to the target data is not smaller than the bit flipping threshold value, a reserved data block meeting a preset condition is determined from a reserved area of the target flash memory, and data in the target data block is backed up to the reserved data block; the block mapping table is used for storing the mapping relation between the target data block and the reserved data block, then error correction is conducted on the target data based on the size relation between the bit flip number and the error-correctable bit number, and therefore the stability of the data in the target flash memory is improved through data backup.

Examples described herein provide a computer-implemented method for storing data pages of a database in a remote storage. The method includes receiving a write request associated with one or more data pages of the database. The method further includes storing, in a first storage layer having a write ahead log, the write request associated with the one or more data pages. The method further includes storing, in a first write buffer, the received write request associated with the one or more data pages that each include a legacy data page identifier. The method further includes storing, in a second storage layer having a second write buffer, a sorted string table (SST) of the received write request. The method further includes storing, in a top level of a log structured merge (LSM) tree disposed in the remote storage, the SST from the second write buffer.

The invention provides a data synchronization method and device, equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical field of databases, distributed storage technologies and data synchronization. According to the specific implementation scheme, the method comprises the steps that according to the sequence number sequence of a to-be-applied first physical log, log stream continuity verification is conducted on each log record contained in the first physical log in sequence; under the condition that the log stream continuity verification is passed, performing modification chain integrity verification according to a first identifier of the log record and a current version identifier of a target data page to be modified of the log record; wherein the first identifier is used for indicating a second physical log which finally modifies the target data page; and under the condition that the integrity verification of the modification chain is passed, modifying a target data page according to the log record.

The invention relates to the technical field of large object data storage, and provides a method, system and equipment for storing large object data in a database, and the method comprises the steps: segmenting the large object data, and setting serial numbers for a plurality of data blocks obtained through segmentation; the data blocks with the serial numbers are stored in different data pages in a scattered mode; constructing a B tree index for each piece of large object data according to the serial number and the storage information; and performing incremental backup on the large object data, and performing positioning recovery on the large object data according to the constructed B-tree index. According to the method, the system and the equipment for storing the large object data in the database, the storage efficiency can be improved, the data access speed can be increased, the system stability can be enhanced, and the development, operation and maintenance efficiency can be improved.

The invention provides a virtual machine live migration method and computing equipment. The method is applied to a host machine, and a virtual machine, a first execution body and a second execution body run in a processor of the host machine. Write permission bits of a plurality of data pages of the virtual machine are stored in the host machine. The processor is provided with a hardware setting function for a write permission bit, and the write permission bit of a first data page in the plurality of data pages is a first value indicating that the first data page is not writable. The method comprises the following steps: when a virtual machine performs write operation on a first data page, a processor modifies a write permission bit of the first data page to represent a writable second value of the first data page; the second executor updates a dirty page bitmap of the virtual machine according to the write permission bit of the first data page; the first executor determines a dirty page of the virtual machine according to the dirty page bitmap, notifies the second executor to modify a write permission bit of the dirty page of the virtual machine to a first value, and migrates the dirty page of the virtual machine, and the dirty page of the virtual machine comprises the first data page.

The invention relates to the technical field of electric digital data processing, in particular to a WAL log data query method and device, a time sequence database and a medium, and the method comprises the steps that a log information query instruction is received to obtain a query target; in response to the log information query instruction, searching metadata of a corresponding data table in global metadata of a time sequence database, and determining write-in position information of a query target based on the metadata; based on the write-in position information and a pre-constructed WAL log coding structure, the query target is positioned, and the WAL log coding structure comprises a plurality of data blocks, a plurality of data pages and a data segment. Therefore, the technical problems that in the related technology, unnecessary data page locking is carried out during data query, the cache utilization rate is low, a large number of useless caches are likely to be generated, and then the database query performance is affected are solved.

A method and associated memory system for randomizing memory storage data. The method and system receive at a data inverter user data and meta data sequence having an inversion seed bit, determine a value for the inversion seed bit from a look up table specifying inversion seeds for different pages of data to be stored in a memory, depending on the value of the inversion seed bit, bit-flip the user data and meta data sequence except for the inversion seed bit; and regardless of bit-flipping, exclusive OR (XOR) the user data and meta data sequence with a random sequence to produce an XORed sequence for storage in the memory as randomized data.

Embodiments of the present disclosure include techniques for backing up data. In one embodiment, a plurality of read requests are issued. In response to the read requests, a plurality of data pages are retrieved. The plurality of data pages are stored in a plurality of buffers. During said storing, for each data page, an indication that storage of a particular data page of the plurality of data pages has been completed is generated. In response to an indication that storage of a particular data page has been completed, the data page is processed with one of a plurality of jobs, where a plurality of data pages are processed by the plurality of jobs in parallel.

The invention provides a data processing method and related equipment, and relates to the field of database application, the method comprises the following steps: when a write request of target data is received, a storage space is applied for the target data, the storage space comprises at least one range, and the range comprises a data page and a verification page; the verification information of the first range is written into a verification page corresponding to the first range, the verification information of the first range is the verification information obtained through calculation when the target data is written into a data page corresponding to the first range, the verification information is used for conducting redundancy verification, and the first range is any range in the at least one range; and writing the first range into persistent storage. Whether the data is damaged or not can be detected by using the verification information, so that the reliability of data storage is improved, and the risk of data damage caused by a silent error and the like is reduced; and meanwhile, the problem of write amplification is reduced, the storage space occupied by configuration of additional copies is reduced, and the storage overhead and the data reliability can be effectively balanced.

The embodiment of the invention provides a data reading method and device, electronic equipment and a storage medium, and belongs to the technical field of databases. The method comprises the following steps: determining a first page number occupied by a corresponding table-level composite index based on the area size of a candidate area in an initial data table, a preset index field number and a preset index type; dividing pages contained in the candidate area based on the first page number to determine index pages and data pages of the first page number; constructing a table-level composite index based on a region-level preset bloom filter and a preset double-layer sparse index, and storing the table-level composite index to an index page; storing field data of all candidate pages in the candidate area to a data page, and constructing a target area structure corresponding to the candidate area based on the stored index page and the data page; and constructing a target data table based on the multiple target area structures of the same initial data table, and performing data reading based on the target data table. According to the embodiment of the invention, the data reading efficiency between the database nodes can be improved.

The invention relates to an openGauss database mirror image backup recovery method and an openGauss database mirror image backup recovery device. The method comprises the following steps: executing full backup, and backing up a dependency item of database service, a data file and a WAL file; incremental backup is executed, and only the data pages and the WAL files which are changed since the last backup are backed up; and executing recovery operation: executing full-amount recovery and specified time point recovery, and supporting single-node recovery and cluster recovery. According to the openGauss database mirror image backup and recovery method based on the CDM technology flow, the backup and recovery efficiency is remarkably improved through the functions of original format backup, incremental backup, specified time point recovery and the like; the method is high in flexibility, supports full backup, incremental backup and specified time point recovery, and can meet backup requirements in different scenes; according to the method, the backup data is kept in a native format, additional processing is not needed during recovery, and data integrity and availability are ensured.

The invention provides a data caching method and computing equipment. The computing device comprises a hybrid memory, and the hybrid memory comprises a volatile memory and a persistent memory. The method comprises: reading first data from a persistent memory, the first data corresponding to a first mode in a plurality of modes; under the condition that it is determined that the first data is cached, the first data is written into a data page corresponding to the first mode in the volatile memory, the volatile memory comprises a plurality of data pages corresponding to the modes, and the data pages corresponding to the modes are organized according to the first data structure. In the method, under the condition of determining the cache data, the data is written into the data page corresponding to the mode corresponding to the data in the volatile memory, so that the space management efficiency of the volatile memory can be improved, and the efficiency of acquiring the cached data from the volatile memory is saved.

The invention provides a UI automatic test system and method, computer equipment and a storage medium, a manual case structuring module receives a manual case and converts the manual case into structured data in a preset format; the page element obtaining module calls a browser driving engine according to the target URL to obtain full page elements; the AI element matching module is used for scoring full elements of a page through a pre-matching algorithm based on operation types and interface element descriptions in the structured data, screening out first K elements, and carrying out secondary screening through a large model to determine target elements; and the automatic use case execution module generates an automatic test use case according to the target element and the operation data in the structured data, executes the automatic test use case and verifies whether an execution result conforms to expectation or not to realize UI automatic testing. By the adoption of the scheme, the labor cost of UI testing is reduced, the efficiency and adaptability of UI testing are improved, and then the automation level and the large-scale capacity of UI testing are improved.

Embodiments of the present technology provide non-volatile memory devices comprising memory dies that natively generate “exclusive OR (XOR) data pages” that can be used to recover data pages corrupted by UECC errors. Through memory die native-XOR data page generation, embodiments can recover data pages corrupted by UECC errors more efficiently, more rapidly, and with fewer resources than potential alternative technologies.

The invention provides a data processing method and device. The method comprises the steps of obtaining a target instruction; the target instruction is used for reading a target data page in a target flash memory; in response to the target instruction, reading the target data page according to a first threshold voltage; if the reading fails, obtaining a first offset voltage from the first data table; the first data table comprises a plurality of offset voltages with a variable traversal sequence, the first offset voltage is a first traversal voltage, and the traversal sequence is related to a target parameter of the electronic equipment; the target parameter represents a use environment, use duration and / or an operation mode of the electronic equipment; and re-reading the target data page based on the first offset voltage and the first threshold voltage.

The invention provides a hardware-assisted isolation execution method for an eBPF program, which comprises the following steps: acquiring a to-be-executed eBPF program, setting access data pages in the eBPF program as non-privileged pages, and transmitting all memory access instructions in the eBPF program as non-privileged access instructions through instant JIT compiling to obtain a to-be-executed program; running the to-be-executed program in a kernel mode EL1 of a server operating system to obtain a running result; in the process of running the to-be-executed program, a non-access instruction in the to-be-executed program runs at a kernel mode EL1 privilege level, and in the running process, an access instruction in the to-be-executed program is a non-privilege access instruction, and the non-privilege access instruction runs at a user mode EL0 non-privilege level. By setting the memory of the eBPF program as a non-privileged page, the eBPF program still runs at a privileged level, but is transmitted to use a non-privileged memory access instruction. The internal memory of the kernel is set as a privileged page, so that the eBPF program cannot access the privileged page, and the internal memory security of the kernel is ensured.

The invention relates to a data page recovery method and device, equipment, a medium and a product. The method comprises the steps that when it is detected that a current data page is a faulty data page, a corresponding relation associated with a data page identifier is searched for according to the data page identifier of the current data page; reading the address information and the log serial number of the historical version data page from the corresponding relation; and reading the historical version data page according to the address information, and recovering the current data page according to the historical version data page and the log serial number to obtain a target data page. By adopting the method, the recovery efficiency of the data page in a high-concurrency writing scene can be improved.

The embodiment of the invention provides a data stability enhancement method of a flash memory medium and related equipment thereof, and the method comprises the steps: obtaining an initial flash memory type of the flash memory medium, and determining M data pages corresponding to the initial flash memory type; obtaining a target flash memory type of the flash memory medium, and obtaining N groups of user data according to the target flash memory type; generating redundant data corresponding to the N groups of user data based on a preset operation rule; respectively writing the N groups of user data into any N data pages, and writing redundant data into the remaining data pages of the M data pages; and performing offset correction processing on the 2M initial reading voltages corresponding to the initial flash memory type to obtain 2N target reading voltages, so as to read the user data in the N data pages according to the 2N target reading voltages, thereby effectively increasing the threshold voltage interval between different charge states, and not needing to repeatedly adjust the reading voltage by using a rereading command, thereby improving the reading efficiency. Therefore, the read delay can be reduced, and the data stability is effectively improved.

A memory system and method for performing adaptive prefetch in a memory system are provided. The method includes receiving a first data request including a first address of a first data page stored in a memory device; and performing a lookup operation using an adaptive prefetch table, to determine whether a prefetch operation should be performed based on the received first data request. The adaptive prefetch table includes an entry for the first data page corresponding to the received first data request, and the entry for the first data page includes an address of a predicted next page, and a weight factor of the predicted next page.

A memory device includes a memory cell array configured to store data, a page buffer circuit configured to store data in the memory cell array or read data stored in the memory cell array, and a scan register circuit configured to receive a pass / fail result of data from the page buffer circuit and store the pass / fail result in a plurality of scan registers. The scan register circuit obtains information about the scan register where the fail result is stored through a scan operation, and uses the information about the scan register where the fail result is stored to determine the number of fail bits and / or a position index indicating the position of the fail bits according to the operation mode.

The invention provides a method and device for testing the randomization effect of a flash memory controller, equipment and a medium, and relates to the technical field of storage. The method for testing the randomization effect of the flash memory controller comprises the following steps of: randomizing different original data, writing the randomized data into different types of data pages of a flash memory device, and reading and caching the data from the different types of data pages to obtain cached data of the different types of data pages; according to a preset state data generation rule and a bit reading sequence, performing combination processing on different cache data to obtain multiple types of state data; performing statistical calculation on the quantity of the state data of each type to obtain a quantity ratio of the state data of each type; and finally, calculating and comparing the quantity ratio, and judging the randomization effect of the flash memory controller. According to the method, reliable reference can be provided for evaluating the randomization effect of the flash memory controller by directly counting the quantity ratio of each type of state data in the flash memory device.