423 results about "Data page" patented technology

The present application relates to a management method and management apparatus for a hybrid memory system, and a device and a medium. The method comprises: acquiring a write popularity value and an access popularity value of a currently accessed memory page; comparing the write popularity value with a specified write popularity threshold value, and generating write popularity page migration flag information; comparing the access popularity value with a specified access popularity threshold value, and generating access popularity page migration flag information; grouping hybrid memories in a hybrid memory system, and generating, on the basis of memory page information of all grouped memory banks, memory page management information corresponding to the hybrid memories; and on the basis of the write popularity page migration flag information, the access popularity page migration flag information, and current memory page management information corresponding to the currently accessed memory page, determining whether the current access memory page is a popular data page, and performing data migration according to a preset unpopular and popular data page migration strategy. The technical solution can improve the overall performance of a hybrid memory system having a horizontal architecture.

The invention discloses a data processing method, a storage medium, electronic equipment and a program product, and relates to the technical field of computers.The method comprises the steps that firstly, data access information is extracted from a solid state disk, and a currently accessed data block is determined; obtaining an access historical record table corresponding to the currently accessed data block, wherein the access historical record table comprises a data block access historical record table and a data page access historical record table; determining target pre-read data based on the weights of the plurality of data blocks and the weights of the plurality of data pages in the access historical record table; and if it is detected that the position of the target pre-read data is not in the pre-read position linked list, loading the target pre-read data into a cache of the solid state disk. Compared with the prior art, the method has the advantages that by maintaining the access history record table and recording the access frequency and weight of each data block and data page, the next data block and data page which are most likely to be accessed can be predicted more accurately, and then the pre-reading hit rate is increased.

The invention provides a data rereading method, device and equipment, which are applied to the technical field of data storage, and the method comprises the following steps: when data reading is performed on a flash memory, if an error rate exceeds an error correction range value, acquiring a first offset voltage parameter group from a preset rereading table, and rereading a target data page by using the first offset voltage parameter group. And if the target data page fails, performing voltage threshold distribution scanning processing on the target data page to obtain a second offset parameter set of the target data page. And comparing the second offset parameter set with the first offset parameter set to determine the voltage offset direction. And according to the voltage offset direction, searching a target reread offset voltage item in the selectable offset voltage items, determining a second offset voltage parameter group, reading the target data page again by using the reread offset voltage item, and performing reread meter parameter updating, the problems that in the current data rereading process, the flash memory performance is reduced, the historical experience dependency degree is high, and the process of calculating the optimal voltage threshold value is limited are solved.

The invention discloses a data recovery method and a related device, the method can be applied to a server running a database system, and the method comprises the steps that in response to modification of a first data page, a first log is generated, modification of the first data page is recorded in the first log, and a first identifier indicates the version of the first data page; under the condition that the database system fails, the version of a second data page is determined, the second data page is the data page of the data to be recovered, and the page identifier of the second data page is the same as the page identifier of the first data page; and under the condition that the version of the first log is greater than that of the second data page and the version indicated by the first identifier is the same as that of the second data page, performing data recovery on the second data page by using the first log. Through the first identifier, data recovery on the second data page by using a log with a wrong version is avoided, and the accuracy of data recovery is ensured.

The invention belongs to the technical field of storage media, and particularly relates to a storage medium abrasion balancing system based on cold and hot data perception, which comprises a cold and hot data perception module used for monitoring the access frequency and the update time of data in real time so as to judge the cold and hot degree of the data, and establishing a heat table, recording a heat value of each data block or data page; and the data feature prediction module is used for predicting features of a future heat change trend and a data volume increase trend of the data by utilizing a machine learning algorithm according to historical data access and update records. According to the method, the hot and cold data sensing module is used for monitoring data access and updating conditions in real time to calculate a hot value, and the storage medium management module is combined to preferentially store hot data into blocks with less erasing times and store cold data into blocks with more erasing times. Therefore, the erasing times of the blocks of the storage medium tend to be balanced, and excessive abrasion of individual blocks is avoided.

The invention discloses a row-level data recovery method and device for a damaged file of a GoldenDB database, and the method comprises the following steps: S1, scanning a to-be-recovered GoldenDB data file, and recognizing and caching all data pages; s2, on the basis of metadata of the data pages, cluster index leaf node pages containing user data are screened out from the cached data pages; s3, for each screened leaf node page, analyzing an internal record organization structure of the leaf node page, and performing row-level data recovery; the row-level data recovery comprises the steps of traversing user records according to a virtual record pointer, and performing cross check and recovery on a traversal path in combination with a page directory slot so as to extract and mark the user records one by one; and S4, outputting all the recovered row-level data.

The embodiment of the invention provides a data migration method, an access processing method, a data migration system and electronic equipment. The data migration method is applied to a switch, the switch communicates with a plurality of memory devices in a memory pool based on an interconnection protocol, and each memory device is provided with a plurality of memory areas. When data migration needs to be carried out on a first memory area in a memory pool, a switch sequentially migrates a plurality of data pages obtained by dividing data in the first memory area into available second memory areas according to pages, and in the migration process, a page bitmap determined for the first memory region is utilized to record migration states of the plurality of data pages. By adopting the scheme, the problem of large-area fault of the server caused by the memory problem can be effectively solved, the page-level granularity is adopted to migrate the data in the first memory region page by page in the migration process, the access of the server to the first memory region is continuously processed in the migration process, and the influence on the performance of the server is reduced.

Examples described herein provide a computer-implemented method for storing data pages of a database in a remote storage. The method includes receiving a write request associated with one or more data pages of the database. The method further includes storing, in a first storage layer having a write ahead log, the write request associated with the one or more data pages. The method further includes storing, in a first write buffer, the received write request associated with the one or more data pages that each include a legacy data page identifier. The method further includes storing, in a second storage layer having a second write buffer, a sorted string table (SST) of the received write request. The method further includes storing, in a top level of a log structured merge (LSM) tree disposed in the remote storage, the SST from the second write buffer.

The invention provides a data synchronization method and device, equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical field of databases, distributed storage technologies and data synchronization. According to the specific implementation scheme, the method comprises the steps that according to the sequence number sequence of a to-be-applied first physical log, log stream continuity verification is conducted on each log record contained in the first physical log in sequence; under the condition that the log stream continuity verification is passed, performing modification chain integrity verification according to a first identifier of the log record and a current version identifier of a target data page to be modified of the log record; wherein the first identifier is used for indicating a second physical log which finally modifies the target data page; and under the condition that the integrity verification of the modification chain is passed, modifying a target data page according to the log record.

The invention relates to the technical field of large object data storage, and provides a method, system and equipment for storing large object data in a database, and the method comprises the steps: segmenting the large object data, and setting serial numbers for a plurality of data blocks obtained through segmentation; the data blocks with the serial numbers are stored in different data pages in a scattered mode; constructing a B tree index for each piece of large object data according to the serial number and the storage information; and performing incremental backup on the large object data, and performing positioning recovery on the large object data according to the constructed B-tree index. According to the method, the system and the equipment for storing the large object data in the database, the storage efficiency can be improved, the data access speed can be increased, the system stability can be enhanced, and the development, operation and maintenance efficiency can be improved.

The invention provides a virtual machine live migration method and computing equipment. The method is applied to a host machine, and a virtual machine, a first execution body and a second execution body run in a processor of the host machine. Write permission bits of a plurality of data pages of the virtual machine are stored in the host machine. The processor is provided with a hardware setting function for a write permission bit, and the write permission bit of a first data page in the plurality of data pages is a first value indicating that the first data page is not writable. The method comprises the following steps: when a virtual machine performs write operation on a first data page, a processor modifies a write permission bit of the first data page to represent a writable second value of the first data page; the second executor updates a dirty page bitmap of the virtual machine according to the write permission bit of the first data page; the first executor determines a dirty page of the virtual machine according to the dirty page bitmap, notifies the second executor to modify a write permission bit of the dirty page of the virtual machine to a first value, and migrates the dirty page of the virtual machine, and the dirty page of the virtual machine comprises the first data page.

The invention relates to the technical field of electric digital data processing, in particular to a WAL log data query method and device, a time sequence database and a medium, and the method comprises the steps that a log information query instruction is received to obtain a query target; in response to the log information query instruction, searching metadata of a corresponding data table in global metadata of a time sequence database, and determining write-in position information of a query target based on the metadata; based on the write-in position information and a pre-constructed WAL log coding structure, the query target is positioned, and the WAL log coding structure comprises a plurality of data blocks, a plurality of data pages and a data segment. Therefore, the technical problems that in the related technology, unnecessary data page locking is carried out during data query, the cache utilization rate is low, a large number of useless caches are likely to be generated, and then the database query performance is affected are solved.

A method and associated memory system for randomizing memory storage data. The method and system receive at a data inverter user data and meta data sequence having an inversion seed bit, determine a value for the inversion seed bit from a look up table specifying inversion seeds for different pages of data to be stored in a memory, depending on the value of the inversion seed bit, bit-flip the user data and meta data sequence except for the inversion seed bit; and regardless of bit-flipping, exclusive OR (XOR) the user data and meta data sequence with a random sequence to produce an XORed sequence for storage in the memory as randomized data.

Embodiments of the present disclosure include techniques for backing up data. In one embodiment, a plurality of read requests are issued. In response to the read requests, a plurality of data pages are retrieved. The plurality of data pages are stored in a plurality of buffers. During said storing, for each data page, an indication that storage of a particular data page of the plurality of data pages has been completed is generated. In response to an indication that storage of a particular data page has been completed, the data page is processed with one of a plurality of jobs, where a plurality of data pages are processed by the plurality of jobs in parallel.

The invention provides a data processing method and related equipment, and relates to the field of database application, the method comprises the following steps: when a write request of target data is received, a storage space is applied for the target data, the storage space comprises at least one range, and the range comprises a data page and a verification page; the verification information of the first range is written into a verification page corresponding to the first range, the verification information of the first range is the verification information obtained through calculation when the target data is written into a data page corresponding to the first range, the verification information is used for conducting redundancy verification, and the first range is any range in the at least one range; and writing the first range into persistent storage. Whether the data is damaged or not can be detected by using the verification information, so that the reliability of data storage is improved, and the risk of data damage caused by a silent error and the like is reduced; and meanwhile, the problem of write amplification is reduced, the storage space occupied by configuration of additional copies is reduced, and the storage overhead and the data reliability can be effectively balanced.

The invention relates to an openGauss database mirror image backup recovery method and an openGauss database mirror image backup recovery device. The method comprises the following steps: executing full backup, and backing up a dependency item of database service, a data file and a WAL file; incremental backup is executed, and only the data pages and the WAL files which are changed since the last backup are backed up; and executing recovery operation: executing full-amount recovery and specified time point recovery, and supporting single-node recovery and cluster recovery. According to the openGauss database mirror image backup and recovery method based on the CDM technology flow, the backup and recovery efficiency is remarkably improved through the functions of original format backup, incremental backup, specified time point recovery and the like; the method is high in flexibility, supports full backup, incremental backup and specified time point recovery, and can meet backup requirements in different scenes; according to the method, the backup data is kept in a native format, additional processing is not needed during recovery, and data integrity and availability are ensured.

The invention provides a UI automatic test system and method, computer equipment and a storage medium, a manual case structuring module receives a manual case and converts the manual case into structured data in a preset format; the page element obtaining module calls a browser driving engine according to the target URL to obtain full page elements; the AI element matching module is used for scoring full elements of a page through a pre-matching algorithm based on operation types and interface element descriptions in the structured data, screening out first K elements, and carrying out secondary screening through a large model to determine target elements; and the automatic use case execution module generates an automatic test use case according to the target element and the operation data in the structured data, executes the automatic test use case and verifies whether an execution result conforms to expectation or not to realize UI automatic testing. By the adoption of the scheme, the labor cost of UI testing is reduced, the efficiency and adaptability of UI testing are improved, and then the automation level and the large-scale capacity of UI testing are improved.

The invention relates to a data page recovery method and device, equipment, a medium and a product. The method comprises the steps that when it is detected that a current data page is a faulty data page, a corresponding relation associated with a data page identifier is searched for according to the data page identifier of the current data page; reading the address information and the log serial number of the historical version data page from the corresponding relation; and reading the historical version data page according to the address information, and recovering the current data page according to the historical version data page and the log serial number to obtain a target data page. By adopting the method, the recovery efficiency of the data page in a high-concurrency writing scene can be improved.

A memory system and method for performing adaptive prefetch in a memory system are provided. The method includes receiving a first data request including a first address of a first data page stored in a memory device; and performing a lookup operation using an adaptive prefetch table, to determine whether a prefetch operation should be performed based on the received first data request. The adaptive prefetch table includes an entry for the first data page corresponding to the received first data request, and the entry for the first data page includes an address of a predicted next page, and a weight factor of the predicted next page.

A memory device includes a memory cell array configured to store data, a page buffer circuit configured to store data in the memory cell array or read data stored in the memory cell array, and a scan register circuit configured to receive a pass / fail result of data from the page buffer circuit and store the pass / fail result in a plurality of scan registers. The scan register circuit obtains information about the scan register where the fail result is stored through a scan operation, and uses the information about the scan register where the fail result is stored to determine the number of fail bits and / or a position index indicating the position of the fail bits according to the operation mode.

The invention provides a method and device for testing the randomization effect of a flash memory controller, equipment and a medium, and relates to the technical field of storage. The method for testing the randomization effect of the flash memory controller comprises the following steps of: randomizing different original data, writing the randomized data into different types of data pages of a flash memory device, and reading and caching the data from the different types of data pages to obtain cached data of the different types of data pages; according to a preset state data generation rule and a bit reading sequence, performing combination processing on different cache data to obtain multiple types of state data; performing statistical calculation on the quantity of the state data of each type to obtain a quantity ratio of the state data of each type; and finally, calculating and comparing the quantity ratio, and judging the randomization effect of the flash memory controller. According to the method, reliable reference can be provided for evaluating the randomization effect of the flash memory controller by directly counting the quantity ratio of each type of state data in the flash memory device.

The embodiment of the invention provides a core dump file generation method and device, electronic equipment and a medium, and relates to the technical field of data processing.The method comprises the steps that in response to abnormity of a target process, a thread causing the abnormity of the target process is determined as a target thread from threads of the target process; retrieving a virtual memory address from a register occupied by the target thread and a stack memory area of the target thread; determining a target address of a data page which belongs to a heap memory area and has mapping from the retrieved virtual memory address; and generating a core dump file containing the data page mapped by the target address. By applying the scheme provided by the embodiment of the invention, the validity of the core dump file can be improved while the size of the core dump file is reduced.

The present invention relates to the technical field of computers. Disclosed are a data processing method, a computer readable storage medium, and an electronic device. The method comprises: acquiring, from a first storage medium, data objects to be evicted, wherein the data objects to be evicted use a data storage format of a preset data granularity; on the basis of a preset customization policy, selecting a target data object from among the data objects to be evicted, and determining an indexing method of the target data object in a second storage medium; and caching the target data object to the second storage medium according to the indexing method in the second storage medium, wherein the preset customization policy comprises: a filtering condition, the filtering condition being used for performing data filtering on the data objects to be evicted, and the second storage medium is an extended storage area of the first storage medium. The present invention solves the technical problems in the related art that database execution efficiency is low and cache efficiency is low due to directly caching evicted data pages in a buffer pool into a buffer pool extension.

The present invention relates to a correction method, device and system for ROWID mapping changes during data synchronization. The method part mainly includes: the target-side data synchronization service adds an auxiliary column to the table synchronized from the source to the target, which is used to store the ROWID of the corresponding row data on the source; after the target-side data synchronization service receives the MOVE operation start command sent by the source, it creates a mapping container; the mapping of the new ROWID and MD5 value received from the source is inserted into the mapping container; after the target-side data synchronization service receives the source-side ROWID range of the cleaned data page sent by the source, it obtains the old data ROWID range from the auxiliary column; constructs a query statement for the corresponding table according to the two ROWID ranges; after the target-side data synchronization service executes the query statement and extracts the result set, it calculates the MD5 value of each row according to the result set and obtains the corresponding new ROWID value in the mapping container, and updates the new ROWID value to the auxiliary column. The present invention can effectively improve the repair performance of the mapping relationship between the source-side ROWID and the target-side data.

Disclosed Methods, Apparatus, and articles of manufacture to profile page tables for memory management are disclosed. An example apparatus includes a processor to execute computer readable instructions to: profile a first page at a first level of a page table as not part of a target group; and in response to profiling the first page as not part of the target group, label a data page at a second level that corresponds to the first page as not part of the target group, the second level being lower than the first level.

The invention discloses a data processing method, a computer readable storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps that a to-be-eliminated data object is obtained from a first storage medium, and the to-be-eliminated data object adopts a data storage format of preset data granularity; based on a preset customization strategy, selecting a target data object from the to-be-eliminated data objects, and determining an indexing mode of the target data object in a second storage medium; caching the target data object to a second storage medium according to an index mode in the second storage medium; wherein the preset customization strategy comprises a filtering condition, the filtering condition is used for performing data filtering on the to-be-eliminated data object, and the second storage medium is an extended storage area of the first storage medium. According to the method and the device, the technical problems of relatively low database execution efficiency and low caching efficiency caused by directly caching the eliminated data pages in the cache pool into the cache pool extension in the related technology are solved.

A data analysis method, an electronic device and a readable medium are provided The data analysis method includes: displaying a target data page, the target data page being used to display target data; displaying a first intelligent object page in response to a trigger operation on a first intelligent object on the target data page; and displaying a target dashboard page in response to target information entered on the first intelligent object page and used to create a dashboard for the target data. The target dashboard page includes at least one data chart, the at least one data chart is obtained by analyzing the target data based on analysis dimension information, and the analysis dimension information is determined based on a data dimension in the target data.

The invention relates to an exit-entry certificate authenticity identification method and system based on two-dimensional code identification, and a medium. The method comprises the following steps: constructing a cross verification rule base comprising logical relationship information between two-dimensional code information of the entry-exit certificate and signing and issuing information of the entry-exit certificate; acquiring a data page image of an entry-exit certificate to be detected; identifying a two-dimensional code in the preprocessed data page image by using a two-dimensional code identification engine; identifying signing and issuing information in the preprocessed data page image; according to the dynamic two-dimensional code verification library, the dynamic blacklist library and the identified signing and issuing information, uniqueness verification and legality verification are carried out on the identified two-dimensional code information; comparing the corresponding relationship between the identified two-dimensional code information and the identified signing and issuing information according to a cross verification rule base; and judging the authenticity of the entry-exit certificate and generating a report. According to the invention, true and false identification is carried out through three dimensions, the identification accuracy is high, and high-risk false and false situations such as same codes, different certificates and tampering fields can be accurately identified.

Fast iterative rewinding of a database to a past time point is supported. Responsive to a database rewind request, a target LSN for a target time is obtained, along with a set of persisted LSNs corresponding to a set of slices. Each slice includes a valid anchor checkpoint smaller than and closest to the target LSN. A minimum persisted LSN and change log records based thereon and on the target LSN are also obtained. The change log records are applied to anchor pages of the slices to restore the database to the target time. Also provided for is database management, including generating anchor checkpoints for recording page content corresponding to LSNs and indicating anchor pages. Also provided for is scaling back restorable time window size, including, when a storage threshold is exceeded, releasing stored data pages for deletion based on usage levels.

Techniques can include: recording, in a log, log entries for write operations writing data pages to logical addresses; and flushing the log entries from the log, wherein flushing includes performing deduplication of the data pages. Flushing can include: partitioning the data pages into i) a first portion of the data pages that are determined to be duplicates of existing data pages, and ii) a second portion of the data pages determined to be unique data pages; atomically incrementing reference counts each corresponding to one of the existing data pages determined as identical to a data page of the first portion; and atomically binding the logical addresses to the data pages, including: binding the first portion of data pages to a corresponding third portion of the logical addresses; and binding the second portion of data pages to a corresponding fourth portion of the logical addresses.