362 results about "Data page" patented technology

The present application relates to a management method and management apparatus for a hybrid memory system, and a device and a medium. The method comprises: acquiring a write popularity value and an access popularity value of a currently accessed memory page; comparing the write popularity value with a specified write popularity threshold value, and generating write popularity page migration flag information; comparing the access popularity value with a specified access popularity threshold value, and generating access popularity page migration flag information; grouping hybrid memories in a hybrid memory system, and generating, on the basis of memory page information of all grouped memory banks, memory page management information corresponding to the hybrid memories; and on the basis of the write popularity page migration flag information, the access popularity page migration flag information, and current memory page management information corresponding to the currently accessed memory page, determining whether the current access memory page is a popular data page, and performing data migration according to a preset unpopular and popular data page migration strategy. The technical solution can improve the overall performance of a hybrid memory system having a horizontal architecture.

The invention discloses a data processing method, a storage medium, electronic equipment and a program product, and relates to the technical field of computers.The method comprises the steps that firstly, data access information is extracted from a solid state disk, and a currently accessed data block is determined; obtaining an access historical record table corresponding to the currently accessed data block, wherein the access historical record table comprises a data block access historical record table and a data page access historical record table; determining target pre-read data based on the weights of the plurality of data blocks and the weights of the plurality of data pages in the access historical record table; and if it is detected that the position of the target pre-read data is not in the pre-read position linked list, loading the target pre-read data into a cache of the solid state disk. Compared with the prior art, the method has the advantages that by maintaining the access history record table and recording the access frequency and weight of each data block and data page, the next data block and data page which are most likely to be accessed can be predicted more accurately, and then the pre-reading hit rate is increased.

The invention provides a data rereading method, device and equipment, which are applied to the technical field of data storage, and the method comprises the following steps: when data reading is performed on a flash memory, if an error rate exceeds an error correction range value, acquiring a first offset voltage parameter group from a preset rereading table, and rereading a target data page by using the first offset voltage parameter group. And if the target data page fails, performing voltage threshold distribution scanning processing on the target data page to obtain a second offset parameter set of the target data page. And comparing the second offset parameter set with the first offset parameter set to determine the voltage offset direction. And according to the voltage offset direction, searching a target reread offset voltage item in the selectable offset voltage items, determining a second offset voltage parameter group, reading the target data page again by using the reread offset voltage item, and performing reread meter parameter updating, the problems that in the current data rereading process, the flash memory performance is reduced, the historical experience dependency degree is high, and the process of calculating the optimal voltage threshold value is limited are solved.

The invention belongs to the technical field of storage media, and particularly relates to a storage medium abrasion balancing system based on cold and hot data perception, which comprises a cold and hot data perception module used for monitoring the access frequency and the update time of data in real time so as to judge the cold and hot degree of the data, and establishing a heat table, recording a heat value of each data block or data page; and the data feature prediction module is used for predicting features of a future heat change trend and a data volume increase trend of the data by utilizing a machine learning algorithm according to historical data access and update records. According to the method, the hot and cold data sensing module is used for monitoring data access and updating conditions in real time to calculate a hot value, and the storage medium management module is combined to preferentially store hot data into blocks with less erasing times and store cold data into blocks with more erasing times. Therefore, the erasing times of the blocks of the storage medium tend to be balanced, and excessive abrasion of individual blocks is avoided.

The invention discloses a row-level data recovery method and device for a damaged file of a GoldenDB database, and the method comprises the following steps: S1, scanning a to-be-recovered GoldenDB data file, and recognizing and caching all data pages; s2, on the basis of metadata of the data pages, cluster index leaf node pages containing user data are screened out from the cached data pages; s3, for each screened leaf node page, analyzing an internal record organization structure of the leaf node page, and performing row-level data recovery; the row-level data recovery comprises the steps of traversing user records according to a virtual record pointer, and performing cross check and recovery on a traversal path in combination with a page directory slot so as to extract and mark the user records one by one; and S4, outputting all the recovered row-level data.

The embodiment of the invention provides a data migration method, an access processing method, a data migration system and electronic equipment. The data migration method is applied to a switch, the switch communicates with a plurality of memory devices in a memory pool based on an interconnection protocol, and each memory device is provided with a plurality of memory areas. When data migration needs to be carried out on a first memory area in a memory pool, a switch sequentially migrates a plurality of data pages obtained by dividing data in the first memory area into available second memory areas according to pages, and in the migration process, a page bitmap determined for the first memory region is utilized to record migration states of the plurality of data pages. By adopting the scheme, the problem of large-area fault of the server caused by the memory problem can be effectively solved, the page-level granularity is adopted to migrate the data in the first memory region page by page in the migration process, the access of the server to the first memory region is continuously processed in the migration process, and the influence on the performance of the server is reduced.

Examples described herein provide a computer-implemented method for storing data pages of a database in a remote storage. The method includes receiving a write request associated with one or more data pages of the database. The method further includes storing, in a first storage layer having a write ahead log, the write request associated with the one or more data pages. The method further includes storing, in a first write buffer, the received write request associated with the one or more data pages that each include a legacy data page identifier. The method further includes storing, in a second storage layer having a second write buffer, a sorted string table (SST) of the received write request. The method further includes storing, in a top level of a log structured merge (LSM) tree disposed in the remote storage, the SST from the second write buffer.

The invention provides a data synchronization method and device, equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical field of databases, distributed storage technologies and data synchronization. According to the specific implementation scheme, the method comprises the steps that according to the sequence number sequence of a to-be-applied first physical log, log stream continuity verification is conducted on each log record contained in the first physical log in sequence; under the condition that the log stream continuity verification is passed, performing modification chain integrity verification according to a first identifier of the log record and a current version identifier of a target data page to be modified of the log record; wherein the first identifier is used for indicating a second physical log which finally modifies the target data page; and under the condition that the integrity verification of the modification chain is passed, modifying a target data page according to the log record.

The invention relates to the technical field of electric digital data processing, in particular to a WAL log data query method and device, a time sequence database and a medium, and the method comprises the steps that a log information query instruction is received to obtain a query target; in response to the log information query instruction, searching metadata of a corresponding data table in global metadata of a time sequence database, and determining write-in position information of a query target based on the metadata; based on the write-in position information and a pre-constructed WAL log coding structure, the query target is positioned, and the WAL log coding structure comprises a plurality of data blocks, a plurality of data pages and a data segment. Therefore, the technical problems that in the related technology, unnecessary data page locking is carried out during data query, the cache utilization rate is low, a large number of useless caches are likely to be generated, and then the database query performance is affected are solved.

A method and associated memory system for randomizing memory storage data. The method and system receive at a data inverter user data and meta data sequence having an inversion seed bit, determine a value for the inversion seed bit from a look up table specifying inversion seeds for different pages of data to be stored in a memory, depending on the value of the inversion seed bit, bit-flip the user data and meta data sequence except for the inversion seed bit; and regardless of bit-flipping, exclusive OR (XOR) the user data and meta data sequence with a random sequence to produce an XORed sequence for storage in the memory as randomized data.

The invention relates to an openGauss database mirror image backup recovery method and an openGauss database mirror image backup recovery device. The method comprises the following steps: executing full backup, and backing up a dependency item of database service, a data file and a WAL file; incremental backup is executed, and only the data pages and the WAL files which are changed since the last backup are backed up; and executing recovery operation: executing full-amount recovery and specified time point recovery, and supporting single-node recovery and cluster recovery. According to the openGauss database mirror image backup and recovery method based on the CDM technology flow, the backup and recovery efficiency is remarkably improved through the functions of original format backup, incremental backup, specified time point recovery and the like; the method is high in flexibility, supports full backup, incremental backup and specified time point recovery, and can meet backup requirements in different scenes; according to the method, the backup data is kept in a native format, additional processing is not needed during recovery, and data integrity and availability are ensured.

The invention provides a UI automatic test system and method, computer equipment and a storage medium, a manual case structuring module receives a manual case and converts the manual case into structured data in a preset format; the page element obtaining module calls a browser driving engine according to the target URL to obtain full page elements; the AI element matching module is used for scoring full elements of a page through a pre-matching algorithm based on operation types and interface element descriptions in the structured data, screening out first K elements, and carrying out secondary screening through a large model to determine target elements; and the automatic use case execution module generates an automatic test use case according to the target element and the operation data in the structured data, executes the automatic test use case and verifies whether an execution result conforms to expectation or not to realize UI automatic testing. By the adoption of the scheme, the labor cost of UI testing is reduced, the efficiency and adaptability of UI testing are improved, and then the automation level and the large-scale capacity of UI testing are improved.

The invention relates to a data page recovery method and device, equipment, a medium and a product. The method comprises the steps that when it is detected that a current data page is a faulty data page, a corresponding relation associated with a data page identifier is searched for according to the data page identifier of the current data page; reading the address information and the log serial number of the historical version data page from the corresponding relation; and reading the historical version data page according to the address information, and recovering the current data page according to the historical version data page and the log serial number to obtain a target data page. By adopting the method, the recovery efficiency of the data page in a high-concurrency writing scene can be improved.

A memory device includes a memory cell array configured to store data, a page buffer circuit configured to store data in the memory cell array or read data stored in the memory cell array, and a scan register circuit configured to receive a pass / fail result of data from the page buffer circuit and store the pass / fail result in a plurality of scan registers. The scan register circuit obtains information about the scan register where the fail result is stored through a scan operation, and uses the information about the scan register where the fail result is stored to determine the number of fail bits and / or a position index indicating the position of the fail bits according to the operation mode.

The embodiment of the invention provides a core dump file generation method and device, electronic equipment and a medium, and relates to the technical field of data processing.The method comprises the steps that in response to abnormity of a target process, a thread causing the abnormity of the target process is determined as a target thread from threads of the target process; retrieving a virtual memory address from a register occupied by the target thread and a stack memory area of the target thread; determining a target address of a data page which belongs to a heap memory area and has mapping from the retrieved virtual memory address; and generating a core dump file containing the data page mapped by the target address. By applying the scheme provided by the embodiment of the invention, the validity of the core dump file can be improved while the size of the core dump file is reduced.

The present invention relates to the technical field of computers. Disclosed are a data processing method, a computer readable storage medium, and an electronic device. The method comprises: acquiring, from a first storage medium, data objects to be evicted, wherein the data objects to be evicted use a data storage format of a preset data granularity; on the basis of a preset customization policy, selecting a target data object from among the data objects to be evicted, and determining an indexing method of the target data object in a second storage medium; and caching the target data object to the second storage medium according to the indexing method in the second storage medium, wherein the preset customization policy comprises: a filtering condition, the filtering condition being used for performing data filtering on the data objects to be evicted, and the second storage medium is an extended storage area of the first storage medium. The present invention solves the technical problems in the related art that database execution efficiency is low and cache efficiency is low due to directly caching evicted data pages in a buffer pool into a buffer pool extension.

The invention discloses a data processing method, a computer readable storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps that a to-be-eliminated data object is obtained from a first storage medium, and the to-be-eliminated data object adopts a data storage format of preset data granularity; based on a preset customization strategy, selecting a target data object from the to-be-eliminated data objects, and determining an indexing mode of the target data object in a second storage medium; caching the target data object to a second storage medium according to an index mode in the second storage medium; wherein the preset customization strategy comprises a filtering condition, the filtering condition is used for performing data filtering on the to-be-eliminated data object, and the second storage medium is an extended storage area of the first storage medium. According to the method and the device, the technical problems of relatively low database execution efficiency and low caching efficiency caused by directly caching the eliminated data pages in the cache pool into the cache pool extension in the related technology are solved.

A data analysis method, an electronic device and a readable medium are provided The data analysis method includes: displaying a target data page, the target data page being used to display target data; displaying a first intelligent object page in response to a trigger operation on a first intelligent object on the target data page; and displaying a target dashboard page in response to target information entered on the first intelligent object page and used to create a dashboard for the target data. The target dashboard page includes at least one data chart, the at least one data chart is obtained by analyzing the target data based on analysis dimension information, and the analysis dimension information is determined based on a data dimension in the target data.

The invention relates to an exit-entry certificate authenticity identification method and system based on two-dimensional code identification, and a medium. The method comprises the following steps: constructing a cross verification rule base comprising logical relationship information between two-dimensional code information of the entry-exit certificate and signing and issuing information of the entry-exit certificate; acquiring a data page image of an entry-exit certificate to be detected; identifying a two-dimensional code in the preprocessed data page image by using a two-dimensional code identification engine; identifying signing and issuing information in the preprocessed data page image; according to the dynamic two-dimensional code verification library, the dynamic blacklist library and the identified signing and issuing information, uniqueness verification and legality verification are carried out on the identified two-dimensional code information; comparing the corresponding relationship between the identified two-dimensional code information and the identified signing and issuing information according to a cross verification rule base; and judging the authenticity of the entry-exit certificate and generating a report. According to the invention, true and false identification is carried out through three dimensions, the identification accuracy is high, and high-risk false and false situations such as same codes, different certificates and tampering fields can be accurately identified.

Fast iterative rewinding of a database to a past time point is supported. Responsive to a database rewind request, a target LSN for a target time is obtained, along with a set of persisted LSNs corresponding to a set of slices. Each slice includes a valid anchor checkpoint smaller than and closest to the target LSN. A minimum persisted LSN and change log records based thereon and on the target LSN are also obtained. The change log records are applied to anchor pages of the slices to restore the database to the target time. Also provided for is database management, including generating anchor checkpoints for recording page content corresponding to LSNs and indicating anchor pages. Also provided for is scaling back restorable time window size, including, when a storage threshold is exceeded, releasing stored data pages for deletion based on usage levels.

Techniques can include: recording, in a log, log entries for write operations writing data pages to logical addresses; and flushing the log entries from the log, wherein flushing includes performing deduplication of the data pages. Flushing can include: partitioning the data pages into i) a first portion of the data pages that are determined to be duplicates of existing data pages, and ii) a second portion of the data pages determined to be unique data pages; atomically incrementing reference counts each corresponding to one of the existing data pages determined as identical to a data page of the first portion; and atomically binding the logical addresses to the data pages, including: binding the first portion of data pages to a corresponding third portion of the logical addresses; and binding the second portion of data pages to a corresponding fourth portion of the logical addresses.

The invention discloses a data addressing method and device, equipment and a storage medium. Comprising the steps of generating a data graph containing data page entity data and data page relational data based on data page data, and performing vectorization processing on the data graph to generate a vectorized graph; performing vectorization processing on query information input by a user, and constructing an addressing vector; generating an addressing knowledge graph based on the addressing vector and the vectorized graph; and outputting a data address matched with the query information by utilizing the query information and the addressing knowledge graph through the created addressing cue word and the large language model. According to the method, the data page data and the system address are fused to generate the knowledge graph, the knowledge graph is used as a data storage medium, and data semantic conversion is completed by writing the cue word template and quoting the understanding ability of the large model to the characters, so that accurate addressing of the system is completed according to the input information, and the accuracy and efficiency of data addressing are improved.

The embodiment of the invention provides a data processing method and device, electronic equipment and a storage medium, belongs to the technical field of data processing, and is suitable for financial science and technology scenes. The method comprises the steps that a data table scanning request is obtained, a service scene is determined, and the data table scanning request is used for scanning at least one data table in a target database; creating a temporary cache queue in a cache pool according to the service scene; wherein the temporary cache queue comprises a temporary hot cache region and a temporary cold cache region; loading a target data page from a target database based on the data table scanning request, and loading the target data page to a temporary cold cache region; in response to the condition that a target data page in the temporary cold cache region meets a preset access condition, transferring the target data page to the temporary hot cache region; wherein the temporary hot cache region forbids data page replacement. According to the embodiment of the invention, the data processing efficiency can be improved.

The invention relates to a solid state disk dynamic wear balancing method and system based on data popularity, and the method comprises the steps: tracking the access characteristics of each data page in a solid state disk through a page-level counter, and enabling the access characteristics to comprise the write operation frequency, the modification period and the survival time; the data pages are dynamically divided into high-heat data, medium-heat data or cold data based on a preset threshold value, and corresponding heat layering labels are marked; collecting accumulated writing times and I / O request queue length of each channel of the solid state disk in real time, and calculating wear imbalance degree and load imbalance degree; performing data exchange according to a preset exchange mode priority based on the popularity hierarchical label, the wear imbalance degree and the load imbalance degree; and updating the logic mapping table and the channel state parameters, carrying out statistics on the balance income and the system overhead, and dynamically correcting the tolerance coefficient based on the current load type according to the income-overhead ratio.

A flush set of data pages to flush from a cache to non-volatile data storage is identified, and a compression group is identified that is made up of similar data pages within the flush set. A previously compressed data page is identified in non-volatile data storage that is similar to the data pages in the compression group. Data compression is performed on the compression group using a compression dictionary for the previously compressed data page. A flush set specific hash index and a global hash index may be used to identify the similar data pages in the compression group and the similar previously compressed data page. An indication of the similar previously compressed data page may be stored in the compressed block generated for the compression group, and the compression dictionary for the previously compressed data page need not be duplicated in the non-volatile data storage.

Embodiments of the present disclosure provide a data migration method, an access processing method, a data migration system, and an electronic device. The data migration method is applied to a switch, wherein the switch communicates with a plurality of memory devices in a memory pool on the basis of an interconnection protocol, and each memory device has a plurality of memory areas. When data migration needs to be performed on a first memory area in the memory pool, the switch sequentially migrates a plurality of data pages, which are obtained by dividing data in the first memory area according to pages, into an available second memory area, and one page bitmap determined for the first memory area can further be used during migration to record the migration states of the plurality of data pages. By using the present solution, the problem of widespread server failures caused by a memory problem can be effectively solved, page-level granularity is used during migration to migrate the data in the first memory area page by page, enabling continuous processing of server accesses to the first memory area throughout the migration process, thus reducing the performance impact on the server.

The invention relates to a time sequence database query method, system and equipment based on direct calculation of compressed data and a medium. The time sequence database query system comprises a client, a query layer and a storage layer, the client is used for sending an SQL query request input by a user to the query layer and receiving a query result returned by the query layer; the query layer is used for reading a target data block from the storage layer according to an SQL query request, constructing the target data block into a compressed state data memory representation by adopting a delay decompression mechanism and a dynamic auxiliary structure management mechanism, and directly executing homomorphic calculation on the generated compressed state data memory representation by utilizing a homomorphic operator; the obtained query result is returned to the client in an uncompressed format; the storage layer is used for storing compressed data blocks, each compressed data block comprises a plurality of compressed data pages, and each compressed data page comprises compressed time sequence data. The method can be widely applied to the technical field of big data processing.

The invention relates to a database fault recovery method and device, a storage medium and electronic equipment. The method comprises the steps that when a database normally runs, a WAL log is written into a disk, and meanwhile, a log record is pre-analyzed through a background asynchronous thread to extract core semantic information of the log record; constructing a cache structure sorted according to table space identifiers and data page numbers in the shared memory, and storing the core semantic information in the cache structure in a recovery-oriented compact format; when the check point is executed, the memory dirty page is flashed to a disk, and the cache initial position corresponding to the check point in the cache structure serves as metadata to be written into a control file; and after the database crashes and is restarted, acquiring a cache initial position from the control file, loading all core semantic information entries behind the cache initial position in the cache structure according to the cache initial position, positioning a target data page, and modifying the target data page. The technical problem that an existing database collapse recovery technology is low in recovery speed is solved.

Methods, systems, and devices for a parity scheme for read distress protection in a memory system are described. The parity scheme described herein may protect data against read distress errors. A set of consecutive word lines in a memory system may include one or more word line groups and one or more sub-block groups. The memory system may store, in a word line group of the set of consecutive word lines, parity information for the set of consecutive word lines. Each parity page may be a combination of one page of data per sub-block group and per word line group of the set of consecutive word lines. For example, each sub-block group and each word line group may contain N pages, where each of the N pages may correspond to a unique parity page to provide for recovery from errors caused by both read and program distresses.