45 results about "Block level" patented technology

The invention provides a data association maintenance processing method and system based on file reference, and relates to the technical field of data association maintenance, which comprises the following steps: constructing a directed reference graph, identifying direct and indirect reference paths, and constructing a reference dependence intensity matrix in combination with a time sequence, an interaction amount and a reference propagation probability. And carrying out file clustering based on the matrix, and determining a core reference file set. And when the core file is updated, acquiring incremental update data by adopting data block level difference comparison, and merging the data blocks based on the content similarity and the semantic association degree. After the merged data blocks are segmented according to threshold values and semantic boundaries, updating is propagated to a target node through a multi-path updating propagation network, and the data consistency is ensured through version consistency check and a compensation updating mechanism. According to the method, data association between files can be effectively maintained, and the data updating efficiency and consistency are improved.

The invention discloses a block-level data backup method and device, relates to the technical field of data processing, and mainly aims to solve the problem of poor validity and accuracy of existing data backup. Comprising the steps that block division information of a target data file is obtained, and the block division information comprises the block data size and the block identification for block division of the target data file; when it is monitored that target block-level data in the target data file is updated based on the block division information, extracting the target block-level data; and sending the target block-level data with the target block identifier to a server side, so that the server side performs merging, duplicate removal and backup on the data content of the target data file and the target block-level data based on the target block identifier.

The invention provides a file fuzzy copying method and system based on a big data file cluster, and relates to the technical field of file copying, the file fuzzy copying method comprises the steps that file content, file names and metadata feature vectors are extracted from a to-be-matched file set, and the file content feature vectors are obtained based on deep learning model coding; afterwards, a distributed computing framework is used for parallel computing of similarity scores of the to-be-matched files and the files in the target file set, the scores are obtained through weighted computing of file content, file names and metadata feature vector similarity, and a to-be-copied file list is generated through screening according to a preset threshold value. And finally, the distributed file system dynamically allocates a replication task according to system resources, executes file replication based on a data block level parallel transmission mechanism, verifies data integrity and generates a replication task execution report. According to the method, fuzzy file copying can be efficiently and accurately carried out in the big data file cluster, the file copying efficiency and accuracy are improved, and the system resource consumption is reduced.

Example implementations relate to data storage. An example includes inspecting a block level input / output (I / O) request to be executed by a block-based storage device, and in response to a determination that the block level I / O request includes a filesystem operation, generating a copy of the block level I / O request. The example also includes parsing the copy of the block level I / O request to extract a plurality of attributes of the filesystem operation, where the parsing is asynchronous to an execution of the block level I / O request by the block-based storage device. The example also includes storing the extracted plurality of attributes of the filesystem operation in an entry of a filesystem operation database, where each entry of the filesystem operation database is associated with a different filesystem operation in a filesystem stored on the block-based storage device.

The invention relates to a three-dimensional model fingerprint generation method and system, a terminal and a storage medium. The method comprises the following steps: identifying a file type, analyzing a file structure for the file type containing structured entity information, and extracting triangular mesh entity data; executing a block sampling strategy based on the file size for the file type which does not contain the structured entity information; counting the total number of triangular facets for the triangular mesh entity data, and selecting an adaptive hierarchical sampling strategy according to the total number of the triangular facets; triangular surface area sampling is carried out on a triangular mesh entity, a sampling block-jumping block alternating traversal mode is adopted, and compared with an existing mode, the performance is improved by 69-83 times; besides, a two-level Hash structure is adopted, the first level is to calculate the MD5 value of the area accumulated sum for each sampling block, the second level is to calculate the overall MD5 value for the connection sequence of all block level MD5 values, and the two-level structure enhances the uniqueness and collision resistance of fingerprints.

A memory sub-system, having: a host interface configured to operate on a computer bus; non-volatile memory cells; and a controller. In response to a command to identify information about the memory sub-system, the controller is to provide structured data indicating that the memory sub-system supports sub block access and specifying a sub block granularity level for the sub block access. In response to an access command with a sub block descriptor embedded within the access command, the controller is to determine, based on the sub block descriptor provided within the access command, a portion of a logical block identified by the access command and implemented using a subset of the non-volatile memory cells. An operation is performed on the portion of the logical block according to an opcode specified by the access command.

A computing system having a host system connected to a memory sub-system via a computer bus. In response to the host system sending a first command configured to identify a first logical block addressing data size at a block level, the memory sub-system can format data storage for a namespace in a non-volatile memory at the block level. In response to the host system sending a second command configured to identify a second logical block addressing data size at a sub block level, the memory sub-system can configure access to the namespace at the second logical block addressing data size without changing the data storage in the non-volatile memory for the namespace at the block level.

The invention relates to a quantization acceleration method for a stable video diffusion model, and the method comprises the steps: carrying out the comprehensive quantization acceleration design of a reasoning optimizer based on a U-Net network structure in three granularities: Kernel, Layer and Block, and carrying out the universal matrix multiplication quantization or operator fusion of different types of Kernel in a Kernel level; in Layer and Block levels, end-to-end schemes oriented to different scenes are provided through qualitative and quantitative analysis, and finally, the speed-up ratio of 1.63 is up to the maximum on the premise that the video precision is guaranteed. When the method is used for generating large model information in a diffusion type video, the loading and utilization rate of video data can be improved on the premise that the image and video precision is guaranteed.

Aspects of the disclosure include methods, apparatuses, and non-transitory computer-readable storage mediums for video encoding / decoding. An apparatus includes processing circuitry that decodes a video bitstream to obtain a reduced-resolution residual block for a current block. The processing circuitry determines that a block level flag is set to a pre-defined value. The pre-defined value indicates that the current block is coded in reduced-resolution coding. Based on the block level flag, the processing circuitry generates a reduced-resolution prediction block for the current block by down-sampling a full-resolution reference block of the current block. The processing circuitry generates a reduced-resolution reconstruction block for the current block based on the reduced-resolution prediction block and the reduced-resolution residual block. The processing circuitry generates a full-resolution reconstruction block for the current block by up-sampling the reduced-resolution reconstruction block.

A method, apparatus, and system for tracking row and object database activity into block level heatmaps is provided. Database activity including reads, writes, and creates can be tracked by a database management system at the finest possible level of granularity, or the row and object level. To efficiently record the tracked database activity, a two-part structure is described for writing the activity into heatmaps. A hierarchical in-memory component may use a dynamically allocated sparse pool of bitmap blocks. Periodically, the in-memory component is persisted to a stored representation component, sharable with multiple database instances, which may include consolidated last access times and / or a history of heatmap snapshots to reflect access over time. The heatmaps may then be externalized to database users and applications to provide and support a variety of features.

The invention provides a block-level incremental backup method and device for a database, equipment and a medium. The method comprises the steps that data files in the database are traversed; for any traversed data file, whether the data file is a small file or not is determined according to a preset size, and the small file is a data file with the file size smaller than the preset size; under the condition that at least two data files which are small files are determined, merging the at least two data files which are small files to obtain at least one merged file; and partitioning the at least one combined file to obtain at least one data block, and performing block-level incremental backup according to the at least one data block. On the basis, the number of the data blocks can be reduced, the backup time consumption is reduced, and the database incremental backup efficiency is improved.

The invention relates to the technical field of computers, and provides a service point burying method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining an instrumentation configuration file corresponding to an intermediate representation file of an underlying virtual machine; the instrumentation configuration file comprises an instrumentation position and an instrumentation mode of the intermediate representation file; according to the instrumentation configuration file, performing instrumentation on code blocks of a service class method in the intermediate representation file to obtain calling information and method parameters of each code block; and based on the calling information and the method parameters, generating a point burying group corresponding to each code block, and performing point burying on each code block according to the point burying group. According to the technical scheme, the calling information and the method parameters of the code blocks of the method are obtained in an instrumentation mode, the burying point groups of the code blocks are generated for burying points, and the burying points of the code block level can be achieved, so that full coverage of the business code burying points is achieved, and the coverage degree of business methods is improved.

A system is disclosed. A first storage device may supporting a cache coherent interconnect protocol, the cache coherent interconnect protocol including a block level protocol and a byte level protocol. A second storage device may also support the cache coherent interconnect protocol. A redundant array of independent disks (RAID) circuit may communicate with the first storage device and the second storage device. The RAID circuit may apply a RAID level to the first storage device and the second storage device. The RAID circuit may be configured to receive a request using the byte level protocol and to access data on the first storage device.

The application discloses a kind of automatic promotion and cross backend generation method and device of tensor kernel, belong to GPU heterogeneous computing field.This method includes: obtaining target CUDA tensor kernel and converting into structured SIMT intermediate representation;According to thread block execution domain, build block context;For thread related branch and loop, generate corresponding predicate context and shared driving loop;Generate block intermediate representation, and through access image analysis, memory to register promotion, reduction and matrix multiplication normalization, SLP packing and layout allocation etc.Optimization, the block level calculation implied in low-level SIMT scheduling is restored to explicit block operation;The block level program after optimization is output to block backend generation target code.The application can reduce the artificial rewriting work required for the migration of CUDA tensor kernel to block level programming model, and improve the portability and optimization space of existing GPU kernel on different backends and hardware platforms.

This application discloses a video scheduling method, apparatus, device, and readable storage medium, comprising: acquiring multiple segments of a video file to be stored; storing the multiple segments in a resource node cluster and recording the storage addresses of the multiple segments; determining a logical block partitioning strategy for the video file to be stored; generating a logical block mapping table corresponding to the video file to be stored based on the logical block partitioning strategy, the multiple segments, and the storage addresses of the multiple segments; and storing the logical block mapping table corresponding to the video file to be stored. As can be seen, this application, by logically partitioning the video file and generating a logical block mapping table, enables scheduling of the video file at the logical block level. This allows for adaptation to the storage capacity of resource nodes when storing the video file and avoids generating massive scheduling requests, thereby improving the efficiency of video scheduling.

A system and method for efficiently restoring one or more data containers is provided. A common persistent consistency point image (PCPI) is identified between a source and a destination storage systems prior to the destination storage system performing a rollback operation to the commonly identified PCPI. Differential data is then transmitted from the source storage system in a line efficient manner to the destination storage system.

The invention provides a verification transmission method and system for data asset exchange, and relates to the technical field of data transmission, and the method comprises the steps: dividing a storage space into multiple levels of storage blocks; reading to-be-submitted data of a data submitting user; performing data expression level verification on the to-be-submitted data, determining the data expression level to which the to-be-submitted data belongs, reading stored data in the storage block level mapped by the data expression level to which the to-be-submitted data belongs, and performing redundancy verification on the to-be-submitted data to obtain a redundancy verification result; and according to the data expression level and the redundancy verification result, generating a sub-block transmission protocol, and transmitting the to-be-submitted data to the storage sub-block of the corresponding storage block level. By means of the data management method and device, the technical problem that in the prior art, due to the fact that storage redundancy is prone to occurring in the data processing process, data management is disordered is solved, redundant data are verified and stored through a multi-layer data storage structure, and the flexibility and efficiency of data management are improved.

The invention discloses a storage device and an execution method of memory operation of the storage device. The storage device comprises a storage array and an address processing module, the storage array comprises a plurality of storage blocks, and each storage block comprises a plurality of normal storage pages and redundant storage pages; the redundant storage page is configured to have the capability of recording a mapping relationship between a logic page address and a physical page address of any normal storage page in the storage block; the address processing module is connected with the storage array, and when the storage device is configured to execute an out-of-order programming function, the address processing module is configured to respond to an addressing instruction in a user command and construct or obtain a mapping relation between a logic page address and a physical page address so as to carry out addressing. By means of the mode, the storage blocks of the storage device can be independently managed, the interference influence of programming on data in the storage blocks can be reduced at the levels of the storage blocks through the out-of-order programming technology, and therefore the data reading and writing efficiency of the storage device is improved.

This disclosure provides a method and apparatus for online migration of a business system. It parses the system structure of each first file system in the source first business system to obtain multiple data blocks corresponding to the first business system and first sector information corresponding to each data block. Using the first sector information, each data block is written to the destination to construct a second business system at the destination. During the construction of the second business system at the destination, data block change information corresponding to any data block in the source first business system is obtained. This data block change information is recorded in a preset bitmap. After the construction of the second business system at the destination is completed, the data block change information in the preset bitmap is read to update the data in the second business system. This disclosure, by providing block-level full object copying and data block migration based on disk sector logic, facilitates the smooth online migration of various business systems to the destination.

A data storage device includes a memory block failure anticipation system. The memory block failure anticipation system proactively detects defects in wordlines and / or memory blocks of the data storage device. To determine the presence of defects, the memory block failure anticipation system determines a failed bit count associated with one or more wordlines of a selected memory block and / or a failed bit count associated with the memory block. The failed bit count is compared against two thresholds—a block level data loss threshold and a wordline level data loss threshold. If the failed bit count exceeds the block level data loss threshold, the memory block is retired. If the failed bit count exceeds the wordline level data loss threshold but not the block level data loss threshold, a particular wordline of the memory block is retired and / or the memory block is retired.

The present invention proposes an encryption deduplication method for resisting convergence key leakage, the method comprising the following steps: S1 : Set security parameters, the number of data block fragments and n × n The matrix is ​​constructed by generating a matrix of the same order, and the element values ​​therein are set, a fixed-length hash function and a multi-set hash function based on an XOR operation are output, and the master key of the storage server is set; S2 : The client first calculates the file F The file key of the file F Split into fixed-size chunks F :{ f1 , f2 , f3 ..., fn}. Calculate the hash value of each file block to obtain the convergence key; the present invention supports two-level deduplication at the file level and the block level. The client and the storage server first perform file-level deduplication, and then perform block-level deduplication when there is no duplication, saving storage space to the greatest extent. Secondly, the present invention supports client-side deduplication, reducing the communication cost between the client and the storage server.

The invention belongs to the technical field of nonvolatile memories, and particularly discloses a ZNS SSD page-level perception-based block erasure optimization method, which further introduces a page-level perception erasure capability on the basis of the existing block-level partial erasure scheme. By constructing a lightweight page-level write-in state tracking structure, the actually written-in pages are identified, and then which pages in which blocks are written in are accurately positioned during zone reset, so that the situation that the whole block is erased on the block in which only a very few pages are written is avoided. According to the method and the device, the abrasion waste caused by invalid erasure of unused Page in the Block can be further reduced, so that the service life of the ZNS SSD is remarkably prolonged, and the space efficiency of the ZNS SSD is remarkably improved.

Disclosed herein are system, method, and computer program products for storing a snapshot of block-level changes in a data volume using a file system-level backup tool. An embodiment operates by receiving the block level changes in the snapshot from the data volume stored in an application in a cluster. The embodiment creates a root directory for the snapshot. The root directory may have one or more subdirectories. The embodiment maps one or more block changes in the data volume onto the one or more subdirectories and creates a manifest for identifying the snapshot in the file system-level backup tool. The manifest may be associated with the root directory. The embodiment then instructs the file system-level backup tool to store the snapshot by storing the root directory in content-addressable object storage based on the one or more subdirectories and storing the manifest in a manifest storage.

Example implementations relate to data storage. An example includes inspecting a block level input / output (I / O) request to be executed by a block-based storage device, and in response to a determination that the block level I / O request includes a filesystem operation, generating a copy of the block level I / O request. The example also includes parsing the copy of the block level I / O request to extract a plurality of attributes of the filesystem operation, where the parsing is asynchronous to an execution of the block level I / O request by the block-based storage device. The example also includes storing the extracted plurality of attributes of the filesystem operation in an entry of a filesystem operation database, where each entry of the filesystem operation database is associated with a different filesystem operation in a filesystem stored on the block-based storage device.

The present disclosure describes techniques of metadata management for transparent block level compression. A first area may be created in a backend solid state drive. The first area may comprise a plurality of entries. The plurality of entries may be indexed by addresses of a plurality of blocks of uncompressed data. Each of the plurality of entries comprises a first part configured to store metadata and a second part configured to store compressed data. Each of the plurality blocks of uncompressed data may be compressed individually to generate a plurality of compressed blocks. Metadata and at least a portion of compressed data associated with each of the plurality of compressed blocks may be stored in one of the plurality of entries based on an address of a corresponding block of uncompressed data. A second area may be created in the backend solid state drive for storing the rest of the compressed data.

The disclosure relates to data adaptation in a hardware acceleration subsystem. In some examples, a hardware acceleration subsystem (310) includes: a first scheduler (382a); a first hardware accelerator (350a) coupled to the first scheduler (382a) to process at least a first data element and a second data element; and a first load storage engine (352a) coupled to the first hardware accelerator (350a), the first load storage engine (352a) configured to communicate with the first scheduler (382a) at a super block level by sending a completion signal to the first scheduler (382a) in response to determining that a block count is equal to a first BPR value, and aggregate the first data element with the second data element based on the first BPR value to generate a first aggregated data element.

The invention discloses a data integrity verification method, device and equipment and a readable storage medium, and is applied to the technical field of cloud computing and data storage, and the method comprises the following steps: determining a to-be-backed-up data block of which the data is changed based on a backup chain topological graph corresponding to a virtual machine; the backup chain topological graph is a graph for dynamically recording the changed address of the virtual machine data; calculating a first verification value corresponding to the to-be-backed-up data block by using a verification method; when the to-be-backed-up data block is backed up successfully, calculating a second check value of backup data corresponding to the to-be-backed-up data block based on the check method, and determining a hash value of an associated data block associated with the to-be-backed-up data based on the backup chain topological graph; and performing data integrity verification based on the first verification value, the second verification value and the hash value of the associated data block, and determining a data integrity result. According to the method, only the changed data blocks are checked, so that the checking time consumption is greatly reduced, the checking efficiency can be improved, and the block-level definition is adopted, so that the checking precision can also be improved.

A system and method for efficiently restoring one or more data containers is provided. A common persistent consistency point image (PCPI) is identified between a source and a destination storage systems prior to the destination storage system performing a rollback operation to the commonly identified PCPI. Differential data is then transmitted from the source storage system in a line efficient manner to the destination storage system.

A system and method for forensic analysis of block level storage devices for detecting cybersecurity attacks is presented. The method includes accessing a disk in a computing environment, the disk including deleted data; recovering at least a file from the deleted data; initiating a forensic analysis on the at least a file to detect a forensic artifact, wherein the forensic artifact indicates a cybersecurity attack; detecting the forensic artifact on the on the at least a file; and initiating a remediation action on a virtualization associated with the disk based on the detected forensic artifact.