25 results about "Volume (computing)" patented technology

This application provides a data processing method, system, apparatus, device, storage medium, and program product, relating to the field of big data. Applied to any parent company computing node in a confidential environment cluster, the parent company computing node is configured with a confidential environment. The method includes: initiating a data usage request to a subsidiary node and receiving a subsidiary key sent by the subsidiary node; combining the parent company key and the subsidiary key to obtain a disk storage key; decrypting the disk storage data of the subsidiary node stored in the confidential environment using the disk storage key; performing distributed processing on the decrypted disk storage data in the confidential environment to obtain a processing result, and sending the processing result to the subsidiary node. This application's method reduces the risk of data leakage and meets data protection requirements; by distributing the processing tasks through the parent company computing node, it simultaneously solves the problems of large data volumes and high processing time requirements.

A computing device, non-transitory computer-readable storage medium, and method for automatically recreating a lost storage volume are provided. A volume command including volume metadata associated with a storage volume of an SCM storage device is received. The volume metadata is stored in another storage device that is more non-volatile than the SCM storage device. The storage volume is configured in accordance with the volume command, including forwarding the volume command to an SCM device driver. After forwarding the volume command, a notification of a persistent loss at the SCM storage device is received. After receiving the notification of the persistent loss, the volume metadata is retrieved from the other storage device. The storage volume is automatically reconfigured in accordance with the volume command, including submitting the retrieved volume metadata to the SCM device driver.

A computing device comprising a storage device and processing circuitry may perform the techniques of this disclosure. The storage device may have a plurality of blocks forming a volume. The processing circuitry may obtain volume changed block tracking (CBT) information identifying one or more of the blocks storing updated data that has changed relative to a previous backup of the one or more blocks, and determine file mapping information identifying one or more blocks of the plurality of blocks that store file data associated with a file. The processing circuitry may also determine, based on the volume CBT information and the file mapping information, file system CBT information identifying whether at least one of the one or more blocks store file data associated with the file have changed, and initiate, based on the file system CBT information, a subsequent backup of at least a portion of the file data.

The invention provides a disk traffic control method and device, equipment, a medium and a program product, and the method comprises the steps: creating a volume type of a storage performance development suite type through a block storage service, and associating the volume type of the storage performance development suite to a service quality specification strategy; binding the volume type to a target back-end storage system; creating a volume by specifying a volume type through a block storage service, generating logic information of the volume on a corresponding open stacking platform, and generating a block storage volume entity on a corresponding target back-end storage system; mounting the volume to a virtual machine of the computing service, and adding the service quality value of the volume to the computing service according to a service quality specification strategy; and calling an interface of the storage performance development suite to create a block device corresponding to the volume in the target back-end storage system, and configuring the block device according to the service quality value. The method provided by the invention is used for achieving the technical effect of improving the disk flow control efficiency and universality.

The application provides a bare metal server instance management method, system, storage medium and program product, relates to the technical field of cloud computing, and the method comprises the following steps: obtaining the out-of-band management address of a target bare metal server, obtaining the hardware manufacturer identifier according to the out-of-band management address, and determining a target virtual local area network; injecting a hardware driver set matched with the hardware manufacturer identifier into an operating system image associated with a machine installation template to generate a target deployment image; allocating a temporary boot IP address in the target virtual local area network and configuring a pre-boot network parameter through a substrate out-of-band interface; controlling the target bare metal server to obtain the target deployment image according to the pre-boot network parameter to install the operating system; adjusting the computing resource quota to a target resource quota through a management and control agent program during the operation of the operating system; and when a storage expansion request is received, creating a target storage volume according to a target physical disk identifier through the management and control agent program and mounting the target storage volume to a target mounting point.

The invention relates to computing. What is claimed is a method for managing data in a block data storage system consisting of a random-access memory (RAM) and a solid-state drive (SSD), which involves: dividing the SSD into consecutive volumes, each of which consists of a metadata region and a user data region; dividing the RAM into dedicated memory for uploading segments from the SSD, volume arrays and service information; transferring valid tuples of metadata from each metadata-containing segment for each volume from the SSD to the RAM; creating a metadata storage segment in the RAM; updating a metadata tuple in the metadata storage segment; writing the metadata storage segment and a backup copy thereof to the SSD. The technical result consists in reducing the number of metadata writing operations for block data storage systems.

There is provided an I / O latency reduction system, comprising at: (i) least one computing platform comprising at least one storage volume having at least two storage media; (ii) a storage stack striping component designated to perform data striping of at least one large I / O data segment into several minimally sized I / O data segments, wherein the data striping is configured to logically segment and re / compose sequential data segments such that consecutive segments are stored on different storage media; and (iii) computing resources associated with each of the at least two storage media, wherein the data striping and storage of the minimally sized I / O data segments in different storage media is configured to be conducted parallelly, wherein the minimally sized I / O data segments are a obtained by a low granularity striping process, wherein the system is configured to increase amplification of the minimally sized IO data segments while replicating them, and wherein said data striping and re / composing is designated to be performed by the said computing resources associated with each of the at least two storage media.

This invention relates to a method and apparatus for determining inter-particle interactions based on a matrix, comprising: acquiring multiple computing units currently participating in the computation, and dividing the physical space corresponding to the computing units to obtain a tower disk model corresponding to the computing units; selecting virtual subspace pairs that meet preset conditions from multiple virtual subspaces of each tower disk model, and constructing a target tower disk model based on the virtual subspace pairs; migrating the particles to be computed to the target tower disk model to determine the target particle pairs participating in the computation of Hamiltonians; and calculating the interactions between the target particle pairs based on the target tower disk model and the target particle pairs. Thus, by filtering out a large number of redundant particle pair computations, the computational load is reduced, computational efficiency is improved, and data transmission volume is reduced, load is lowered, and transmission rate is increased, thereby greatly improving supercomputing efficiency.

The embodiment of the invention relates to the field of large models and the field of near storage computing, and discloses a heterogeneous storage computing architecture scheduling method and equipment oriented to a hybrid expert model, and a medium. In the invention, the heterogeneous storage and calculation architecture comprises N storage channels and L accelerators. Each accelerator is connected with at least one of the N storage channels, and each storage channel is connected with at least one in-storage computing unit. The method comprises the following steps: acquiring channel delay of each storage channel and accelerator accumulated delay of L accelerators; obtaining a to-be-processed data volume distributed by the expert model in the in-memory calculation unit; in response to a target storage channel of which the channel delay is greater than the accumulated delay of the accelerator, moving the position of the expert model with the maximum to-be-processed data volume in the target storage channel to the accelerator from the in-memory calculation unit, and updating the channel delay and the accumulated delay of the accelerator; and in response to the channel delay of all the storage channels being less than or equal to the accelerator cumulative delay, terminating the scheduling. The problems of local overload and performance reduction during near storage calculation are solved.

Techniques discussed herein relate to an object-relational database management system (ODMS) (e.g., a PostgreSQL ODMS) that utilizes a shared block storage volume (SBSV) for storage. The SBSV may utilize a file system that enables a single-writer-multiple reader model in which a primary computing node may read or write to the SBSV and one or more replica computing nodes are restricted from writing to the SBSV. To enable the primary computing node to write data at its own pace, regardless of the status of synchronization at each of the one or more replica computing nodes, the SBSV may include a staging area that maintains data that has not yet been updated at one or more of the replica computing nodes. When the primary computing node no longer maintains the data in local memory, the one or more replica computing nodes may obtain the data from the staging area of the SBSV.

Systems, methods, and software are disclosed herein for identifying duplicate blocks of a storage system and deduplicating the storage system. In one example, a method of operating a computing device includes scanning first metadata of blocks of a container file of a virtual volume to generate a first log file including records of virtual volume block numbers (VVBNs) and fingerprints of the blocks; scanning second metadata of blocks of an active file system of the virtual volume to generate a second log file including records of VVBNs and file block numbers (FBNs) of the blocks; generating tuples based on merging the records of the first log file and the second records of the second log file according to the VVBNs; identifying duplications among the blocks based on the tuples; and deduplicating the blocks based on the duplications in the active file system identified based on the tuples.

The embodiment of the invention provides a large model compiling optimization method and system based on in-memory computing equipment. According to the method, a high-dimensional tensor of a large model is split into sub-tensors adaptive to a storage array, waste of storage resources caused by mismatching of data sizes is avoided, weight data with the reuse rate larger than a preset threshold value is preloaded to an equipment storage area, and repeated carrying of the weight data can be avoided; on the basis of a preset operator instruction matching table, a mapping relation between a large model operator and an in-memory calculation instruction is constructed, and it is ensured that equipment instruction resources can be efficiently called by the large model operator; then, based on the sub-tensor data size and the mapping relation between operators and instructions, the target calculation task is split into sub-tasks and distributed to a target calculation unit, and accurate matching of calculation unit resources and task types is achieved; and finally, generating an equipment executable instruction stream according to an allocation result to ensure that storage, instructions and computing unit resources of the computing equipment in the optimized memory are all adaptive to large model computing requirements, so that the utilization rate of the equipment resources can be improved.

A method, a non-transitory computer-readable medium, and a system for optimizing storage in a cloud computing environment that comprises receiving, by a processor, a persistent volume claim (PVC) for storage by a workload; automatically providing, by the processor, a storage class to the PVC for storage provisioning; dynamically creating, by the processor, a persistent volume (PV) based on the storage class; and automatically binding, by the processor, the PV to the PVC, wherein the PV is formatted with a B-tree file system (Btrfs).

The invention relates to the technical field of data security, in particular to a data security storage method and system in big data transaction. According to the method, physical characteristics such as the number of data blocks, the number of key segments and the number of deleted entries are extracted, and hardware reference parameters and environment correction factors are combined, so that file volume is mapped to CPU estimation time consumption; the method comprises the following steps: constructing a virtual scheduler in a memory and simulating task allocation of a background thread pool to predict the latest completion time of a system and a thread load unbalance ratio; a suggested delay coefficient is generated based on the prediction result, and a lockless hibernation operation is performed before the write request acquires the mutex lock resource. According to the method, the hidden overhead of the computing power of the CPU is analyzed, the task allocation is simulated by means of the virtual scheduler, the thread load state is pre-judged in advance, and flow control is implemented before the storage engine mutual exclusion lock is obtained by the write-in request through lock-free delay execution, so that the deadlock risk is reduced, and the stability and throughput capacity of the system in a compliance encryption scene are improved.

Systems and methods are provided for data migration including cloning a multiple-logical unit number (LUN) volume into a plurality of cloned single-LUN volumes on a first storage node of a computing system; creating a plurality of new volumes on a second storage node of the computing system based at least in part on the plurality of cloned single-LUN volumes; selectively copying snapshot data from the plurality of cloned single-LUN volumes to the plurality of new volumes; establishing snapshot mirror relationships between the plurality of cloned single-LUN volumes and the plurality of new volumes; synchronizing the plurality of cloned single-LUN volumes and the plurality of new volumes; and performing a migration of data logical interface failovers (LIFs) from the plurality of cloned single-LUN volumes to the plurality of new volumes.

A computing device comprising a storage device and processing circuitry may perform the techniques of this disclosure. The storage device may have a plurality of blocks forming a volume. The processing circuitry may obtain volume changed block tracking (CBT) information identifying one or more of the blocks storing updated data that has changed relative to a previous backup of the one or more blocks, and determine file mapping information identifying one or more blocks of the plurality of blocks that store file data associated with a file. The processing circuitry may also determine, based on the volume CBT information and the file mapping information, file system CBT information identifying whether at least one of the one or more blocks store file data associated with the file have changed, and initiate, based on the file system CBT information, a subsequent backup of at least a portion of the file data.

Systems and methods are provided for data migration including cloning a multiple-logical unit number (LUN) volume into a plurality of cloned single-LUN volumes on a first storage node of a computing system; creating a plurality of new volumes on a second storage node of the computing system based at least in part on the plurality of cloned single-LUN volumes; selectively copying snapshot data from the plurality of cloned single-LUN volumes to the plurality of new volumes; establishing snapshot mirror relationships between the plurality of cloned single-LUN volumes and the plurality of new volumes; synchronizing the plurality of cloned single-LUN volumes and the plurality of new volumes; and performing a migration of data logical interface failovers (LIFs) from the plurality of cloned single-LUN volumes to the plurality of new volumes.

A method facilitating soft deletion and restoration of clustered file systems includes, in response to receiving an instruction to temporarily disable a computing cluster, creating, by a system including at least one processor, snapshots of volumes associated with a virtual machine (VM) of the computing cluster; generating, by the system, mapping data representative of a mapping between snapshot identifiers, of the snapshots of the volumes associated with the VM, and an instance identifier of the VM; and, in response to the mapping data being determined to have been successfully generated, deleting, by the system, the VM.

The technology disclosed herein enables a higher-level process to perform storage volume management with knowledge of a physical storage backend underlying a storage volume. In a particular example, a method includes mounting a storage volume to a computing node of the computing nodes. The storage volume is stored in a storage pool of a plurality of underlying storage pools. The method further includes determining an identifier for the storage pool, receiving a request to duplicate the storage volume, and determining a second identifier for a second storage pool of the plurality of underlying storage pools to which the storage volume will be duplicated. When the second identifier matches the identifier, creating a clone of the storage volume rather than copying the storage volume to the second storage pool.

A method facilitating storage system repurposing with preservation of cluster configuration deltas and data in-place includes removing, by a node device including at least one processor, a volume associated with the node device from a file system of a computing cluster in which the node device operates in response to an operating system upgrade instruction being received by the node device; in response to determining that the volume has successfully been removed from the file system of the computing cluster, populating, by the node device, a partition of the volume with cluster configuration data representative of a configuration of the computing cluster; and booting, by the node device based on the cluster configuration data as stored on the partition of the volume, the node device from an operating system image installed on the volume.

The invention discloses a shuffle method, device and equipment for data shuffle and a cluster, and relates to the technical field of data analysis. In the method, when multiple pieces of data on multiple computational nodes need to be subjected to shuffle, each computational node reads partition data of the computational node from a storage space corresponding to an old partition and then directly writes the read partition data into a storage space corresponding to a new partition. And after the partition data of the other computing nodes are read from the storage space corresponding to the old partition, writing the read partition data into the disk of the computing node, and then transmitting the data in the disk to the other computing nodes. Therefore, the data volume written into the local disk can be reduced, so that the data writing efficiency of the disk is improved, the data needing to be distributed to different data partitions in the local disk can be reduced, the data reading efficiency of the disk is improved, and then the shuffle efficiency can be improved.

Systems, methods, and software are disclosed herein for identifying duplicate blocks of a storage system and deduplicating the storage system. In one example, a method of operating a computing device includes scanning first metadata of blocks of a container file of a virtual volume to generate a first log file including records of virtual volume block numbers (VVBNs) and fingerprints of the blocks; scanning second metadata of blocks of an active file system of the virtual volume to generate a second log file including records of VVBNs and file block numbers (FBNs) of the blocks; generating tuples based on merging the records of the first log file and the second records of the second log file according to the VVBNs; identifying duplications among the blocks based on the tuples; and deduplicating the blocks based on the duplications in the active file system identified based on the tuples.

In some examples, a system provides a representation of a storage topology including a plurality of levels of entities that store data in a computing environment, the plurality of levels of entities including a first level including storage volumes and a second level including entities that request storage of data in the storage volumes. A policy repository stores data protection policies for respective entities in the computing environment. The system determines, based on the representation of the storage topology, whether an overlap exists between a first data protection policy and any of the data protection policies in the policy repository. Based on determining that the overlap exists between the first data protection policy and a second data protection policy in the policy repository, the system initiates an action including making a change associated with the first and second data protection policies or making a change associated with an entity in the computing environment.

This invention provides a data management method, apparatus, medium, and computing device for MHA high availability architecture. The method includes identifying the master database and each slave database in the MHA high availability architecture; creating storage disks corresponding to the master and slave databases, wherein the storage disks are disk volumes that can be unmounted and mounted at any time; and storing the binlog logs of the master and slave databases to the corresponding storage disks. This invention achieves master-slave data synchronization by placing the binlogs of the master database and the standby databases during peer failover on external storage disks, which are mounted on the host via volumes, instead of storing them on their respective local disks. This truly achieves zero data loss and confirms the integrity of production data.

A method and computing device for write-back caching. A client write request to write new data to a selected file on a volume (for which the client has mounted the corresponding cache) may be received at a network module of a node and processed to generate a write request that can be forwarded to a disk module hosting the cache (at a same or different node than received the client write request). The data is written to the cache and confirmation of the write is sent to the client. Accumulated data in the cache is written back to the volume (hosted by a different node than the cache) when at least one of a cache file threshold or a cache threshold is met. These parameters are set to values that reduce write latency, increase throughput, and help ensure data consistency and resiliency.