37 results about "Volume (computing)" patented technology

In a data storage subsystem providing data storage to host processors, a process of configuration defines a subset of the data storage that each host may access. A vector specification is a convenient mechanism for specifying a set of storage volumes that a host may access. For example, for each host processor, there is stored in memory of the data storage subsystem a list of contiguous ranges or vectors of the storage volumes that the host may access. To determine whether or not a specified logical volume number is included in the vector, a mudulus of the stride of the vector is computed from the difference between the address of the specified logical volume and the beginning address of the vector, and the modulus is compared to zero. To provide a mapping between logical unit numbers specified by the host and the logical volumes, a contiguous range of logical unit numbers may also be specified for each contiguous range or vector of storage volumes. The logical volume number is computed from a specified logical unit number by computing a difference between the specified logical unit number and the beginning logical unit number, multiplying the difference by the stride of the vector to produce a product, and adding the product to the beginning address of the vector.

The invention discloses a method for realizing file backup for a running virtual machine, and relates to the technical field of data backup in cloud computing. The method comprises the following steps: logging into a management platform service node to configure a backup strategy, and saving in a database; reading the backup strategy from the database; checking the backup strategy; if the backup strategy is triggered, creating a snapshot volume for a logical volume where disk files needing to be backed up of the virtual machine are positioned; creating a novel logical volume on a shared storage; analyzing EXT file system data in the snapshot volume, acquiring the physical sector distribution of the files needing to be backed up on the snapshot volume, reading file data needing to be backed up from the snapshot volume according to the physical sector distribution, and writing into the new logical volume; after writing, deleting the created snapshot volume. By adopting the method, file backup operation of the running virtual machine can be realized in a cloud data environment, and backup operation can be finished when the files are locked; the method is easy and convenient to operate and manage, and has high efficiency.

Described is a method and system by which storage reports are generated from a volume snapshot set rather than the live volume or volumes, wherein a volume snapshot set comprises a representation or copy of one or more volume at a single point-in-time. By scanning the snapshot, a consistent file system image is obtained. Scanning may take place by enumerating a volume's directories of files, or, when available, by accessing a file system metadata of file information (e.g., a master file table) separately maintained on the volume. With some (e.g., hardware-based) snapshot technologies, the snapshot can be transported to another computing system for scanning by that other computing system, thereby avoiding burdening a live system's resources when scanning. Accurate and consistent storage reports are thus obtained at a single point in time, independent of the number of volumes being scanned.

The invention provides a method and a device for quickly verifying the consistency of multi-duplicate storage. The method includes S1, judging data writing volumes of head files when systems are in idle states, creating snapshot files by the aid of I / O (input / output) redirection technologies when the volumes of data written into the head files reach 30% of the volumes of integral spare files at least, utilizing new head files with continuously written data as null comparison sparse files, computing hash values of the snapshot files and storing the hash values of the snapshot files as extended attributes; S2, carrying out snapshot operation on the head files once at intervals of preset time and comparing hash values of the new head files to the hash values of the snapshot files when the consistency of multi-duplicate data need to be timely verified. The method and the device have the advantages that characteristics of distributed block storage are combined with one another, the method for quickly detecting the consistency of multiple duplicates on the basis of snapshot technologies is proposed, hash value computation tasks are divided to be carried out in multiple time slots, accordingly, influence on normal reading and writing tasks of the systems can be prevented, and the multi-duplicate storage consistency verification speeds can be increased.

A file system executing on a computing system is configured to efficiently manage and store data on a tiered storage volume using hybrid media that includes a shingled magnetic recording (SMR) data store, such as an SMR disk drive, and a randomly-writable data store, such as a conventional magnetic recording (CMR) disk or solid-state drive (SSD) using flash memory. Write operations in the SMR diskare performed in a sequential manner to optimize storage density on the disk. The file system utilizes logic incorporated in an allocator that monitors I / O operations on the computing system, for example, write requests from applications executing on the system - to determine if a request is associated with file data or metadata. If the request is for metadata, then the allocator allocates spaceon the SSD / CMR for the metadata, and if the request is for file data, then space is allocated on the SMR disk.

A method, computer program product, and computer system for creating, by a computing device, a child leaf of a snapshot associated with a parent leaf of a volume. At least one entry associated with one of the parent leaf and the child leaf may be added to a delta log. Only delta changes are written to the child leaf. One of a first destaging technique and a second destaging technique may be used for the child leaf.

The invention discloses a memory snapshot method and a device of a cloud computing platform virtual machine based on a storage system, and a computer readable storage medium. A memory synchronizationinterface matched with each virtual machine operating system is constructed in advance; wherein the memory synchronization interface is used for synchronizing memory data at a preset moment to a corresponding storage disk, and the method can comprise the following steps: when a target virtual machine snapshot creation operation instruction issued by a cloud platform is received, obtaining storageresources which are mounted on a target virtual machine and comprise a disk and a storage volume corresponding to the disk on a storage system; sending an instruction of interrupting the storage resource read-write request, and meanwhile, calling a memory synchronization interface matched with the target virtual machine to synchronize the memory data into the disk; based on the corresponding relation between the disk and the storage system, calling the snapshot interface of the storage system to carry out snapshot operation on the disk, so that the memory snapshot of the virtual machine is efficiently realized on the basis of not occupying IO resources of the virtual machine and IO resources of a cloud platform.

Virtual computing instance data that are stored across multiple storage volumes are replicated in a manner such that the write order is maintained. The frequency of the replication is set so that the recovery point objective defined for the VM data can be satisfied. The replication includes the steps of determining a set of logical storage volumes across which the virtual computing instance issues dependent write IOs, issuing a first command to the virtual computing instance to block new IOs and to block receipt of IO acknowledgements, issuing a command to create replicas of all the logical storage volumes in the set, and then issuing a second command to the virtual computing instance to unblock new IOs and unblock receipt of IO acknowledgements.

The invention discloses a container-based distributed computing method and device, and the method comprises the following steps in a Shuffle stage: calling a bottom driver to initialize a persistent memory connected to a host machine, determining an equipment application mode for the persistent memory, and creating an area and a namespace on the host machine; creating a data volume with a file system for the persistent memory based on the region and the namespace, and mounting the data volume to a host machine to allow the container to access the file system through a container storage interface; and monitoring the Shuffle management interface, determining Shuffle data according to a dependency relationship between the elastic distributed data sets output by the management interface, and accessing the data volume from the container through the container storage interface so as to overwrite and / or cache the Shuffle data to the file system. According to the method, the processing amountof the Shuffle data can be reduced, the reading and writing cache speed of the Shuffle data in the container can be increased, and then the Shuffle efficiency of distributed computing is improved.

The embodiment of the invention discloses a method, device and equipment for creating a system disk in a cloud computing system and a storage medium, and the method comprises the steps: receiving a volume creation request for a to-be-created system disk in the cloud computing system, wherein the volume creation request comprises the identification of a mirror image file of the system disk; querying a block storage service used by the cloud computing system according to the identifier; when it is inquired that the mirror image file corresponding to the identifier exists in the block storage service, copying the mirror image file to the first target volume of the block storage service. In this way, it can be avoided that the network bandwidth is occupied on a large scale in the system disk creation process of the cloud computing system, and then the system disk creation efficiency is improved.

The invention relates to a parallel sequence comparison method and device based on load balancing, computer equipment and a storage medium. The method comprises the following steps: starting a process on each node of a cluster, equally dividing a to-be-compared file according to the data volume of the file, executing sequence comparison of equal data volume on each process, obtaining a pointer starting position and a pointer ending position of a read file corresponding to the process according to the serial number of the process in the cluster, and starting a plurality of threads in the process at the same time, performing parallel comparison on the sequences in the read file, and finally merging sequence comparison results output by each process to obtain a parallel sequence comparison result of load balance realized on the cluster. By means of the method, computing resources can be fully utilized, on one hand, load balancing is achieved among multiple nodes, on the other hand, parallel sequence alignment is achieved through multiple processes and multiple threads, and the operation efficiency of the algorithm is improved.

The invention discloses a tgt-based cluster handle management method, system and device and a computer readable storage medium, and the method comprises the steps: adding one to a cluster reference count in a cluster handle structure body when the cluster handle structure body of a storage cluster is created; when the storage pool structure body of the storage pool corresponding to the LUN creation information is created, adding one to the reference count of the storage pool in the storage pool structure body; when the volume structure body of the volume corresponding to the LUN creation information is created, adding one to the volume reference count in the volume structure body. According to the invention, the corresponding reference count is added in the structure body and used for recording the number of LUNs corresponding to a storage cluster, a storage pool and a volume; the number of LUNs is counted through reference counting, the LUNs are managed through the handle informationand the context information, one tgt thread can manage all the LUNs only by establishing one cluster handle structure body, the load capacity of a cluster is improved, and consumption of computing resources is reduced.

The invention relates to the technical field of distributed storage, in particular to a flow control method and device for a distributed storage system, computing equipment and a computer readable storage medium. The flow control method comprises the following steps: acquiring the priority and burst coefficient of a storage pool of the distributed storage system, the total IOPS quota of the read / write operation times per second, and the priority, IOPS quota limit value and IOPS real-time quota of each volume in the storage pool; comparing the sum of the IOPS real-time quotas of the volumes with the IOPS total quota of the storage pool to obtain a comparison result; and updating the IOPS real-time quota of each volume according to the comparison result and a first preset rule. According tothe embodiments of the invention, the IOPS quota of the storage pool is dynamically allocated according to the priority of the storage pool of the distributed storage system, the priority of each volume in the storage pool and the burst coefficient of the storage pool, so that the flow control efficiency and stability of the distributed storage system are improved.

The invention discloses a bare equipment disaster tolerance method. The method comprises the following steps of obtaining a virtual machine configuration file in a metadata management volume; determining a first identifier of the disk according to path information of the disk recorded in the virtual machine configuration file; obtaining a second identifier of the disk in the metadata management volume according to the first identifier; and starting the corresponding virtual machine according to the data in the disk corresponding to the second identifier. The invention further discloses a system, computer equipment and a readable storage medium. The method is advantaged in that through the disaster recovery scheme after the computing equipment fails, disaster recovery can further be carriedout on the virtual machine of the user even if only the storage equipment is left, so data security of the user is ensured, and a high-performance disk can be provided for the client virtual machine.

The invention discloses a file data feature extraction method and apparatus, and a computing device. The method comprises the steps that data volume information of file data is read, the file data isfragmented according to the preset fragmented data volume, and the starting position and the stopping position of each file data fragment are determined; starting a corresponding number of threads according to the number of the fragments, allocating a to-be-read file data fragment to each thread, so that each thread starts to read line by line from the initial position of the allocated file data fragment and performs eigenvalue calculation until the reading position exceeds the cut-off position of the file data fragment; and summarizing calculation results of all threads to obtain a feature output result of the file data. According to the method, large file data does not need to be completely loaded into a memory, and the file reading speed is greatly increased in a segmented reading mode;moreover, the reading strategy provided by the invention does not occupy too much server resources, so that the calculation efficiency is greatly improved.

The invention discloses a theater data accurate query optimization method based on HBase. The theater data accurate query optimization method comprises the following steps: 1) accessing the HBase by using C + + through Thrift; (2) generating a file writing interface by using Thrift, and putting an HBase. Thritf file in an HBase source code under a main directory of a root user; (3) accessing the HBase in the program; the method comprises the following steps: starting the service of HBASETHRIF2service at a server side, calling an API (Application Program Interface) to write an interface program by referring to a function definition in a generated file, applying for accessing a database from the server by a client side, and calling a Thrift function to open the connection of the database; 4) adopting a PARTITION algorithm to carry out direct connection operation, and reducing data processing time; and 5) performing computation-intensive and large-data-volume operation by using a Spark distributed computing platform, and realizing efficient query of data through the iterative computation advantage of the Spark platform.

A method, computer program product, and computer system for receiving, at a computing device, an I / O request directed to a compressed data portion of a storage system. It may be determined whether the I / O request includes one of a first portion of information and a second portion of information. An address of the compressed data portion may be obtained via downward mapping if the I / O request includes the first portion of information. The address of the compressed data portion may be obtained via upward mapping if the I / O request includes the second portion of information. The I / O request may be executed at the compressed data portion.

The invention discloses a capacity expansion method of a Ceph cluster OSD acceleration disk, and belongs to the field of cloud computing, the technical problem to be solved by the invention is how toredeploy a Ceph cluster without redeploying the Ceph cluster. The space size of the block. Db or the block. Wal of the deployed OSD is expanded; according to the technical scheme, the method comprisesthe steps that according to the management mode of a Ceph cluster OSD, a disk partition mode or a logical volume mode is selected for accelerating disk expansion, OSD service is restarted after the accelerating disk expansion, the state of the Ceph cluster is detected, and it is ensured that the OSD service adopts a new acceleration space; wherein when the OSD acceleration equipment adopts a diskpartition mode, the space size of an acceleration disk is expanded by newly establishing a partition, migrating old data and expanding a BlueFS file system mode; when OSD acceleration equipment adopts a logical volume mode, the space size of an acceleration disk is expanded by expanding a volume group, expanding a logical volume and expanding a BlueFS file system mode.

The invention discloses a method for dynamically creating a local storage volume, relates to the technical field of cloud computing, and solves the technical problem that a plurality of storage pools cannot be supported to perform local storage according to different nodes and different storage performance. The method comprises the following steps: step 1, creating a local storage provider to provide local storage using each node for a k8s user, it is guaranteed that a pod copy is operated on each of part of or all of the nodes; step 2, a service account (ServiceAccount), a role binding (ClusterRole Binding) and a role specification (ClusterRole) of the provisor are generated, and the service account (ServiceAccount), the role binding (ClusterRole) and the role specification (ClusterRole) of the provisor are generated; step 3, creating a hostpath storage class StorageClass to dynamically generate a storage volume, configuring a provider parameter of the storage class to be example.io / hostpath, and configuring a VolumeBindingMode binding mode of the storage class to be a WaitForFirstConsumer, and configuring a VolumeBindingMode binding mode of the hostpath to be a VolumeBindingMode binding mode of the hostpath to be a VolumeBindingMode binding mode of the hostpath to be a VolumeBindingMode binding mode of the hostpath to be a VolumeBindingMode binding mode of the hostpath; and step 4, creating a test-pvc and binding a hostpath storage class, and requesting a specific storage class by the pvc through the Storage ClassName. The invention further discloses a local storage volume dynamic creation device.

The invention relates to the field of computer storage, and discloses a data backup method and device, and a medium, and the method comprises the steps: obtaining the number information of to-be-backed-up data volumes, so as to determine the number of threads in a thread pool according to the number of the to-be-backed-up data volumes, establish the thread pool, and reduce the waste of computing resources through enabling the number of the threads to be positively correlated with the number information. If the current time meets the backup time of any data volume to be backed up, calling a thread to obtain the working state of a target data volume; and if the working state meets a preset condition, calling a thread to back up the target data volume. The threads in the thread pool are called by the timer to execute the working state and the backup task, so that a plurality of threads can execute the backup work at the same time, and data damage caused by untimely backup is prevented. According to the method, the multiple threads in the thread pool are called through the timer to concurrently execute the data backup work, so that the data volume backup efficiency is improved, and data loss caused by untimely backup is prevented.

The invention discloses a bare device disaster recovery method, comprising the following steps: acquiring a virtual machine configuration file in a metadata management volume; determining the first identifier of the disk according to the path information of the disk recorded in the virtual machine configuration file character; acquire the second identifier of the disk in the metadata management volume according to the first identifier; start the corresponding virtual machine according to the data in the disk corresponding to the second identifier. The invention also discloses a system, computer equipment and readable storage medium. The solution proposed by the invention can adopt a disaster recovery solution after a computing device fails, and even if only storage devices are left, the user's virtual machine can be recovered from the disaster, which not only ensures the security of user data, but also provides high-performance disks for the customer's virtual machine.

The invention relates to a novel software and hardware collaborative RAID improvement method, and belongs to the field of hard disk arrays. The RAID software computing module and the monitoring moduleare added to the computer host end, the data path 2 and the counting module are added to the RAID board card end, the current CPU load is monitored through the monitoring module, and when the CPU load is low, the RAID software computing module is used for RAID computing processing, and the computing result is transmitted to the cache module through the data path 2; in addition, whether the data volume in the cache module is larger than or equal to the total data volume (L + 1) or not is calculated through the counting module, if yes, calculation is stopped, and repeated calculation is prevented. Due to the fact that calculation is conducted through the data calculation module and the RAID software calculation module of the RAID board card at the same time, the RAID calculation processingspeed is increased, and remaining CPU resources are utilized. According to the method, the RAID board card is matched with the software, so that faster RAID performance is achieved, and CPU (Central Processing Unit) resource shortage of a computer end cannot be caused.

The present invention discloses a method for accessing a distributed storage system to a container. The method comprises: adding a distributed storage system to a computing node of a container and configuring the computing node to complete access of the distributed storage system to the computing node; creating a storage block and a logical volume group in the computing node, and mapping the storage block to a local disk device; converting the local disk device to a local physical volume, and adding the physical volume to the logic volume group; and performing operations on the logic volume group, and changing container parameters to achieve container distributed storage. According to the method disclosed by the present invention, a distributed storage system can be added to a computing node, relevant configuration is performed on the computing node, a storage block and a logical volume group are created in the computing node, the created storage block and logical volume group are operated, and finally the storage manner is changed into the distributed storage, so that data security and integrity can be improved. In addition, the present invention also provides an apparatus for accessing the distributed storage system into the container and a readable storage medium, and the effect is as described above.

The invention discloses an imported file checking method, a checking system and computing equipment. The method is executed in an operating system, a shared data volume and a container connected with the shared data volume are arranged on the operating system, and the method comprises the steps of: detecting whether a target file imported from the outside is newly added in the shared data volume or not through the container; if yes, checking the target file, generating a checking result file, and storing the checking result file into the shared data volume; determining whether the target file passes the check according to the check result file; and if the check is passed, storing the target file and the check result file in the shared data volume in a predetermined directory of the operating system. According to the technical scheme, the function of checking the externally imported file is achieved based on the operating system, and the safety and integrity of the file imported into the system from the outside are ensured.

A method, computer program product, and computer system for creating, by a computing device, a child leaf of a snapshot associated with a parent leaf of a volume. At least one entry associated with one of the parent leaf and the child leaf may be added to a delta log. Only delta changes are written to the child leaf. One of a first destaging technique and a second destaging technique may be used for the child leaf.

The invention provides a method and a device for quickly verifying the consistency of multi-duplicate storage. The method includes S1, judging data writing volumes of head files when systems are in idle states, creating snapshot files by the aid of I / O (input / output) redirection technologies when the volumes of data written into the head files reach 30% of the volumes of integral spare files at least, utilizing new head files with continuously written data as null comparison sparse files, computing hash values of the snapshot files and storing the hash values of the snapshot files as extended attributes; S2, carrying out snapshot operation on the head files once at intervals of preset time and comparing hash values of the new head files to the hash values of the snapshot files when the consistency of multi-duplicate data need to be timely verified. The method and the device have the advantages that characteristics of distributed block storage are combined with one another, the method for quickly detecting the consistency of multiple duplicates on the basis of snapshot technologies is proposed, hash value computation tasks are divided to be carried out in multiple time slots, accordingly, influence on normal reading and writing tasks of the systems can be prevented, and the multi-duplicate storage consistency verification speeds can be increased.

The invention discloses a method and a device for real-time summarization and interval summarization of data, and relates to the technical field of computers. One specific embodiment of the method comprises the following steps that a predetermined number of RegionServers based on HBase are deployed through an HDFS (Hadoop Distributed File System); and the RegionServers respectively carry out data summarization in parallel according to predetermined fields, and the summarized data of the RegionServers are sent to the client side for carrying out secondary data summarization. According to the embodiment, complete real-time processing and storage are reduced; storage and computing resources are placed on the same machine, and data network transmission is minimized, so that the summarization time is shortened; the resource expansion without perception of the program is realized. Support for the SQL is provided, so that the readability is better; and two modes of real-time summarization and interval summarization are provided to solve scenes with different data volumes.

The invention discloses a system and a method for realizing dynamic allocation of logical volumes and supporting dynamic migration of a virtual machine and relates to the field of cloud computing. Three mapping table cache modes including a write-back mode, a write-through mode and a migration mode are introduced. In the write-back mode, reading and writing of mapping tables are performed in caches, a kernel process regularly writes dirty mapping tables in a disk; in the write-through mode, data are acquired from the caches during reading of the mapping tables, during data writing, the mappingtables are written in the caches and the disk simultaneously, and each data writing operation can trigger refreshing of the dirty mapping tables to the disk; in the migration mode, the mapping tablesare read from a hard disk, the cache mode of the mapping tables is automatically switched to the write-back mode during data writing. The cache mode of the mapping tables is set as the write-throughmode before a source server triggers migration operation; a virtual disk of the virtual machine can be preloaded and the cache mode of the mapping tables is set as the migration mode before a target server triggers migration operation. By means of the system and the method, the HA (high available) problem of the virtual machine can be solved, and dynamic migration of the virtual machine can be realized.

The invention relates to the technical field of cloud computing, and discloses a cloud disk data backup method and device, electronic equipment and a readable storage medium. The method comprises the steps of obtaining a to-be-backed-up cloud disk and to-be-backed-up data corresponding to the to-be-backed-up cloud disk; detecting whether the to-be-backed-up data is valid; and when the to-be-backed-up data is valid, writing the to-be-backed-up data into the object storage corresponding to the to-be-backed-up cloud disk for data backup. By implementing the method and the device, the backup of invalid data is avoided, the space occupation of the invalid data in object storage is reduced, the backup data volume is reduced, and the backup time is saved, so that the backup efficiency is improved.

The invention relates to the field of cloud computing, in particular to a cloud loading method of a physical server, which comprises the following steps: a cloud migration step of migrating data in a disk space of a source end to a target end; and a service switching step of, after the cloud migration step is completed, carrying out service switching between a source end and a target end. The method further comprises an incremental migration step executed together with the cloud migration step, and incremental data generated by a source end is migrated to a target end in multiple times from the time point when the cloud migration step is started. According to the automatic cloud loading method of the physical server, the user can perform migration operation while not closing the service, so that the service of the user is ensured to be affected to the minimum extent. When a system service is online, the whole disk is copied through full-volume migration at a fixed point, full-volume data is transmitted through a network, then the data generated in the full-volume migration process is copied into the target VM again in an increment mode, and the consistency of the data is ensured.