29 results about "IOPS" patented technology

The invention relates to the technical field of computer storage, and particularly provides a large model training storage resource dynamic allocation method, device and system, and the method comprises the steps: collecting a workload index and a storage system state index of a large model training task; inputting the real-time monitoring data into a convolutional neural network (CNN) model, and outputting a feature identifier of a current training stage; inputting the real-time data and the stage identifier into a long short-term memory network LSTM model, and predicting future bandwidth, IOPS and storage space requirements; and generating a resource allocation scheme through multi-objective optimization according to a prediction result in combination with a system state, and executing load allocation, data layering and bandwidth reservation operations. According to the method, training stage perception and resource demand prediction are realized through cooperation of the CNN and the LSTM, and dynamic allocation and advanced scheduling of storage resources are realized, so that the resource utilization rate, the training efficiency and the system stability are improved.

This invention discloses an edge-side out-of-order write reordering method for consumer-grade partitioned storage, belonging to the field of computer storage systems and flash memory management technology. Addressing the problem of degraded multi-threaded parallel write performance in consumer-grade partitioned storage due to kernel-forced sequential writes, and the insufficient capacity of the edge-side SRAM write buffer to meet large-scale out-of-order reordering requirements, this invention proposes a dynamic processing method with a two-layer reordering architecture. By dividing the high-performance SLC flash memory region in the device into page-level SLC and zone-level SLC as an extended write buffer, when processing write requests, the continuity and temperature label of the data in the SRAM buffer are judged; if the data is continuous and hot data, a direct flushing strategy is executed to the zone-level SLC; if out-of-order data occurs and the buffer is insufficient, a degradation strategy is executed to temporarily store the data in the page-level SLC; and continuity is dynamically restored when subsequent data is supplemented. This invention overcomes the limitations of small edge-side caches while releasing the parallel write potential of the device, significantly improving multi-threaded write IOPS.

The invention provides a state self-repairing method and device for a thick backup volume in distributed storage, and relates to the technical field of distributed storage system state self-repairing technology.The method comprises the steps that metadata state information of the thick backup volume is obtained, and whether the volume is in a formatted state or not is judged; if the volume is in the formatting state, performing layered random sampling verification on the physical data of the volume, and identifying that the volume is a false abnormal volume or a true abnormal volume; if the volume is the false abnormal volume, only updating the metadata state into a formatting completion state; if the volume is a true abnormal volume, determining a breakpoint by adopting a binary search method; the method comprises the steps of collecting IOPS and bandwidth data of a cluster in real time, dynamically calculating a formatting quota, pausing formatting operation and storing current breakpoint coordinates when a cluster load exceeds a preset threshold value, and recovering the formatting operation from the stored breakpoint coordinates when the load is reduced to be below the threshold value. By means of the method and device, separation of metadata and physical data is effectively avoided, the repairing efficiency is remarkably improved, and storage resource consumption is reduced.

Embodiments of the present application disclose a kind of SSD equipment, the steady-state detection device, method, medium and program product of SSD equipment, the device includes: acquisition unit, acquisition unit and determination unit, wherein acquisition unit collects the IOPS data of pre-set time period, obtains multiple IOPS data;Acquisition unit is divided into multiple time periods according to time sequence in pre-set time period, obtains the IOPS data corresponding to multiple time periods from multiple IOPS data, obtains multiple IOPS data sets;Determination unit determines the average IOPS of each time period in multiple time periods according to multiple IOPS data sets, obtains multiple average IOPS values;Determine current SSD steady-state evaluation value according to multiple average IOPS values;When current SSD steady-state evaluation value is greater than pre-set threshold, determine that SSD is in steady-state data state.

The application discloses a kind of block storage performance automation test method, system, storage medium based on FIO, it is related to cloud computing and storage performance test technical field, it is applied to the cloud platform environment including management node and multiple test nodes.By running environment deployment script on management node, automatically complete test node IP management, SSH password-free configuration, firewall processing and according to X86 / ARM architecture adaptive deployment FIO;Run test execution script, precheck is carried out to test disk and time synchronization, start FIO service on multiple nodes according to preset task queue, in turn complete the stress test of covering 4K / 32K / 128K / 1M block size and random, order, 7:3 mixed read-write scene;Run result arrangement script, automatically parse FIO log, extract and uniformly output the IOPS, bandwidth and latency index of each scene.The method reduces manual operation while improving the consistency and result reliability of block storage performance test.

A storage node, a storage cluster and a method for achieving accelerating data access thereof are provided. The storage cluster includes a plurality of storage nodes. The method is performed by a processor of each of the storage nodes in the storage cluster. Each of the storage nodes includes the processor and a plurality of storage devices. The storage devices have at least two different input / output operations per second (IOPS) speeds. The method evaluates each storage device according to the IOPS speed thereof to perform an optimized sorting of the storage devices, and then migrates metadata previously stored in non-optimized storage devices to optimized storage devices so as to accelerate data access. The method further migrates the metadata stored in an optimized storage device when the IOPS speed of the optimized storage device has dropped to a threshold value.

A system is described. The system includes a processing resource and a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the processing resource to collect telemetry data of a distributed storage system associated with a client device, monitor a first set of the IOPS values, select a first IOPS value in the first set of the IOPS values as a highest IOPS value, determine whether the first IOPS value is unequal to a current Max-IOPS parameter value and adjust the Max-IOPS parameter value to be equal to the first IOPS value upon a determination that the first IOPS value is unequal to the current Max-IOPS parameter value.

A shared file system performance test method, tool, device and storage medium, comprising: obtaining the startup parameters of the test node in the shared file system to start the test program to perform write operation and shift operation on the target file system; when the write file flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory composed of year, serial day and hour; when the shift flag is 1, the data file is shifted from the write directory to the shift target directory by directly calculating the file path and file name in the memory; the success and failure times and time consumption of writing and shifting in the write operation and the shift operation are recorded, the test result is generated and output, the technical problem that the related technology generally starts from IOPS and cannot truly reflect the business application performance is solved, the performance test data truly reflecting the business scenario is generated, and the IOPS utilization rate is improved.

The invention relates to the field of flash memory storage control, in particular to a delay monitoring device based on streaming timestamp sampling. Comprising a timestamp acquisition module which is arranged in the SSD controller and is used for acquiring a timestamp on an IO path; the delay calculation and classification module is used for calculating the collected timestamps to obtain sub-path delays; the streaming statistical unit is used for calculating average delay, maximum delay and minimum delay in real time and supporting generation of delay distribution percentile data; and the threshold value alarm and PMU linkage module is used for configuring a delay threshold value by firmware, triggering the threshold value alarm and PMU linkage module when the I / O delay exceeds the threshold value, giving an alarm and storing execution snapshots of N instructions before and after the abnormal I / O. On the premise that the area and the power consumption of the SSD controller are not obviously increased, delay monitoring with the precision of ten nanoseconds is achieved. The influence of the device on the IOPS performance is lower than 1%, and high-reliability delay diagnosis and QoS guarantee can be provided in a data center scene.

The invention discloses a hard disk garbage collection method and device. The method comprises the following steps of: dividing a to-be-recycled storage object into a plurality of continuous intervals with fixed sizes, and determining starting and ending intervals of to-be-migrated data according to the offset and length of the to-be-migrated data; dividing a reading scene into a single-interval scene, a cross-double-interval scene and a cross-multi-interval scene based on the interval position relationship: in the single-interval scene, if the interval is cached, directly copying data, otherwise, when the reading IOPS does not reach the upper limit, reading complete interval data to the cache; executing the same logic on each interval in a cross-double-interval scene; directly reading data in a cross-multi-interval scene and converting IOPS concurrency according to an actual length; and migrating the valid data to the new storage object and releasing the original object space. According to the method, the number of hard disk read requests is effectively reduced through an interval pre-reading mechanism, the addressing overhead of a mechanical disk magnetic head is reduced, meanwhile, the IOPS flow control conflict is relieved through cache hit, and the garbage collection efficiency and cluster capacity stability of a storage system are remarkably improved.

The application discloses a hard disk garbage collection method and device. The method comprises the following steps: dividing a storage object to be collected into a plurality of fixed-size continuous intervals, determining the starting and ending intervals of the data to be migrated according to the offset and length of the data to be migrated; dividing a reading scene into three categories, i.e., a single-interval, a cross-double-interval and a cross-multiple-interval, based on the interval position relationship; in the single-interval scene, if the interval has been cached, the data is directly copied, otherwise, the complete interval data is read to the cache when the reading IOPS does not reach the upper limit; the same logic is executed for each interval in the cross-double-interval scene; in the cross-multiple-interval scene, the data is directly read and the IOPS concurrency is converted according to the actual length; and the effective data is migrated to a new storage object and the original object space is released. The application effectively reduces the number of hard disk reading requests, reduces the mechanical disk head addressing overhead, simultaneously reduces the IOPS flow control conflict by using cache hits, and significantly improves the garbage collection efficiency of the storage system and the stability of the cluster capacity.

The application relates to the technical field of storage array systems, in particular to a protocol adaptive reconfigurable storage array system, a method and a storage device, the system comprising a dynamic recombination configuration module, based on monitoring of storage access modes and protocol requests, the read-write operation frequency and the data block size of each storage unit are recorded, the storage unit matrix is partitioned, and the IOPS value and the data throughput of the partition are calculated. In the application, the read-write frequency, the data block size and the protocol type to which the I / O instruction belongs of the storage unit are monitored in real time, the row and column structures of the storage unit matrix are dynamically recombined, and the FPGA is used to load the NVMe or PCIe protocol stack in real time, the limitation of the traditional fixed interface and array structure is broken, the delay caused by the working mode switching is reduced, the storage system can instantaneously respond to different nature data access requests, and the agile adaptability at the hardware level is realized.

Aspects of the present disclosure relate to optimizing atomic operations in storage arrays by selectively enabling fabric-less mode based on device access patterns. In embodiments, whether multi-path software is enabled is determined, and device-level access control mappings are analyzed to identify candidates for fabric-less mode operation. For devices without multi-path software, a fabric-less mode is enabled when mapped to a single board, allowing atomic operations via software locks instead of fabric hardware. When multi-path software is enabled, the embodiments either leverage ALUA states to identify optimized paths or employ time series analysis using ARIMA models to predict IO access patterns. Fabric-less mode is activated when forecasting indicates localized access patterns, significantly improving response time and IOPS by avoiding fabric bandwidth constraints and hardware vendor limitations. The embodiments enable efficient atomic operations through CPU access to local memory, eliminating unnecessary fabric operations while maintaining data consistency across the storage array.

The application relates to the technical field of data storage, and discloses a compatibility and reliability test method for a solid state disk, which comprises the following steps: initializing a test environment, installing an operating system, writing and executing a test script, and configuring a compiling environment; recognizing basic information of the solid state disk and classifying the solid state disk, writing an automatic script, creating file clusters of different sizes, performing continuous / random read-write tests, and checking file integrity according to the classified basic information of the solid state disk; using an Fio tool to perform performance tests on the classified basic information of the solid state disk and recording performance indexes of the solid state disk, including sequential read-write speed, random IOPS and response delay, writing an automatic script, and simulating a database load to perform small random read-write operations. The application integrates multiple modules, such as basic function and compatibility tests, performance tests, power management and reliability tests, and security characteristic tests, into a complete and automatic test scheme.

The embodiment of the invention provides a monitoring method and device, equipment, a storage medium and a program product, and relates to the field of financial science and technology or other related fields. The method comprises the following steps: determining a mapping relationship between a storage logic unit LUN and a virtual machine according to configuration data of a storage system; in the running process of the storage system, performance data of the LUN are obtained, and the performance data comprise at least one of input and output operation times IOPS, throughput, time delay and processor utilization rate. And under the condition that the target performance data located outside the preset performance interval exists, determining an abnormal virtual machine according to the LUN corresponding to the target performance data and the mapping relationship, and triggering a fault response strategy. According to the method provided by the invention, the efficiency and accuracy of detecting the abnormality of the virtual machine in the storage system are improved.

The invention relates to an NVMe multi-queue-based SSD memory IO parallel processing method, device and equipment, and the method comprises the steps: obtaining an IO request sent by a host, and distributing the IO request into a plurality of parallel SQ queues; obtaining all SQ queues, and calculating weight information of each SQ queue from a global perspective; and allocating channel resources of the SSD memory for each SQ queue based on own weight information so as to process each SQ processing queue in parallel. By constructing a multi-queue parallel processing framework, introducing intelligent calculation of global weight and implementing dynamic resource allocation based on the weight, the core bottleneck of a traditional single-queue framework in an NVMe SSD high-concurrency application scene is systematically solved, the final effect is that the hardware parallel potential of the NVMe SSD is released to the maximum extent, the high-concurrency I / O performance (bandwidth and IOPS) is remarkably improved, and the performance of the NVMe SSD is improved. Delay fluctuation is reduced, and service quality is guaranteed.

One or more embodiments of the specification provide an erasure code-based encoding method, a distributed system, a device and a storage medium. The method comprises: obtaining K data blocks of to-be-processed data; dividing the K data blocks into L groups of data groups; wherein each group of data groups comprises at least K encoding parameters corresponding to the K data blocks respectively; wherein K and L are integers greater than 1; for each group of data groups, using at least part of the data blocks in other groups of data groups and the encoding parameters corresponding to the at least part of the data blocks in the group of data groups to generate a partial check block of the group of data groups; wherein the data groups and the partial check blocks are stored in data nodes of the distributed system. The embodiment guarantees the security of user data while reducing the amplification of IOPS required for data recovery when data is lost or abnormal.

This application discloses an access method, system, and storage medium, belonging to the storage field. The method includes: a computing node receiving a first access request from a first tenant, the first access request corresponding to m first input / output (IO) requests, where m is an integer greater than or equal to 1; initiating the m first IO requests to a storage node based on the first tenant's actual IOPS, the number of first IO requests initiated per second being less than or equal to the first tenant's actual IOPS, the m first IO requests being used to request the storage node to perform m IO operations, and the m IO operations being used to access the storage medium; responding to the m first IO requests and returning a first response result to the computing node; and the computing node returning the access result of the first access request to the first tenant based on the first response result. This application can improve the versatility of computing nodes.

The invention discloses a data processing method, electronic equipment, a storage medium and a computer program product, and the method comprises the steps that a first parameter of double token buckets is configured in a first virtual machine, the double token buckets comprise the first token bucket and the second token bucket, the first token bucket is used for controlling the number of read-write times per second IOPS of the first virtual machine, and the second token bucket is used for controlling the number of read-write times per second IOPS of the second virtual machine; the second token bucket is used for controlling the bandwidth of the first virtual machine; analyzing the IOPS and the bandwidth of the first virtual machine to obtain a first analysis result; adjusting a first parameter of the double token buckets according to the first analysis result; and according to the adjusted first parameter, adjusting the IOPS and the bandwidth of the first virtual machine. According to the method and the device, combined control of two dimensions of the bandwidth of the IOPS can be realized, and more comprehensive resource optimization is realized.

The invention discloses an FIO-based block storage performance automatic test method and system, and a storage medium, relates to the technical field of cloud computing and storage performance test, and is applied to a cloud platform environment comprising a management node and a plurality of test nodes. By operating an environment deployment script on a management node, test node IP management, SSH password-free configuration and firewall processing are automatically completed, and FIO is adaptively deployed according to an X86 / ARM architecture; running a test execution script, pre-checking a test disk and time synchronization, uniformly starting an FIO service on multiple nodes according to a preset task queue, and sequentially completing pressure tests covering the sizes of 4K / 32K / 128K / 1M blocks and random, sequential and 7: 3 mixed read-write scenes; and running a result arrangement script, automatically analyzing the FIO log, and extracting and uniformly outputting IOPS, bandwidth and time delay indexes of each scene. According to the method, the consistency and the result reliability of the block storage performance test are improved while the manual operation is reduced.

A shared file system performance test method, tool and device and a storage medium, the method comprising: acquiring a startup parameter of a test node in a shared file system to start a test program to execute a write operation and a shift operation on a target file system; when the file writing mark is 1, generating a data file containing pseudo random number initialization, and writing the data file into a subdirectory formed by years, sequence days and hours; when the shift mark is 1, shifting the data file from the write-in directory to a shift target directory by directly calculating and generating a file path and a file name in a memory; and recording the success and failure times and time consumption of writing and shifting in the writing operation and shifting operation processes, generating a test result and outputting the test result, thereby solving the technical problem that the related technology generally starts from IOPS and cannot truly reflect the business application performance condition, realizing the generation of the performance test data truly reflecting the business scene, and improving the performance test efficiency. And the IOPS utilization rate is improved.

Various example embodiments enhance random host read performance in a memory system, such as a solid-state drive (SSD), through optimized read command execution using independent read commands. With the intelligence to sense for host data pattern (e.g., continuous host read data pattern versus random host read data pattern), some example embodiments dynamically select and use independent read commands, in place of one or more multi-plane read commands, for random host reads, which can assist in higher input / output operations per second (IOPS) where single-plane read commands are aggregated into one or more multi-plane read commands.

One or more aspects of the present disclosure relate to managing data reduction in a storage system during burst workloads. In embodiments, input / output (IO) operations received by a storage array are monitored to identify a burst workload based on a predefined threshold of IO operations per second (IOPS). In addition, the compression of data corresponding to IO write operations in the burst workload is deferred during the identified burst workload by temporarily storing the data in an uncompressed format. Further, the data is compressed during a self-healing period after a delay period. For example, the delay period is after the identified burst workload.

This invention relates to the field of solid-state storage, specifically a back-end loopback testing method for solid-state storage. This method separates the implementation of the loopback testing interface from the main upper-layer business logic. This allows switching from normal firmware operation mode to loopback testing mode by only appropriately modifying the initialization steps and replacing the callback functions for related flash controller register operations, while retaining the back-end processing architecture. This improves code reusability and compatibility, and reduces the difficulty and cost of code maintenance during subsequent firmware development. A software-implemented circular queue and bitmap mapping table serve as containers to simulate the idle, triggering, and completion register behaviors of the flash controller. The time for calling these operation interfaces is fixed and predictable, allowing for the estimation of the upper limit of CPU computing power in the SSD controller chip. This provides strong guidance for the analysis and improvement of performance indicators such as 4K random read command IOPS.

The invention relates to the technical field of cloud storage, can be applied to business scenes such as financial science and technology and medical health, and discloses a data writing method, device and equipment based on a hybrid storage medium and a medium. The method comprises the steps that a client writing request is responded, and a data storage path containing at least two data storage nodes is determined; sending the to-be-written data to the first data storage node and writing the to-be-written data into a high-speed storage medium; sequentially transmitting the data to subsequent data storage nodes along the data storage path through the first data storage node, and completing writing; receiving write-in completion confirmation information returned by the subsequent data storage nodes step by step and feeding back the write-in completion confirmation information to the client; and controlling the data storage node to asynchronously write the data in the high-speed storage medium into the persistent storage medium. According to the method, the writing efficiency is improved through the high-speed storage medium, the data consistency is ensured through step-by-step confirmation, and the performance and the reliability are considered through asynchronous disk falling, so that the IOPS bottleneck of a mechanical hard disk is broken through, and the requirements of distributed storage for high efficiency and persistence are met.

The invention discloses a Ceph cluster performance tuning method and device, equipment, a medium and a product, and the method comprises the steps: taking a configuration parameter scheme of a Ceph cluster as an individual, and constructing an initialized population; constructing a fitness function according to the performance index of the Ceph cluster; wherein the performance indexes comprise at least one of the following items: read-write IOPS, throughput, read-write delay and error rate; calculating the fitness of each individual in the current population according to the fitness function; performing genetic manipulation on the current population to generate a new population; and judging whether an iteration termination condition is met, if so, outputting a target configuration parameter scheme of the Ceph cluster, and if not, performing fitness calculation by using the new population. By adopting the embodiment of the invention, the tuning efficiency can be improved, and the computing resource and time cost can be greatly reduced.

An example method for provisioning data volume for a virtual compute instance may include receiving a request to provision a data volume for a virtual compute instance. The request may specify a size of the data volume, a type of the data volume, and a first input / output operations per second (IOPS) value for the data volume. Further, the method may include determining a recommended IOPS value for the data volume by applying a logic to the specified size, the specified type, and the first IOPS value. Furthermore, the method may include provisioning the data volume for the virtual compute instance with the specified size, the specified type, and the recommended IOPS value.

The present application belongs to the field of storage. Disclosed are an access method and system, and a storage medium. The method comprises: a computing node receiving a first access request of a first tenant, wherein the first access request corresponds to m first input output (IO) requests, m being an integer greater than or equal to 1; on the basis of actual input output operations per second (IOPS) of the first tenant, initiating the m first IO requests to a storage node, wherein the number of first IO requests initiated per second is less than or equal to the actual IOPS of the first tenant, the m first IO requests are used for requesting that the storage node executes m IO operations, and the m IO operations are used for accessing a storage medium; responding to the m first IO requests, and returning a first response result to the computing node; and on the basis of the first response result, the computing node returning an access result of the first access request to the first tenant. The present application can improve the universality of a computing node.

Aspects of the present disclosure relate to optimizing atomic operations in storage arrays by selectively enabling fabric-less mode based on device access patterns. In embodiments, whether multi-path software is enabled is determined, and device-level access control mappings are analyzed to identify candidates for fabric-less mode operation. For devices without multi-path software, a fabric-less mode is enabled when mapped to a single board, allowing atomic operations via software locks instead of fabric hardware. When multi-path software is enabled, the embodiments either leverage ALUA states to identify optimized paths or employ time series analysis using ARIMA models to predict IO access patterns. Fabric-less mode is activated when forecasting indicates localized access patterns, significantly improving response time and IOPS by avoiding fabric bandwidth constraints and hardware vendor limitations. The embodiments enable efficient atomic operations through CPU access to local memory, eliminating unnecessary fabric operations while maintaining data consistency across the storage array.