80 results about "IOPS" patented technology

The invention relates to the technical field of computer storage, and particularly provides a large model training storage resource dynamic allocation method, device and system, and the method comprises the steps: collecting a workload index and a storage system state index of a large model training task; inputting the real-time monitoring data into a convolutional neural network (CNN) model, and outputting a feature identifier of a current training stage; inputting the real-time data and the stage identifier into a long short-term memory network LSTM model, and predicting future bandwidth, IOPS and storage space requirements; and generating a resource allocation scheme through multi-objective optimization according to a prediction result in combination with a system state, and executing load allocation, data layering and bandwidth reservation operations. According to the method, training stage perception and resource demand prediction are realized through cooperation of the CNN and the LSTM, and dynamic allocation and advanced scheduling of storage resources are realized, so that the resource utilization rate, the training efficiency and the system stability are improved.

A method, computer program product, and computing system for identifying a change in a capacity of a cloud-deployed storage system. A capacity ratio and a IOPS ratio are determined for the cloud-deployed storage system. A portion of a cloud-deployed storage device is modified based upon, at least in part, one or more of the capacity ratio and the IOPS ratio. The portion of the cloud-deployed storage device is mapped to a portion of a logical storage device.

The invention relates to the technical field of data processing, in particular to an enterprise data asset management method, system and device based on cloud data and a storage medium, an activity sensing engine is deployed to collect indexes such as data access frequency, business key weight and cross-department calling popularity, and the indexes are processed into multi-dimensional activity factors through a rule engine; and dividing hot, warm and cold grades after calculating scores. Hot data is controlled by a trusted domain and stored in a high-IOPS cloud disk, warm data generates role self-adaptive desensitization view retention standard cloud disk, and cold data starts full mask and manual approval transfer object storage. And if the cold data is overdue, starting compliance review and physically erasing the cold data. According to the method, dynamic collaboration of data value evaluation, security policy and storage scheduling is realized, traditional static management contradictions are solved, full-life-cycle intelligent management of data is achieved, the value utilization rate is improved, the resource cost is optimized, and compliance risks are reduced.

A computer-implemented method for optimizing input / output (I / O) operations per second (IOPS) of remote storage. The computer-implemented method includes receiving requests for read operations and write operations from users of an application, dynamically creating batches into which the requests can be aggregated, aggregating the requests into the batches and asynchronously passing each completed batch to the remote storage.

The invention relates to the technical field of large model reasoning, in particular to a reasoning performance optimization method and system based on a large model, and the method comprises the following steps: initializing a reasoning performance optimization agent; collecting hardware environment indexes in real time, wherein the hardware environment indexes comprise a video memory utilization rate, a CPU (Central Processing Unit) exchange number, residual video card resources, storage IOPS (Input / Output Per Second) and network throughput; the method has the beneficial effects that related statistical indexes, including model types, model weight file total volume, model average sequence length, the number of tokens per second output by the model, first token time of the model, a display card list occupied by the model, the size of a KVcache block, the size of a KVcache sliding window and the like, of each model in a service system are comprehensively collected and analyzed; the system performance is comprehensively evaluated, and the defects that in the prior art, performance evaluation is not comprehensive, and a real-time monitoring mechanism for key indexes such as memory occupation and network bandwidth is lacked are overcome.

Methods and systems for a networked storage system are provided. One method includes predicting an IOPS limit for a plurality of storage pools based on a maximum allowed latency of each storage pool, the maximum allowed latency determined from a relationship between the retrieved latency and a total number of IOPS from a resource data structure; identifying a storage pool whose utilization has reached a threshold value, the utilization based on a total number of IOPS directed towards the storage pool and a predicted IOPS limit; detecting a bully workload based on a numerical value determined from a total number of IOPS issued by the bully workload for the storage pool and a rising step function; and implementing a corrective action to reduce an impact of the bully workload on a victim workload.

The invention relates to the technical field of data storage, and discloses a solid state disk compatibility and reliability testing method which comprises the steps of initializing a testing environment, installing an operating system, compiling and executing a testing script and configuring a compiling environment. Basic information of the solid state disk is recognized and classified, an automatic script is written, the classified basic information of the solid state disk is created into file clusters of different sizes according to the automatic script, continuous / random reading and writing are conducted, and file integrity is verified; and performing performance test on the classified basic information of the solid state disk by using a Fio tool and recording performance indexes of the solid state disk, wherein the performance indexes comprise sequential read-write speed, random IOPS and response delay, compiling an automatic script, and simulating a database load to perform small block random read-write operation. According to the invention, a plurality of modules for basic function and compatibility test, performance test, power management and reliability test, safety characteristic test and the like are integrated into a complete and automatic test scheme.

The invention discloses a hard disk temperature prediction method and device, electronic equipment and a storage medium, and relates to the technical field of computer hardware management, and the method comprises the steps: obtaining a PCIe real-time bandwidth utilization rate, a hard disk IOPS and an environment parameter, inputting the PCIe real-time bandwidth utilization rate, the hard disk IOPS and the environment parameter into a pre-trained deep learning network prediction model, the initial temperature prediction result of the hard disk is obtained and then corrected, a target temperature prediction result of the hard disk is obtained, a temperature heat dissipation strategy of the hard disk is generated based on the target temperature prediction result, and the temperature of the hard disk is dynamically regulated and controlled according to the temperature heat dissipation strategy. The problem that the heat dissipation risk of the hard disk is increased due to the fact that the obtained hard disk temperature information is lagged and the read-write performance is poor is solved, the hard disk temperature is predicted through the PCIe bandwidth actual utilization rate and the hard disk IOPS, and therefore the hard disk temperature change in the future time period can be predicted in advance, and the heat dissipation safety of the hard disk is guaranteed.

The present invention discloses a data storage method and device based on address redirection. The method comprises receiving a write request, extracting a data block identifier, data offset, and data length from the write request, determining whether the write is a large block or a small block based on the data length, querying a metadata database based on the data block identifier and data offset, and determining whether the write is a new write operation or an overwrite write operation; obtaining the bandwidth, IOPS, and latency of the HDD device within the current sliding time window to determine the load of the HDD device; writing data based on the load of the HDD device; and persisting data distribution information and address space usage information to the metadata database to complete data storage. The present invention utilizes the characteristics of the disk and the current load to perform IO scheduling between the SSD device and the HDD device, and combines the address redirection and small block write aggregation characteristics of the storage system to improve the performance of the storage system.

Systems and methods for using object storage as a primary storage tier for a file system of a virtual storage system are provided. According to one embodiment, ephemeral storage of a host on which a virtual storage system is running in the cloud is used as an external cache in which all or some subset of file system metadata may be stored, for example, to boost read IOPS and reduce latency of the read path. In various examples, external cache policies may be defined to trigger adding to, updating, and / or re-warming the external cache to accelerate access to file system metadata, thereby making a virtual storage solution that maintains all file system data and file system metadata within object storage a suitable and cost-effective platform for certain types of workloads. In some example, the working set of a workload may also be kept in the external cache to further accelerate read performance of the virtual storage system.

This invention discloses an edge-side out-of-order write reordering method for consumer-grade partitioned storage, belonging to the field of computer storage systems and flash memory management technology. Addressing the problem of degraded multi-threaded parallel write performance in consumer-grade partitioned storage due to kernel-forced sequential writes, and the insufficient capacity of the edge-side SRAM write buffer to meet large-scale out-of-order reordering requirements, this invention proposes a dynamic processing method with a two-layer reordering architecture. By dividing the high-performance SLC flash memory region in the device into page-level SLC and zone-level SLC as an extended write buffer, when processing write requests, the continuity and temperature label of the data in the SRAM buffer are judged; if the data is continuous and hot data, a direct flushing strategy is executed to the zone-level SLC; if out-of-order data occurs and the buffer is insufficient, a degradation strategy is executed to temporarily store the data in the page-level SLC; and continuity is dynamically restored when subsequent data is supplemented. This invention overcomes the limitations of small edge-side caches while releasing the parallel write potential of the device, significantly improving multi-threaded write IOPS.

A method, computer program product, and computing system for defining a plurality of IO processing queues within a storage system coupled to a storage platform, wherein the plurality of IO processing queues have differing priority levels; receiving one or more IO requests on the storage system coupled to the storage platform; assigning the one or more IO requests to one or more of the plurality of IO processing queues based, at least in part, upon a priority level of the one or more IO requests; and servicing the plurality of IO processing queues in a distributed fashion.

The invention provides a state self-repairing method and device for a thick backup volume in distributed storage, and relates to the technical field of distributed storage system state self-repairing technology.The method comprises the steps that metadata state information of the thick backup volume is obtained, and whether the volume is in a formatted state or not is judged; if the volume is in the formatting state, performing layered random sampling verification on the physical data of the volume, and identifying that the volume is a false abnormal volume or a true abnormal volume; if the volume is the false abnormal volume, only updating the metadata state into a formatting completion state; if the volume is a true abnormal volume, determining a breakpoint by adopting a binary search method; the method comprises the steps of collecting IOPS and bandwidth data of a cluster in real time, dynamically calculating a formatting quota, pausing formatting operation and storing current breakpoint coordinates when a cluster load exceeds a preset threshold value, and recovering the formatting operation from the stored breakpoint coordinates when the load is reduced to be below the threshold value. By means of the method and device, separation of metadata and physical data is effectively avoided, the repairing efficiency is remarkably improved, and storage resource consumption is reduced.

An example method of input / output (IO) between system software executing in a computer and a storage device includes: sending, from the system software, IO commands to the storage device; handling, by the system software, IO completion messages from the storage device in an interrupt mode; determining, while processing the IO commands, a first value for a measure of IO operations per second (IOPs) of the storage device; determining, by a device driver of the system software, that a first condition is met, the first condition being that the measure of IOPs satisfies a threshold for enabling a polling mode; determining, by the device driver, that a second condition is met, the second condition being that a timer started in an interrupt mode has not expired; and maintaining, by the device driver, the interrupt mode in response to the second condition despite the first condition.

Embodiments of the present application disclose a kind of SSD equipment, the steady-state detection device, method, medium and program product of SSD equipment, the device includes: acquisition unit, acquisition unit and determination unit, wherein acquisition unit collects the IOPS data of pre-set time period, obtains multiple IOPS data;Acquisition unit is divided into multiple time periods according to time sequence in pre-set time period, obtains the IOPS data corresponding to multiple time periods from multiple IOPS data, obtains multiple IOPS data sets;Determination unit determines the average IOPS of each time period in multiple time periods according to multiple IOPS data sets, obtains multiple average IOPS values;Determine current SSD steady-state evaluation value according to multiple average IOPS values;When current SSD steady-state evaluation value is greater than pre-set threshold, determine that SSD is in steady-state data state.

The application discloses a kind of block storage performance automation test method, system, storage medium based on FIO, it is related to cloud computing and storage performance test technical field, it is applied to the cloud platform environment including management node and multiple test nodes.By running environment deployment script on management node, automatically complete test node IP management, SSH password-free configuration, firewall processing and according to X86 / ARM architecture adaptive deployment FIO;Run test execution script, precheck is carried out to test disk and time synchronization, start FIO service on multiple nodes according to preset task queue, in turn complete the stress test of covering 4K / 32K / 128K / 1M block size and random, order, 7:3 mixed read-write scene;Run result arrangement script, automatically parse FIO log, extract and uniformly output the IOPS, bandwidth and latency index of each scene.The method reduces manual operation while improving the consistency and result reliability of block storage performance test.

A storage node, a storage cluster and a method for achieving accelerating data access thereof are provided. The storage cluster includes a plurality of storage nodes. The method is performed by a processor of each of the storage nodes in the storage cluster. Each of the storage nodes includes the processor and a plurality of storage devices. The storage devices have at least two different input / output operations per second (IOPS) speeds. The method evaluates each storage device according to the IOPS speed thereof to perform an optimized sorting of the storage devices, and then migrates metadata previously stored in non-optimized storage devices to optimized storage devices so as to accelerate data access. The method further migrates the metadata stored in an optimized storage device when the IOPS speed of the optimized storage device has dropped to a threshold value.

A system is described. The system includes a processing resource and a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the processing resource to collect telemetry data of a distributed storage system associated with a client device, monitor a first set of the IOPS values, select a first IOPS value in the first set of the IOPS values as a highest IOPS value, determine whether the first IOPS value is unequal to a current Max-IOPS parameter value and adjust the Max-IOPS parameter value to be equal to the first IOPS value upon a determination that the first IOPS value is unequal to the current Max-IOPS parameter value.

The invention provides a space recovery method and device, a storage medium and electronic device.The space recovery method comprises the steps that for a target storage space of a storage chip in terminal equipment, the size of a reserved space in the target storage space is determined based on the data updating rate of the target storage space within a first preset time period; and based on the determined size of the reserved space, dividing the target storage space into a data space and the reserved space, and storing the data written into the target storage space through the data space. In response to the space recovery condition being reached, a space recovery operation is performed on the data space. According to the space recovery method provided by the embodiment of the invention, the host dynamically adjusts the size of the reserved space in the target storage space based on the data updating rate of the target storage space in the first preset time period, so that the space utilization rate of the storage space can be improved on the premise of reducing the limitation on IOPS of the host.

The present invention provides an SSD write performance regulation method, device, equipment and medium, which relates to the technical field of flash memory storage control. The method includes: determining a first running time according to SSD particle parameters; determining a second running time according to a preset IOPS threshold; determining a time slice coefficient according to the first running time and the second running time; determining the number M of garbage collection time slices and the number N of data writing time slices according to the time slice coefficient; judging whether it is necessary to allocate data writing time slices to a data writing module according to the number N of data writing time slices. If so, performing a data writing operation on the SSD through the data writing module; if not, judging whether it is necessary to allocate garbage collection time slices to a garbage collection module according to the number M of garbage collection time slices. If so, performing a garbage collection operation on the SSD through the garbage collection module. The present invention improves the minimum write performance of the SSD.

A shared file system performance test method, tool, device and storage medium, comprising: obtaining the startup parameters of the test node in the shared file system to start the test program to perform write operation and shift operation on the target file system; when the write file flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory composed of year, serial day and hour; when the shift flag is 1, the data file is shifted from the write directory to the shift target directory by directly calculating the file path and file name in the memory; the success and failure times and time consumption of writing and shifting in the write operation and the shift operation are recorded, the test result is generated and output, the technical problem that the related technology generally starts from IOPS and cannot truly reflect the business application performance is solved, the performance test data truly reflecting the business scenario is generated, and the IOPS utilization rate is improved.

The invention relates to the field of flash memory storage control, in particular to a delay monitoring device based on streaming timestamp sampling. Comprising a timestamp acquisition module which is arranged in the SSD controller and is used for acquiring a timestamp on an IO path; the delay calculation and classification module is used for calculating the collected timestamps to obtain sub-path delays; the streaming statistical unit is used for calculating average delay, maximum delay and minimum delay in real time and supporting generation of delay distribution percentile data; and the threshold value alarm and PMU linkage module is used for configuring a delay threshold value by firmware, triggering the threshold value alarm and PMU linkage module when the I / O delay exceeds the threshold value, giving an alarm and storing execution snapshots of N instructions before and after the abnormal I / O. On the premise that the area and the power consumption of the SSD controller are not obviously increased, delay monitoring with the precision of ten nanoseconds is achieved. The influence of the device on the IOPS performance is lower than 1%, and high-reliability delay diagnosis and QoS guarantee can be provided in a data center scene.

The embodiment of the invention discloses a fault early warning method for storage equipment, which is used for quickly and accurately evaluating the performance state of the storage equipment and the use reliability of the storage equipment. The method provided by the embodiment of the invention comprises the following steps: acquiring equipment state information of the storage equipment; the equipment state information at least comprises one of the following information: sequential write bandwidth SW, sequential read bandwidth SR, write amplification coefficient WA, bad Die number, total Die number and nominal bandwidth; determining reliability evaluation information of the storage device based on the device state information; the reliability evaluation information at least comprises one of the following information: actual read-write times per second IOPS and effective bandwidth; and determining whether the storage device is abnormal or not according to the reliability evaluation information. According to the embodiment of the invention, the reliability evaluation information such as the actual IOPS and / or the effective bandwidth is determined according to the state information of various devices, the actual load, the throughput capacity and possible bottlenecks of the devices are accurately detected, and performance reduction, potential faults or other abnormities of the devices are found in time.

An example method for provisioning data volume for a virtual compute instance may include receiving a request to provision a data volume for a virtual compute instance. The request may specify a size of the data volume, a type of the data volume, and a first input / output operations per second (IOPS) value for the data volume. Further, the method may include determining a recommended IOPS value for the data volume by applying a logic to the specified size, the specified type, and the first IOPS value. Furthermore, the method may include provisioning the data volume for the virtual compute instance with the specified size, the specified type, and the recommended IOPS value.

Data storage devices with an open-ended protocol can be limited in terms of input-output operations per second (IOPS) due to time spent on stop-transmission clean-up activities. A data storage device is disclosed that can increase IOPS by using command length estimation logic that stores / remembers the command length of a previous write / read command and restricts the next open-ended read / write command to the predicted / remembered length. This increases IOPS by eliminating a need to perform a stop-transmission clean-up activity.

Autonomous power control is provided on a storage system to enable service level maximum response time controls to be achieved, while also minimizing power consumption, by enabling service level minimum response times to be specified and enforced. A workload / CPU clock speed model is created for the storage system correlating a maximum number of IOPS that the storage system can process for different CPU clock speeds for each workload type. A service level agreement specifies a maximum storage system response time. A storage system minimum response time is also specified, that is used to identify a target CPU clock speed for the workload type being provided by a host. The CPU clock speed is lowered to reduce energy consumption by the storage system toward the target CPU clock speed, and storage system performance is monitored to ensure that the storage system complies with the storage system maximum response time.

A method of optimizing the scheduling of storage resource for a server used in cloud computing obtains a first input / output operations per seconds (IOPS) value of a virtual disk in an instance. A warning message is generated when the first IOPS value is larger than a predefined threshold value, which represents an overloaded state of the virtual disk. An predicted IOPS value is obtained and the predicted IOPS value is set as the maximum IOPS value for limiting the IOPS value of the virtual disk, thus other virtual disks are not affected by a virtual disk being overloaded, and storing efficiency of the server is improved. A server and a computer readable storage medium applying the method are also disclosed.

The present application discloses a data reading and writing method, apparatus, device, and medium, which includes: using a bilateral interface RDMA recv to receive a data reading and writing request sent by a storage client on a second node in the distributed storage system; obtaining a unilateral memory address from the data reading and writing request; performing an RDMA unilateral data reading and writing operation on the memory corresponding to the unilateral memory address in response to the data reading and writing request; using a bilateral interface RDMA send to send a reply message corresponding to the data reading and writing request to the storage client on the second node, so that the storage client on the second node receives the reply message using the bilateral interface RDMA recv, wherein the reply message is used to indicate that the operation corresponding to the data reading and writing request has been completed. This is compatible with existing network communication architectures, reduces data reading and writing latency, improves IOPS, and saves communication overhead.

The invention discloses a 6G full cloudization-oriented multi-dimensional virtualization resource intelligent allocation method and system, and belongs to the technical field of artificial intelligence, and the method is implemented by the following steps: multi-dimensional resource perception: collecting bandwidth, GPU utilization rate and computing power in real time, storing IOPS and generating feature vectors; resource demand grading: dividing the business into multiple levels of demands based on a clustering algorithm; rAG + MCP intelligent decision making: connecting a 6G full-clouding heterogeneous computing power virtualization platform through a retrieval enhancement generation technology in combination with MCP, and dynamically reasoning an optimal resource allocation strategy; 6G full-clouding heterogeneous computing power platform dynamic and static resource allocation: binding high-level demands to straight-through hardware resources, and elastically allocating low-level demands to virtualized resources; and continuous learning optimization: finely adjusting the RAG knowledge ability according to real-time feedback. According to the method, the resource scheduling real-time performance can be improved, the edge computing resource utilization rate is improved, and intelligent scheduling of application granularity is realized.

Data storage devices with an open protocol may be limited in terms of input / output operations per second (IOPS) due to the time taken to stop the transmission cleanup activity. A data storage device is disclosed that may increase IOPS by using command length estimation logic that stores / memorizes a command length of a previous write / read command and limits a next open read / write command to a predicted / memorized length. This increases IOPS by eliminating the need to perform a stop transmission cleaning activity.