47 results about "IOPS" patented technology

The invention relates to the technical field of computer storage, and particularly provides a large model training storage resource dynamic allocation method, device and system, and the method comprises the steps: collecting a workload index and a storage system state index of a large model training task; inputting the real-time monitoring data into a convolutional neural network (CNN) model, and outputting a feature identifier of a current training stage; inputting the real-time data and the stage identifier into a long short-term memory network LSTM model, and predicting future bandwidth, IOPS and storage space requirements; and generating a resource allocation scheme through multi-objective optimization according to a prediction result in combination with a system state, and executing load allocation, data layering and bandwidth reservation operations. According to the method, training stage perception and resource demand prediction are realized through cooperation of the CNN and the LSTM, and dynamic allocation and advanced scheduling of storage resources are realized, so that the resource utilization rate, the training efficiency and the system stability are improved.

The invention relates to the technical field of large model reasoning, in particular to a reasoning performance optimization method and system based on a large model, and the method comprises the following steps: initializing a reasoning performance optimization agent; collecting hardware environment indexes in real time, wherein the hardware environment indexes comprise a video memory utilization rate, a CPU (Central Processing Unit) exchange number, residual video card resources, storage IOPS (Input / Output Per Second) and network throughput; the method has the beneficial effects that related statistical indexes, including model types, model weight file total volume, model average sequence length, the number of tokens per second output by the model, first token time of the model, a display card list occupied by the model, the size of a KVcache block, the size of a KVcache sliding window and the like, of each model in a service system are comprehensively collected and analyzed; the system performance is comprehensively evaluated, and the defects that in the prior art, performance evaluation is not comprehensive, and a real-time monitoring mechanism for key indexes such as memory occupation and network bandwidth is lacked are overcome.

The invention relates to the technical field of data storage, and discloses a solid state disk compatibility and reliability testing method which comprises the steps of initializing a testing environment, installing an operating system, compiling and executing a testing script and configuring a compiling environment. Basic information of the solid state disk is recognized and classified, an automatic script is written, the classified basic information of the solid state disk is created into file clusters of different sizes according to the automatic script, continuous / random reading and writing are conducted, and file integrity is verified; and performing performance test on the classified basic information of the solid state disk by using a Fio tool and recording performance indexes of the solid state disk, wherein the performance indexes comprise sequential read-write speed, random IOPS and response delay, compiling an automatic script, and simulating a database load to perform small block random read-write operation. According to the invention, a plurality of modules for basic function and compatibility test, performance test, power management and reliability test, safety characteristic test and the like are integrated into a complete and automatic test scheme.

Systems and methods for using object storage as a primary storage tier for a file system of a virtual storage system are provided. According to one embodiment, ephemeral storage of a host on which a virtual storage system is running in the cloud is used as an external cache in which all or some subset of file system metadata may be stored, for example, to boost read IOPS and reduce latency of the read path. In various examples, external cache policies may be defined to trigger adding to, updating, and / or re-warming the external cache to accelerate access to file system metadata, thereby making a virtual storage solution that maintains all file system data and file system metadata within object storage a suitable and cost-effective platform for certain types of workloads. In some example, the working set of a workload may also be kept in the external cache to further accelerate read performance of the virtual storage system.

This invention discloses an edge-side out-of-order write reordering method for consumer-grade partitioned storage, belonging to the field of computer storage systems and flash memory management technology. Addressing the problem of degraded multi-threaded parallel write performance in consumer-grade partitioned storage due to kernel-forced sequential writes, and the insufficient capacity of the edge-side SRAM write buffer to meet large-scale out-of-order reordering requirements, this invention proposes a dynamic processing method with a two-layer reordering architecture. By dividing the high-performance SLC flash memory region in the device into page-level SLC and zone-level SLC as an extended write buffer, when processing write requests, the continuity and temperature label of the data in the SRAM buffer are judged; if the data is continuous and hot data, a direct flushing strategy is executed to the zone-level SLC; if out-of-order data occurs and the buffer is insufficient, a degradation strategy is executed to temporarily store the data in the page-level SLC; and continuity is dynamically restored when subsequent data is supplemented. This invention overcomes the limitations of small edge-side caches while releasing the parallel write potential of the device, significantly improving multi-threaded write IOPS.

The invention provides a state self-repairing method and device for a thick backup volume in distributed storage, and relates to the technical field of distributed storage system state self-repairing technology.The method comprises the steps that metadata state information of the thick backup volume is obtained, and whether the volume is in a formatted state or not is judged; if the volume is in the formatting state, performing layered random sampling verification on the physical data of the volume, and identifying that the volume is a false abnormal volume or a true abnormal volume; if the volume is the false abnormal volume, only updating the metadata state into a formatting completion state; if the volume is a true abnormal volume, determining a breakpoint by adopting a binary search method; the method comprises the steps of collecting IOPS and bandwidth data of a cluster in real time, dynamically calculating a formatting quota, pausing formatting operation and storing current breakpoint coordinates when a cluster load exceeds a preset threshold value, and recovering the formatting operation from the stored breakpoint coordinates when the load is reduced to be below the threshold value. By means of the method and device, separation of metadata and physical data is effectively avoided, the repairing efficiency is remarkably improved, and storage resource consumption is reduced.

Embodiments of the present application disclose a kind of SSD equipment, the steady-state detection device, method, medium and program product of SSD equipment, the device includes: acquisition unit, acquisition unit and determination unit, wherein acquisition unit collects the IOPS data of pre-set time period, obtains multiple IOPS data;Acquisition unit is divided into multiple time periods according to time sequence in pre-set time period, obtains the IOPS data corresponding to multiple time periods from multiple IOPS data, obtains multiple IOPS data sets;Determination unit determines the average IOPS of each time period in multiple time periods according to multiple IOPS data sets, obtains multiple average IOPS values;Determine current SSD steady-state evaluation value according to multiple average IOPS values;When current SSD steady-state evaluation value is greater than pre-set threshold, determine that SSD is in steady-state data state.

The application discloses a kind of block storage performance automation test method, system, storage medium based on FIO, it is related to cloud computing and storage performance test technical field, it is applied to the cloud platform environment including management node and multiple test nodes.By running environment deployment script on management node, automatically complete test node IP management, SSH password-free configuration, firewall processing and according to X86 / ARM architecture adaptive deployment FIO;Run test execution script, precheck is carried out to test disk and time synchronization, start FIO service on multiple nodes according to preset task queue, in turn complete the stress test of covering 4K / 32K / 128K / 1M block size and random, order, 7:3 mixed read-write scene;Run result arrangement script, automatically parse FIO log, extract and uniformly output the IOPS, bandwidth and latency index of each scene.The method reduces manual operation while improving the consistency and result reliability of block storage performance test.

A storage node, a storage cluster and a method for achieving accelerating data access thereof are provided. The storage cluster includes a plurality of storage nodes. The method is performed by a processor of each of the storage nodes in the storage cluster. Each of the storage nodes includes the processor and a plurality of storage devices. The storage devices have at least two different input / output operations per second (IOPS) speeds. The method evaluates each storage device according to the IOPS speed thereof to perform an optimized sorting of the storage devices, and then migrates metadata previously stored in non-optimized storage devices to optimized storage devices so as to accelerate data access. The method further migrates the metadata stored in an optimized storage device when the IOPS speed of the optimized storage device has dropped to a threshold value.

A system is described. The system includes a processing resource and a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the processing resource to collect telemetry data of a distributed storage system associated with a client device, monitor a first set of the IOPS values, select a first IOPS value in the first set of the IOPS values as a highest IOPS value, determine whether the first IOPS value is unequal to a current Max-IOPS parameter value and adjust the Max-IOPS parameter value to be equal to the first IOPS value upon a determination that the first IOPS value is unequal to the current Max-IOPS parameter value.

A shared file system performance test method, tool, device and storage medium, comprising: obtaining the startup parameters of the test node in the shared file system to start the test program to perform write operation and shift operation on the target file system; when the write file flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory composed of year, serial day and hour; when the shift flag is 1, the data file is shifted from the write directory to the shift target directory by directly calculating the file path and file name in the memory; the success and failure times and time consumption of writing and shifting in the write operation and the shift operation are recorded, the test result is generated and output, the technical problem that the related technology generally starts from IOPS and cannot truly reflect the business application performance is solved, the performance test data truly reflecting the business scenario is generated, and the IOPS utilization rate is improved.

The invention relates to the field of flash memory storage control, in particular to a delay monitoring device based on streaming timestamp sampling. Comprising a timestamp acquisition module which is arranged in the SSD controller and is used for acquiring a timestamp on an IO path; the delay calculation and classification module is used for calculating the collected timestamps to obtain sub-path delays; the streaming statistical unit is used for calculating average delay, maximum delay and minimum delay in real time and supporting generation of delay distribution percentile data; and the threshold value alarm and PMU linkage module is used for configuring a delay threshold value by firmware, triggering the threshold value alarm and PMU linkage module when the I / O delay exceeds the threshold value, giving an alarm and storing execution snapshots of N instructions before and after the abnormal I / O. On the premise that the area and the power consumption of the SSD controller are not obviously increased, delay monitoring with the precision of ten nanoseconds is achieved. The influence of the device on the IOPS performance is lower than 1%, and high-reliability delay diagnosis and QoS guarantee can be provided in a data center scene.

Data storage devices with an open-ended protocol can be limited in terms of input-output operations per second (IOPS) due to time spent on stop-transmission clean-up activities. A data storage device is disclosed that can increase IOPS by using command length estimation logic that stores / remembers the command length of a previous write / read command and restricts the next open-ended read / write command to the predicted / remembered length. This increases IOPS by eliminating a need to perform a stop-transmission clean-up activity.

A method of optimizing the scheduling of storage resource for a server used in cloud computing obtains a first input / output operations per seconds (IOPS) value of a virtual disk in an instance. A warning message is generated when the first IOPS value is larger than a predefined threshold value, which represents an overloaded state of the virtual disk. An predicted IOPS value is obtained and the predicted IOPS value is set as the maximum IOPS value for limiting the IOPS value of the virtual disk, thus other virtual disks are not affected by a virtual disk being overloaded, and storing efficiency of the server is improved. A server and a computer readable storage medium applying the method are also disclosed.

The invention discloses a 6G full cloudization-oriented multi-dimensional virtualization resource intelligent allocation method and system, and belongs to the technical field of artificial intelligence, and the method is implemented by the following steps: multi-dimensional resource perception: collecting bandwidth, GPU utilization rate and computing power in real time, storing IOPS and generating feature vectors; resource demand grading: dividing the business into multiple levels of demands based on a clustering algorithm; rAG + MCP intelligent decision making: connecting a 6G full-clouding heterogeneous computing power virtualization platform through a retrieval enhancement generation technology in combination with MCP, and dynamically reasoning an optimal resource allocation strategy; 6G full-clouding heterogeneous computing power platform dynamic and static resource allocation: binding high-level demands to straight-through hardware resources, and elastically allocating low-level demands to virtualized resources; and continuous learning optimization: finely adjusting the RAG knowledge ability according to real-time feedback. According to the method, the resource scheduling real-time performance can be improved, the edge computing resource utilization rate is improved, and intelligent scheduling of application granularity is realized.

Data storage devices with an open protocol may be limited in terms of input / output operations per second (IOPS) due to the time taken to stop the transmission cleanup activity. A data storage device is disclosed that may increase IOPS by using command length estimation logic that stores / memorizes a command length of a previous write / read command and limits a next open read / write command to a predicted / memorized length. This increases IOPS by eliminating the need to perform a stop transmission cleaning activity.

The invention relates to the technical field of intelligent resource scheduling, in particular to an intelligent microhost virtualization resource scheduling method and system.The method comprises the following steps that a demand vector of each virtual machine in a to-be-scheduled virtual machine set is obtained, mirror image metadata is extracted to determine the architecture and instruction set demand of a central processing unit, and the demand of the central processing unit is determined; and analyzing the configuration parameters to obtain a memory demand, a central processing unit core number demand, a storage per-second input and output operation number demand and a network bandwidth demand. According to the method, virtual machine mirror image metadata and configuration parameters are extracted, a demand vector covering a CPU architecture, an instruction set, a memory, a core number, IOPS and network bandwidth is constructed, meanwhile, hardware information and real-time load data of a heterogeneous micro host are collected, a quantitative resource portrait set containing capacity and availability is generated, and the capacity and availability of the heterogeneous micro host are improved. And dynamic quantitative alignment between the resource demand and the capability is realized.

The application relates to the technical field of distributed storage, and discloses a cache data flushing flow control method and device, equipment and a storage medium, which comprises the following steps: acquiring the maximum service bandwidth and the maximum IOPS that can be carried by an HDD device at the same time; before each service thread and / or flushing thread writes data to the HDD device, calculating the bandwidth resource amount and the IOPS resource amount required by the current write operation; judging whether the bandwidth resource amount required is not more than the maximum service bandwidth and whether the IOPS resource amount required is not more than the maximum IOPS; if yes, executing the write operation and releasing the occupied bandwidth resource and IOPS resource after the write operation is executed; and if no, waiting for the release of the bandwidth resource and the IOPS resource. In this way, the HDD disk service bandwidth and IOPS can be simply and effectively controlled, the disk performance can be fully utilized, and the flushing performance of the cache data of an ICFS distributed layered storage system can be effectively improved.

The invention discloses a hard disk garbage collection method and device. The method comprises the following steps of: dividing a to-be-recycled storage object into a plurality of continuous intervals with fixed sizes, and determining starting and ending intervals of to-be-migrated data according to the offset and length of the to-be-migrated data; dividing a reading scene into a single-interval scene, a cross-double-interval scene and a cross-multi-interval scene based on the interval position relationship: in the single-interval scene, if the interval is cached, directly copying data, otherwise, when the reading IOPS does not reach the upper limit, reading complete interval data to the cache; executing the same logic on each interval in a cross-double-interval scene; directly reading data in a cross-multi-interval scene and converting IOPS concurrency according to an actual length; and migrating the valid data to the new storage object and releasing the original object space. According to the method, the number of hard disk read requests is effectively reduced through an interval pre-reading mechanism, the addressing overhead of a mechanical disk magnetic head is reduced, meanwhile, the IOPS flow control conflict is relieved through cache hit, and the garbage collection efficiency and cluster capacity stability of a storage system are remarkably improved.

The application discloses a hard disk garbage collection method and device. The method comprises the following steps: dividing a storage object to be collected into a plurality of fixed-size continuous intervals, determining the starting and ending intervals of the data to be migrated according to the offset and length of the data to be migrated; dividing a reading scene into three categories, i.e., a single-interval, a cross-double-interval and a cross-multiple-interval, based on the interval position relationship; in the single-interval scene, if the interval has been cached, the data is directly copied, otherwise, the complete interval data is read to the cache when the reading IOPS does not reach the upper limit; the same logic is executed for each interval in the cross-double-interval scene; in the cross-multiple-interval scene, the data is directly read and the IOPS concurrency is converted according to the actual length; and the effective data is migrated to a new storage object and the original object space is released. The application effectively reduces the number of hard disk reading requests, reduces the mechanical disk head addressing overhead, simultaneously reduces the IOPS flow control conflict by using cache hits, and significantly improves the garbage collection efficiency of the storage system and the stability of the cluster capacity.

The application relates to the technical field of storage array systems, in particular to a protocol adaptive reconfigurable storage array system, a method and a storage device, the system comprising a dynamic recombination configuration module, based on monitoring of storage access modes and protocol requests, the read-write operation frequency and the data block size of each storage unit are recorded, the storage unit matrix is partitioned, and the IOPS value and the data throughput of the partition are calculated. In the application, the read-write frequency, the data block size and the protocol type to which the I / O instruction belongs of the storage unit are monitored in real time, the row and column structures of the storage unit matrix are dynamically recombined, and the FPGA is used to load the NVMe or PCIe protocol stack in real time, the limitation of the traditional fixed interface and array structure is broken, the delay caused by the working mode switching is reduced, the storage system can instantaneously respond to different nature data access requests, and the agile adaptability at the hardware level is realized.

Aspects of the present disclosure relate to optimizing atomic operations in storage arrays by selectively enabling fabric-less mode based on device access patterns. In embodiments, whether multi-path software is enabled is determined, and device-level access control mappings are analyzed to identify candidates for fabric-less mode operation. For devices without multi-path software, a fabric-less mode is enabled when mapped to a single board, allowing atomic operations via software locks instead of fabric hardware. When multi-path software is enabled, the embodiments either leverage ALUA states to identify optimized paths or employ time series analysis using ARIMA models to predict IO access patterns. Fabric-less mode is activated when forecasting indicates localized access patterns, significantly improving response time and IOPS by avoiding fabric bandwidth constraints and hardware vendor limitations. The embodiments enable efficient atomic operations through CPU access to local memory, eliminating unnecessary fabric operations while maintaining data consistency across the storage array.

The application relates to the technical field of data storage, and discloses a compatibility and reliability test method for a solid state disk, which comprises the following steps: initializing a test environment, installing an operating system, writing and executing a test script, and configuring a compiling environment; recognizing basic information of the solid state disk and classifying the solid state disk, writing an automatic script, creating file clusters of different sizes, performing continuous / random read-write tests, and checking file integrity according to the classified basic information of the solid state disk; using an Fio tool to perform performance tests on the classified basic information of the solid state disk and recording performance indexes of the solid state disk, including sequential read-write speed, random IOPS and response delay, writing an automatic script, and simulating a database load to perform small random read-write operations. The application integrates multiple modules, such as basic function and compatibility tests, performance tests, power management and reliability tests, and security characteristic tests, into a complete and automatic test scheme.

The invention relates to the field of solid-state storage, in particular to a back-end loopback test method for a solid-state storage. According to the method, the implementation of the loopback test interface is separated from the main business logic of the upper layer, so that the normal firmware operation mode can be switched to the loopback test mode only by properly modifying the initialization step and replacing the callback function of the related register operation of the flash memory controller on the basis of reserving the back-end processing flow architecture; the reusability and compatibility of codes are improved, and the code maintenance difficulty and cost in the subsequent firmware development process are reduced. A round-robin queue and a bitmap mapping table which are realized by software are used as containers for simulating idle, trigger and completion of register behaviors of a flash memory controller, the time for calling the operation interfaces is fixed and predictable, and the upper limit of the computing power of a CPU (Central Processing Unit) in an SSD (Solid State Disk) main control chip can be estimated, so that a relatively strong guiding effect is achieved on analysis and improvement of performance indexes such as 4K random read command IOPS (Input / Output Operations Per Second) and the like.

The embodiment of the invention provides a monitoring method and device, equipment, a storage medium and a program product, and relates to the field of financial science and technology or other related fields. The method comprises the following steps: determining a mapping relationship between a storage logic unit LUN and a virtual machine according to configuration data of a storage system; in the running process of the storage system, performance data of the LUN are obtained, and the performance data comprise at least one of input and output operation times IOPS, throughput, time delay and processor utilization rate. And under the condition that the target performance data located outside the preset performance interval exists, determining an abnormal virtual machine according to the LUN corresponding to the target performance data and the mapping relationship, and triggering a fault response strategy. According to the method provided by the invention, the efficiency and accuracy of detecting the abnormality of the virtual machine in the storage system are improved.

A method for adjusting random write performance, an electronic device, and a storage medium are provided. The method includes: obtaining a remaining space of a target module of a hard drive, and performing balancing processing on a storage unit of the remaining space; calculating a quantity of data frames of each storage block in the storage unit based on a balancing processing result to obtain a first data set; performing convergence processing on the first data set to obtain a second data set; and calculating Input / Output Operations Per Second (IOPS) of the hard drive based on the second data set, and performing write arbitration adjustment on the IOPS of the hard drive.

The invention discloses a method for remotely analyzing disk performance indexes of a Linux server by Python, which comprises the following steps of: establishing SSH (Secure Shell) secure connection through a paramiko library, and configuring automatic host key verification, two-factor authentication and overtime parameters; executing optimized iostat and vmstat commands in an SSH channel, and collecting original data through environment variable standardization and regular expression filtering; performing structured conversion on the data, and generating a standardized data set after unifying units; indexes such as IOPS and MBPS are dynamically calculated through time sequence analysis; the result is output through a multi-protocol interface after being packaged, dynamic threshold warning, trend prediction and a Bokeh-based visual instrument panel are integrated, and remote, safe and intelligent monitoring and analysis of the disk performance are achieved.

The invention relates to an NVMe multi-queue-based SSD memory IO parallel processing method, device and equipment, and the method comprises the steps: obtaining an IO request sent by a host, and distributing the IO request into a plurality of parallel SQ queues; obtaining all SQ queues, and calculating weight information of each SQ queue from a global perspective; and allocating channel resources of the SSD memory for each SQ queue based on own weight information so as to process each SQ processing queue in parallel. By constructing a multi-queue parallel processing framework, introducing intelligent calculation of global weight and implementing dynamic resource allocation based on the weight, the core bottleneck of a traditional single-queue framework in an NVMe SSD high-concurrency application scene is systematically solved, the final effect is that the hardware parallel potential of the NVMe SSD is released to the maximum extent, the high-concurrency I / O performance (bandwidth and IOPS) is remarkably improved, and the performance of the NVMe SSD is improved. Delay fluctuation is reduced, and service quality is guaranteed.

The invention discloses a solid state disk automatic test tool and a test method thereof.The solid state disk automatic test tool comprises a hard disk automatic test module and a test data collection module, the hard disk automatic test module can be started by executing a specific script in a Linux environment, command line parameter dynamic input is supported so as to flexibly configure core parameters such as test duration, turns and IO modes, and the test data collection module can be started by executing the specific script in the Linux environment; and an interactive command is also provided to realize pause, continuation and termination operations of the test. The test data collection module can automatically collect data such as test process states, command line parameter input records, real-time throughput, IOPS, delay and the like, and fully records original test logs; meanwhile, the data are filtered and analyzed, invalid data are removed, and key data are reserved and displayed in a visual mode, for example, a chart or a report is generated. The invention belongs to the technical field of computer hardware testing, and aims to solve the problems of low efficiency, poor accuracy, single function, insufficient compatibility and the like of a traditional solid state disk testing method.

One or more embodiments of the specification provide an erasure code-based encoding method, a distributed system, a device and a storage medium. The method comprises: obtaining K data blocks of to-be-processed data; dividing the K data blocks into L groups of data groups; wherein each group of data groups comprises at least K encoding parameters corresponding to the K data blocks respectively; wherein K and L are integers greater than 1; for each group of data groups, using at least part of the data blocks in other groups of data groups and the encoding parameters corresponding to the at least part of the data blocks in the group of data groups to generate a partial check block of the group of data groups; wherein the data groups and the partial check blocks are stored in data nodes of the distributed system. The embodiment guarantees the security of user data while reducing the amplification of IOPS required for data recovery when data is lost or abnormal.