Block storage caching method, device, equipment and storage medium

Abstract

The present application discloses a block storage cache method, device, equipment and storage medium, which relates to the field of data processing technology and is applied to a server, including: when using a block storage cache system to process input and output requests, monitoring the real-time performance data of the block storage cache system through a user-mode program built based on an extended Berkeley packet filter; the block storage cache system is a system containing configuration parameters pre-built based on a machine learning algorithm; the parameters characterizing the processing order of the input and output requests are calculated according to the real-time performance data to obtain the parameter value; based on the parameter value, the configuration parameters of the block storage cache system are adjusted through a file system interface to process the input and output requests using the adjusted block storage cache system. In this way, in the process of the block storage cache system processing input and output requests, the present application can dynamically adjust the relevant configuration parameters according to the real-time performance data, which can meet the needs of multiple business scenarios.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a block storage cache method, device, equipment and storage medium. Background Art

[0002] Cache acceleration technology adds high-speed media between the main storage device and the host, such as SSD (Solid State Disk or Solid State Drive), NVMe (Non-Volatile Memory Express, a high-performance, low-latency storage protocol, here refers to the hard disk of the corresponding protocol); combined with the cache acceleration system, it can significantly improve the system's IO (Input / Output) performance without changing the existing storage architecture. However, the parameters of the existing block storage cache acceleration system are complex, and it is difficult to use the same parameters to meet the ever-changing user needs. The test method based on fio (Flexible I / OTester, a disk performance testing tool) can only evaluate the average read and write performance of the block device and the preset optimal configuration parameters, and cannot quantitatively give the optimal parameters for configuring specific business request scenarios based on test data. In addition, the existing bio (Blocking Input / Output) series of tools provided in the bcc (BPF Compiler Collection, a toolkit for creating efficient kernel tracing and operation programs) toolkit based on ebpf (extended Berkeley Packet Filter) cannot observe the real-time read and write performance of block storage cache, and cannot dynamically configure the parameters of block devices based on real-time observation data to adapt to the storage performance requirements of different scenarios. Especially in virtualization and cloud computing scenarios, the block storage cache acceleration system can provide fast storage access for virtual machines or containers, improve overall system performance, and thus improve the user experience of cloud services.

[0003] It can be seen that how to achieve efficient and stable operation of block storage is a problem to be solved in this field. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a block storage cache method, device, equipment and storage medium, which can dynamically adjust relevant configuration parameters according to real-time performance data in the process of processing input and output requests by the block storage cache system, and can meet the needs of multiple business scenarios. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a block storage caching method, applied to a server, comprising:

[0006] When processing input and output requests using a block storage cache system, real-time performance data of the block storage cache system is monitored by a user-mode program built based on an extended Berkeley packet filter; the block storage cache system is a block storage cache system that includes configuration parameters and is pre-built based on a machine learning algorithm;

[0007] Calculating parameters representing the processing order of input and output requests according to the real-time performance data to obtain corresponding parameter values;

[0008] Based on the parameter value, the configuration parameters of the block storage cache system are adjusted through the file system interface, so as to use the adjusted block storage cache system to process related input and output requests.

[0009] Optionally, before monitoring the real-time performance data of the block storage cache system by a user-mode program constructed based on an extended Berkeley packet filter, the method further includes:

[0010] Fitting the configuration parameters of the block storage cache system based on a preset type of machine learning algorithm to determine initial configuration parameters that meet preset constraints;

[0011] The block storage cache system is configured with the initial configuration parameters so that when the block storage cache system is used to process input and output requests, real-time performance data of the block storage cache system is monitored by a user-mode program built based on an extended Berkeley packet filter.

[0012] Optionally, the machine learning algorithm based on a preset type performs fitting processing on the configuration parameters of the block storage cache system to determine the initial configuration parameters that meet the preset constraint conditions, including:

[0013] Based on a preset type of machine learning algorithm, a configuration parameter of the block storage cache system is fitted to obtain a number of corresponding candidate configuration parameters;

[0014] A weighted evaluation is performed on a number of the candidate configuration parameters using a preset formula, and initial configuration parameters that meet preset constraint conditions are determined according to corresponding evaluation indicators.

[0015] Optionally, monitoring the real-time performance data of the block storage cache system by using a user-mode program constructed based on an extended Berkeley packet filter includes:

[0016] The user-state program pre-built based on the extended Berkeley packet filter is compiled by a just-in-time compiler, so as to monitor the block storage cache system through the user-state program and obtain corresponding real-time performance data.

[0017] Optionally, the monitoring the block storage cache system through the user state program to obtain corresponding real-time performance data includes:

[0018] When a preset event occurs, the user state program is executed to monitor the block storage cache system through the user state program to obtain corresponding real-time performance data.

[0019] Optionally, calculating the parameter characterizing the processing order of the input and output requests according to the real-time performance data to obtain the corresponding parameter value includes:

[0020] According to the real-time performance data and based on the configuration parameters of the block storage cache system at the current moment, the parameters characterizing the processing order of the input and output requests are calculated to obtain corresponding parameter values.

[0021] Optionally, the method further includes:

[0022] Obtaining the operating status data of the block storage cache system through the file system interface;

[0023] If the running status data indicates that there is a state abnormality, a preset corresponding abnormality recovery strategy is executed through the file system interface, so as to process related input and output requests using the block storage cache system after the abnormality is recovered.

[0024] In a second aspect, the present application provides a block storage cache device, applied to a server, including:

[0025] A monitoring module, used for monitoring the real-time performance data of the block storage cache system through a user-mode program built based on an extended Berkeley packet filter when the block storage cache system is used to process input and output requests; the block storage cache system is a block storage cache system containing configuration parameters pre-built based on a machine learning algorithm;

[0026] A calculation module, used to calculate the parameters representing the processing order of the input and output requests according to the real-time performance data to obtain corresponding parameter values;

[0027] The adjustment module is used to adjust the configuration parameters of the block storage cache system through the file system interface based on the parameter value, so as to use the adjusted block storage cache system to process related input and output requests.

[0028] In a third aspect, the present application provides an electronic device, including:

[0029] Memory, used to store computer programs;

[0030] A processor is used to execute the computer program to implement the block storage cache method as described above.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the block storage cache method as described above when executed by a processor.

[0032] It can be seen that in this application, when the server uses the block storage cache system to process input and output requests, it can monitor the real-time performance data of the block storage cache system through a user-mode program built based on the extended Berkeley packet filter; the block storage cache system is a block storage cache system that contains configuration parameters pre-built based on a machine learning algorithm; then, according to the real-time performance data, the parameters that characterize the processing order of the input and output requests are calculated to obtain the corresponding parameter values; then, based on the parameter values, the configuration parameters of the block storage cache system are adjusted through the file system interface to use the adjusted block storage cache system to process the relevant input and output requests. In this way, in the process of the block storage cache system processing input and output requests, the present application can obtain the performance data of the block storage cache system, adjust the configuration parameters of the block storage cache system according to the actual processing situation, and realize dynamic adjustment of the performance of the block storage cache system, which can meet the storage performance requirements of various business scenarios and improve the performance of block storage cache acceleration. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1 A flow chart of a block storage caching method disclosed in this application;

[0035] Figure 2 A flowchart of a specific block storage caching method disclosed in this application;

[0036] Figure 3 A specific initial parameter setting flow chart disclosed in this application;

[0037] Figure 4 A specific real-time parameter adjustment flow chart disclosed in this application;

[0038] Figure 5 A block storage cache device structure diagram disclosed in this application;

[0039] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] See also Figure 1 As shown, an embodiment of the present invention discloses a block storage caching method, which is applied to a server and includes:

[0042] Step S11: When using the block storage cache system to process input and output requests, monitor the real-time performance data of the block storage cache system through a user-mode program built based on an extended Berkeley packet filter; the block storage cache system is a block storage cache system that includes configuration parameters and is pre-built based on a machine learning algorithm.

[0043] It is understandable that the block storage cache system is usually located between the server and the back-end storage device, which can intercept and cache IO requests; its core principle is to use high-speed media as a cache layer; and store frequently accessed data blocks in the cache. When the host requests data, the system first checks the cache layer. If it hits, it reads the data directly from the cache, reducing the number of times the back-end slow storage device is accessed. The block storage cache system in the embodiment of the present application is a system that contains configuration parameters pre-built based on a machine learning algorithm. When the block storage cache system processes input and output requests, the real-time performance data of the block storage cache system can be monitored through a user-mode program built based on ebpf.

[0044] In a specific embodiment, before monitoring the real-time performance data of the block storage cache system through a user-mode program built based on an extended Berkeley packet filter, the method may further include: fitting the configuration parameters of the block storage cache system based on a preset type of machine learning algorithm to determine the initial configuration parameters that meet the preset constraints; configuring the parameters of the block storage cache system using the initial configuration parameters, so that when the block storage cache system is used to process input and output requests, the real-time performance data of the block storage cache system is monitored through a user-mode program built based on an extended Berkeley packet filter. Specifically, based on the historical performance data of the block storage cache system, a plurality of machine learning algorithms may be used to fit the configuration parameters of the block storage cache system, and under the preset constraints, the adapted initial configuration parameters may be solved, so that the block storage cache system configured using the initial configuration parameters has the best storage performance in the business scenario corresponding to the constraints; subsequently, the block storage cache system configured with the initial configuration parameters may be used to process related input and output requests, and monitor the real-time performance data of the block storage cache system.

[0045] In a specific embodiment, the configuration parameters of the block storage cache system are fitted based on a preset type of machine learning algorithm to determine the initial configuration parameters that meet the preset constraints, which may include: fitting the configuration parameters of the block storage cache system based on a preset type of machine learning algorithm to obtain a corresponding number of candidate configuration parameters; performing weighted evaluation on a number of the candidate configuration parameters using a preset formula, and determining the initial configuration parameters that meet the preset constraints based on corresponding evaluation indicators. Specifically, in the process of determining the initial configuration parameters that meet the preset constraints, first, a plurality of machine learning algorithms are used to fit the configuration parameters of the block storage cache system to obtain a corresponding number of candidate configuration parameters; then, a weighted evaluation can be performed on these candidate configuration parameters using a preset comprehensive evaluation formula, and the initial configuration parameters that meet the preset constraints can be determined based on the corresponding evaluation indicators. In a specific embodiment, the formula for weighted evaluation of configuration parameters is as follows:

[0046] ;

[0047] Among them, MSE stands for mean square error, MAE stands for mean absolute error, RMSE stands for root mean square error, R 2 represents the coefficient of determination.

[0048] In another specific embodiment, the real-time performance data of the block storage cache system is monitored by a user-mode program constructed based on an extended Berkeley packet filter, which may include: compiling a user-mode program pre-constructed based on an extended Berkeley packet filter through a just-in-time compiler, so as to monitor the block storage cache system through the user-mode program and obtain corresponding real-time performance data. Specifically, the user-mode program constructed based on ebpf can be executed safely and efficiently in the kernel; through JIT (Just-In-Time, just-in-time compiler) compilation, the user-mode monitoring program can be compiled into machine code after being loaded into the kernel, ensuring execution efficiency similar to that of the native kernel code, which means that even in a high-load production environment, using a user-mode monitoring program for system observation and tracking will not introduce significant performance overhead; thus, monitoring and analysis of key system activities can be achieved without changing the kernel code or recompiling the kernel; the block storage cache system can be monitored to obtain corresponding real-time performance data.

[0049] In another specific embodiment, the monitoring of the block storage cache system by the user state program to obtain corresponding real-time performance data may include: executing the user state program when a preset event occurs, so as to monitor the block storage cache system by the user state program to obtain corresponding real-time performance data. Specifically, the observation and analysis of system activities by the user state program may be performed only when a specific event occurs, thereby minimizing the interference of the user state program on the system.

[0050] Step S12: Calculate the parameters characterizing the processing order of the input and output requests according to the real-time performance data to obtain corresponding parameter values.

[0051] In an embodiment of the present application, the real-time performance data of the block storage cache system when processing IO requests can be obtained through the above steps. Based on the real-time performance data, the parameters characterizing the processing order of input and output requests (sequential_cutoff, a mechanism for limiting or controlling access to sequential IO operations) can be calculated to obtain corresponding parameter values. In a specific embodiment, the parameters characterizing the processing order of input and output requests are calculated according to the real-time performance data to obtain corresponding parameter values, which may include: calculating the parameters characterizing the processing order of input and output requests according to the real-time performance data and based on the configuration parameters of the block storage cache system at the current moment to obtain corresponding parameter values. Specifically, based on the acquired real-time performance data of the block storage cache system, the sequential_cutoff value characterizing the processing order of input and output requests can be dynamically calculated in combination with the configuration parameters characterizing the optimal performance set in the previous cycle of the block storage cache system to obtain the corresponding parameter value.

[0052] Step S13: Based on the parameter value, the configuration parameters of the block storage cache system are adjusted through the file system interface, so as to use the adjusted block storage cache system to process related input and output requests.

[0053] In an embodiment of the present application, through the above steps, a parameter value characterizing the processing order of input and output requests can be obtained based on the real-time performance data of the block storage cache system and the configuration parameters at the corresponding time, and the block storage cache system can be controlled based on the parameter value; specifically, the configuration parameters of the block storage cache system can be adjusted through the file system interface, and the adjusted block storage cache system can be used to process the relevant input and output requests; bypass control of the input and output requests of the block storage cache system can be achieved.

[0054] In a specific embodiment, it may also include: obtaining the operating status data of the block storage cache system through the file system interface; if the operating status data indicates that there is a state abnormality, executing a pre-set corresponding abnormal recovery strategy through the file system interface, so as to use the block storage cache system after the abnormality is recovered to process related input and output requests. Specifically, if the operating status data of the block storage cache system obtained through the file system interface indicates that there is a state abnormality, such as a garbage collection mechanism abnormality, etc., in this case, the block storage cache system can be abnormally recovered through a pre-set abnormal recovery strategy; specifically, the corresponding type of abnormal recovery strategy is executed through the file system interface to implement the abnormal recovery operation of the block storage cache system.

[0055] It can be seen that in the process of the block storage cache system processing input and output requests, the present application can obtain the performance data of the block storage cache system, adjust the configuration parameters of the block storage cache system according to the actual processing situation, and realize dynamic adjustment of the performance of the block storage cache system; and can realize the internal abnormal state recovery of the block storage cache system according to the pre-set abnormal recovery strategy; it can meet the storage performance requirements of various business scenarios and can improve the performance of block storage cache acceleration.

[0056] like Figure 2 As shown, the embodiment of the present application discloses a block storage caching method, which specifically includes:

[0057] In this embodiment, the block storage cache system is located between the host and the back-end storage device, and is divided into a model building module, an initialization configuration decision module, a real-time configuration decision module, a component abnormal recovery module, and a component monitoring module; the initial parameters of the block storage cache system can be configured, and the parameters can be controlled according to the actual performance, that is, the cache acceleration system can be adapted to different business scenarios through intelligent configuration decisions. Among them, the model building module is used to model historical data using a variety of machine learning methods, including linear regression, ridge regression, lasso regression, support vector machine, single factor multi-factor, stepwise regression, robust linear regression and other methods, and then select the optimal model (i.e., the optimal configuration parameters) according to the model evaluation index. The model building module establishes an initial configuration optimal model for the initialization decision module. In the subsequent processing of input and output requests, the model building module can establish a real-time performance optimal model for the real-time configuration decision module (i.e., determine the optimal configuration parameters according to the actual performance). It can be understood that the initialization configuration decision module is used to solve the optimal initial configuration parameters under constraints, so that the block storage system can obtain the optimal storage performance in a specific business scenario. The real-time configuration decision module can be used to directly calculate the sequential_cutoff prediction value according to the real-time performance model, and dynamically set relevant parameters through the sysfs (SystemFileSystem, system file system) interface, so as to achieve performance adjustment of the block storage cache acceleration system. Furthermore, the component abnormality recovery module can handle the garbage collection mechanism abnormality of the cache acceleration system through the trigger_gc (GarbageCollection, an interface that represents the triggering of garbage collection) interface provided by the sysfs interface. The component monitoring module obtains the writeback_rate (writeback rate) and cache_avaliable_percent (cache available ratio) parameters through the sysfs interface, and obtains the internal performance data of the block storage cache acceleration system through the kernel observation and tracking module, such as random read and write iops (Input / OutputOperationsPerSecond, referring to the number of input and output operations per second), random read and write bandwidth, random read and write latency, sequential read and write iops, sequential read and write bandwidth, and sequential read and write latency; then these data are normalized and saved as historical performance data. The historical performance data can provide data support for the model building module.

[0058] It should be pointed out that the kernel observation and tracking module in this embodiment can support the safe and efficient execution of user-mode monitoring programs in the kernel, as well as its flexible data sharing and event-driven. The user-mode monitoring program can be dynamically loaded into the kernel and bound to various kernel events and Hook points (hooking into the program execution process at a specific time point or event), such as system calls, network events, process scheduling, etc., so that the monitoring and analysis of key system activities can be achieved without changing the kernel code or recompiling the kernel. Through JIT (Just-In-Time, real-time compiler) compilation, the user-mode monitoring program is compiled into machine code after being loaded into the kernel, ensuring execution efficiency similar to that of the native kernel code. This means that even in a high-load production environment, using user-mode monitoring programs for system observation and tracking will not introduce significant performance overhead. The flexible data sharing mapping structure, as an efficient data structure, allows the user-mode monitoring program to share data with user space applications, making real-time data collection, processing and user space analysis possible. In addition, the event-driven model of the observation and tracking module allows these tools to be executed only when specific events occur, thereby minimizing interference to the system. It is understandable that the Sysfs interface can present kernel objects and their attributes to the user space in the form of files and directories. Through the Sysfs interface, users and applications can easily access and modify the parameters and status of kernel components. The Sysfs interface can provide the component monitoring module with an interface for obtaining the writeback_rate and cache_avaliable_percent parameters of the cache acceleration system. When the component abnormality recovery module handles abnormal conditions of the block storage cache system, it can also be completed through the Sysfs interface.

[0059] It is understandable that NVMe / SSD hard disk is a hard disk that uses a communication protocol and interface standard designed for high-speed non-volatile storage media. Compared with the traditional SATA (Serial Advanced Technology Attachment, a disk interface standard) interface, NVMe makes full use of the high-speed transmission capability of the PCIe (Peripheral Component Interconnect Express, a high-speed serial computer expansion bus standard) bus, providing higher bandwidth and lower latency. Due to its high read and write speed, NVMe / SSD hard disks are often used as cache media to store hot data and improve the overall IO performance of the system. NVMe / SSD hard disks can significantly improve random read and write iops and are suitable for high concurrent access scenarios. NVMe / SSD hard disks can accelerate data access, reduce waiting time, and improve user experience. SATA hard disks and SAS (Serial Attached SCSI, representing serial attached SCSI) hard disks are traditional storage interface standard hard disks, usually in mechanical hard disks (Hard Disk Drive, HDD) and some SSDs. Although they are not as good as NVMe in performance, they have advantages in capacity and cost. SATA / SAS hard disks usually provide a large storage space, suitable for storing large amounts of data; compared to NVMe / SSD, SATA / SAS hard disks have lower unit storage costs; they are highly compatible and suitable for a variety of systems and devices. SATA / SAS hard disks are usually used to store cold data or non-hot data, forming a layered storage architecture with the cache layer; they can ensure reliable storage and long-term preservation of important data; by adding SATA / SAS hard disks, the storage capacity of the system can be easily expanded.

[0060] In a specific embodiment, Figure 3The figure shows the initial parameter configuration process of the block storage cache system. First, the fio test parameters are designed according to the block request size and average request time in the business type. And the characteristic parameters of the block storage cache acceleration system are designed and the value range is set. For example, the writeback ratio writeback_percent is (51015203040). The minimum writeback speed writeback_rate_minmum is (8512102410240). The sequential cutoff value sequential_cutoff is (016K128K1M2M3M4M), the cache pre-read size readahead_kbs is (409681921228816384), the cache replacement strategy is (lrufiforandom), and the ratio of nvme and mechanical hard disk capacity is (5101520). When the cache mode of the block device cache acceleration system is writeback, different configuration parameters such as writeback ratio, minimum writeback speed, sequential cutoff value and cache pre-read size can be set respectively. Then execute the fio test three times and take the average value as the test result. Further, the comprehensive sequential read and write performance, comprehensive random read and write performance and comprehensive read and write performance value can be calculated according to the test results. Comprehensive random read and write performance score random_weight=(0.25×randread_iops+0.25×randwrite_iops+0.10×randread_bw+0.10×randwrite_bw)-(0.10×randread_lat+0.10randwrite_lat); Comprehensive sequential read and write performance score seq_weight=(0.10×read_iops+0.10×write_iops+0.05×read_bw+0.05×write_bw)-(0.05×read_lat+0.05×write_lat); Comprehensive performance score weight=0.5×comprehensive random read and write performance score+0.5×comprehensive sequential read and write performance score. Then the data can be further modeled; specifically, linear regression, ridge regression, lasso regression, support vector machine, single factor multi-factor, stepwise regression, robust linear regression and other machine learning methods can be used to fit the data, and the dependent variable is the comprehensive performance score. The independent variables are write-back ratio, minimum write-back speed, sequential cutoff value and cache pre-read size. In addition, according to the mean square error MSE, mean absolute error MAE, root mean square error RMSE and determination coefficient R of each model, the performance of the model is improved. 2Calculate the comprehensive evaluation index of the model, select the best model with the largest comprehensive evaluation index, and save the model and related parameters. It can be understood that according to the pre-set cost constraints, fmincon can be used to solve the optimal value of the nonlinear optimization problem under the cost constraints; set the optimal initial configuration value in the block storage cache system.

[0061] In a specific embodiment, Figure 4 The figure shows the process of adjusting the parameters of the block storage cache system based on real-time performance data. First, the data collected by the component monitoring module is normalized. For example, the cache replacement strategies lru, fifo, and random are mapped to 1, 2, and 3 respectively; the request types random read, random write, sequential read, and sequential write are mapped to 1, 2, 3, and 4. Then, the comprehensive sequential read and write performance, comprehensive random read and write performance, and comprehensive read and write performance values ​​are calculated respectively. Among them, the comprehensive random read and write performance score random_weight=(0.25×randread_iops+0.25×randwrite_iops+0.10×randread_bw+0.10×randwrite_bw)-(0.10×randread_lat+0.10randwrite_lat); the comprehensive sequential read and write performance score seq_weight=(0.10×read_iops+0.10×write_iops+0.05×read_bw+0.05×write_bw)-(0.05×read_lat+0.05×write_lat); the comprehensive performance score weight=0.5×comprehensive random read and write performance score+0.5×comprehensive sequential read and write performance score. Further model the data. A variety of machine learning methods such as linear regression, ridge regression, lasso regression, support vector machine, single factor and multi-factor, stepwise regression, and robust linear regression can be used to fit the data. The dependent variables are comprehensive random read and write performance score, comprehensive sequential read and write performance score, and comprehensive performance score. The independent variables are minimum write-back speed, cache pre-read size, random read and write iops, random read and write bandwidth, random read and write latency, sequential read and write iops, sequential read and write bandwidth, sequential read and write latency, cache available ratio cache_avaliable_percent, request type, request block size, request data length, NVMe and mechanical hard disk capacity ratio, and sequential cutoff value. Afterwards, the mean square error MSE, mean absolute error MAE, root mean square error RMSE, and determination coefficient R of each model can be used to represent the model. 2Calculate the comprehensive evaluation index of the model, select the three best models with the largest comprehensive evaluation index of the model, and save the model and related parameters. The model with the comprehensive random read and write performance score as the dependent variable is used to correspond to the business scenario with high random read and write requirements; the three models are used to calculate the sequential cutoff value in different business scenarios. The model with the comprehensive random read and write performance score as the dependent variable corresponds to the business scenario with high random read and write requirements; the model with the comprehensive sequential read and write performance score as the dependent variable corresponds to the business scenario with high sequential read and write requirements; the model with the comprehensive read and write performance score as the dependent variable corresponds to the business scenario with high average read and write requirements. In a specific embodiment, the model expression for calculating the sequential cutoff value can be y=wx+b, x=[x1,x2,…,xn]T, w=[w1,w2,…,wn]; xk is the sequential cutoff value, and wk is the coefficient of the sequential cutoff value.

[0062] ;

[0063] If wk>0, then:

[0064] ;

[0065] If wk<0, then:

[0066] ;

[0067] Among them, y max is the maximum read and write performance of the NVMe hard disk, which can be obtained by querying the hardware manual. max .xk min The minimum value of the corresponding sequential cutoff is 1. max The maximum value of the corresponding sequence cutoff value is the hard disk capacity of nvme.

[0068] Furthermore, the previously established model and parameters can be loaded according to the business scenario type of the business configuration. The business scenario type can be sequential, random, and average. By default, the average model is used to calculate the optimal value of sequential_cutoff; and the configuration parameters corresponding to the optimal sequential_cutoff are set.

[0069] In another specific embodiment, during the process of performing abnormal recovery, the values ​​of cache_avalible_percent and state can be determined. If cache_avalible_percent is 95 and state is clean, the garbage collection mandatory condition is true. If the garbage collection mandatory condition is true, the GC is triggered to force garbage collection. It is determined that the / sys / block / bcacheN / bcache directory exists, but the / sys / block / bcacheN / bcache / stop file does not exist, then the bcache device stop abnormal condition is true. If the bcache device stop abnormal condition is true, the / sys / fs / bcache / pendings_cleanup file content can be set to the value of the / sys / block / bcacheN / bcache / backing_dev_name content to force the abnormal device to be cleared.

[0070] In another specific embodiment, when bpf (Berkeley Packet Filter) observes bcache (cache management tool, here refers to the block storage cache system), the time start_ts for saving the service request can be set in the service request initialization function tracepoint (trace point) function to bpfmap (a data structure used for data exchange and information transmission between eBPF programs and user space applications). In the service request end function tracepoint function, cmd_flags is marked according to the request type and set to the read-write request type. If cmd_flags is marked as write, the request type is write. If cmd_flags is marked as read, the request type is read. In the service request end function tracepoint function, the requested sector number is saved as last_sector in bpfmap. If the requested sector is last_sector+1 saved in bpfmap, it is judged as a sequential type. In the service request end function tracepoint function, the current time is obtained minus the time start_ts in the bpfmap type to obtain the delay value lat. The requested data length datalen is saved in the service request end function tracepoint function. Furthermore, if it is a sequential read type, the sequential iopssequential_reads in the bpfmap type is incremented by 1, the sequential bandwidth sequential_reads_bytes data length in the bpfmap type is added with the requested length datalen, and lat is written to the sequential read delay sequential_reads_lat. If it is a sequential write type, the sequential iopssequential_writes in the bpfmap type is incremented by 1, the sequential bandwidth sequential_writes_bytes data length in the bpfmap type is added with the requested length datalen, and lat is written to the sequential read delay sequential_writes_lat. If it is a random read type, the sequential iopsrandom_reads in the bpfmap type is incremented by 1, the sequential bandwidth random_reads_bytes data length in the bpfmap type is added with the requested length datalen, and lat is written to the sequential read delay random_reads_lat. If it is a random write type, add 1 to the sequential iopsrandom_writes in the bpfmap type, add the sequential bandwidth random_writes_bytes data length in the bpfmap type to the requested length datalen, and write lat to the sequential read latency random_writes_lat.

[0071] It can be seen that the present application can use the initial optimal configuration parameter settings of the block storage cache acceleration component to process the corresponding input and output requests, and combine the real-time performance data and real-time cache sequential_cutoff to realize intelligent parameter decision-making and internal abnormality recovery of the cache acceleration system. The present application can intelligently calculate different optimal initial parameters for different scenarios; it can repair internal abnormalities of the block storage cache acceleration system without modifying the kernel; it can intelligently set the cache bypass strategy to realize dynamic adjustment of the performance of the business block storage cache acceleration system. In other words, the present application can intelligently optimize the initial configuration parameters under the condition of meeting the cost constraint, and significantly improve the performance of the block storage cache acceleration. At the same time, it can realize intelligent adjustment of the storage IO path, which can meet the storage performance requirements of various business application scenarios. In addition, the intelligent decision-making model can continuously improve itself and continuously improve storage performance; this not only improves the performance of block storage cache acceleration, but also reduces costs, and has broad application prospects.

[0072] like Figure 5 As shown, the embodiment of the present application discloses a block storage cache device, which is applied to a server, including:

[0073] A monitoring module 11 is used to monitor the real-time performance data of the block storage cache system through a user-mode program constructed based on an extended Berkeley packet filter when the block storage cache system is used to process input and output requests; the block storage cache system is a block storage cache system containing configuration parameters pre-constructed based on a machine learning algorithm;

[0074] A calculation module 12, configured to calculate parameters representing the processing order of input and output requests according to the real-time performance data to obtain corresponding parameter values;

[0075] The adjustment module 13 is used to adjust the configuration parameters of the block storage cache system through the file system interface based on the parameter value, so as to use the adjusted block storage cache system to process related input and output requests.

[0076] It can be seen that in the process of the block storage cache system processing input and output requests, the present application can obtain the performance data of the block storage cache system, adjust the configuration parameters of the block storage cache system according to the actual processing situation, and realize dynamic adjustment of the performance of the block storage cache system. It can meet the storage performance requirements of various business scenarios and improve the performance of block storage cache acceleration.

[0077] In a specific embodiment, the device may further include:

[0078] A parameter determination module, used to perform fitting processing on the configuration parameters of the block storage cache system based on a preset type of machine learning algorithm to determine initial configuration parameters that meet preset constraints;

[0079] A parameter configuration module is used to configure the parameters of the block storage cache system using the initial configuration parameters so that when the block storage cache system is used to process input and output requests, real-time performance data of the block storage cache system is monitored through a user-mode program built based on an extended Berkeley packet filter.

[0080] In a specific embodiment, the parameter determination module may include:

[0081] A fitting processing unit, used for fitting the configuration parameters of the block storage cache system based on a preset type of machine learning algorithm to obtain a corresponding number of candidate configuration parameters;

[0082] The parameter determination unit is used to perform weighted evaluation on the plurality of candidate configuration parameters using a preset formula, and determine the initial configuration parameters that meet the preset constraint conditions according to the corresponding evaluation indicators.

[0083] In a specific embodiment, the monitoring module 11 may include:

[0084] The monitoring unit is used to compile a user-mode program pre-built based on the extended Berkeley packet filter through a just-in-time compiler, so as to monitor the block storage cache system through the user-mode program and obtain corresponding real-time performance data.

[0085] In another specific embodiment, the monitoring module 11 may include:

[0086] The program execution unit is used to execute the user state program when a preset event occurs, so as to monitor the block storage cache system through the user state program and obtain corresponding real-time performance data.

[0087] In a specific embodiment, the calculation module 12 may include:

[0088] The parameter calculation unit is used to calculate the parameters representing the processing order of the input and output requests according to the real-time performance data and based on the configuration parameters of the block storage cache system at the current moment to obtain corresponding parameter values.

[0089] In a specific embodiment, the device may further include:

[0090] A data acquisition module, used for acquiring the operation status data of the block storage cache system through a file system interface;

[0091] The strategy execution module is used to execute a preset corresponding abnormality recovery strategy through the file system interface when the running status data indicates that there is an abnormal state, so as to process related input and output requests using the block storage cache system after the abnormality is recovered.

[0092] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0093] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the block storage cache method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0094] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0095] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0096] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the block storage caching method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0097] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed block storage cache method. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0098] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0099] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0101] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0102] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A block storage caching method, characterized in that: Applicable to servers, including: When processing input and output requests using a block storage cache system, real-time performance data of the block storage cache system is monitored by a user-mode program built based on an extended Berkeley packet filter; the block storage cache system is a block storage cache system that includes configuration parameters and is pre-built based on a machine learning algorithm; Calculating parameters representing the processing order of input and output requests according to the real-time performance data to obtain corresponding parameter values; Based on the parameter value, adjusting the configuration parameters of the block storage cache system through the file system interface, so as to use the adjusted block storage cache system to process the related input and output requests; Wherein, before monitoring the real-time performance data of the block storage cache system by a user-mode program constructed based on the extended Berkeley packet filter, the method further includes: Fitting the configuration parameters of the block storage cache system based on a preset type of machine learning algorithm to determine initial configuration parameters that meet preset constraints; The block storage cache system is configured with the initial configuration parameters so that when the block storage cache system is used to process input and output requests, real-time performance data of the block storage cache system is monitored by a user-mode program built based on an extended Berkeley packet filter.

2. The block storage caching method according to claim 1, characterized in that: The machine learning algorithm based on the preset type performs fitting processing on the configuration parameters of the block storage cache system to determine the initial configuration parameters that meet the preset constraint conditions, including: Based on a preset type of machine learning algorithm, a configuration parameter of the block storage cache system is fitted to obtain a number of corresponding candidate configuration parameters; A weighted evaluation is performed on a number of the candidate configuration parameters using a preset formula, and initial configuration parameters that meet preset constraint conditions are determined according to corresponding evaluation indicators.

3. The block storage caching method according to claim 1, characterized in that: The monitoring of the real-time performance data of the block storage cache system by a user-mode program constructed based on an extended Berkeley packet filter includes: The user-state program pre-built based on the extended Berkeley packet filter is compiled by a just-in-time compiler, so as to monitor the block storage cache system through the user-state program and obtain corresponding real-time performance data.

4. The block storage cache method according to claim 3, characterized in that: The block storage cache system is monitored by the user state program to obtain corresponding real-time performance data, including: When a preset event occurs, the user state program is executed to monitor the block storage cache system through the user state program to obtain corresponding real-time performance data.

5. The block storage cache method according to claim 1, characterized in that: The calculating of the parameters characterizing the processing order of the input and output requests according to the real-time performance data to obtain corresponding parameter values ​​includes: According to the real-time performance data and based on the configuration parameters of the block storage cache system at the current moment, the parameters characterizing the processing order of the input and output requests are calculated to obtain corresponding parameter values.

6. The block storage cache method according to any one of claims 1 to 5, characterized in that: Also includes: Obtaining the operating status data of the block storage cache system through the file system interface; If the running status data indicates that there is a state abnormality, a preset corresponding abnormality recovery strategy is executed through the file system interface, so as to process related input and output requests using the block storage cache system after the abnormality is recovered.

7. A block storage cache device, characterized in that: Applicable to servers, including: A monitoring module, used for monitoring the real-time performance data of the block storage cache system through a user-mode program built based on an extended Berkeley packet filter when the block storage cache system is used to process input and output requests; the block storage cache system is a block storage cache system containing configuration parameters pre-built based on a machine learning algorithm; A calculation module, used to calculate the parameters representing the processing order of the input and output requests according to the real-time performance data to obtain corresponding parameter values; An adjustment module, configured to adjust the configuration parameters of the block storage cache system through a file system interface based on the parameter value, so as to process related input and output requests using the adjusted block storage cache system; Wherein, the block storage cache device further includes: A parameter determination module, used to perform fitting processing on the configuration parameters of the block storage cache system based on a preset type of machine learning algorithm to determine initial configuration parameters that meet preset constraints; A parameter configuration module is used to configure the parameters of the block storage cache system using the initial configuration parameters so that when the block storage cache system is used to process input and output requests, real-time performance data of the block storage cache system is monitored through a user-mode program built based on an extended Berkeley packet filter.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the block storage caching method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the block storage cache method according to any one of claims 1 to 6.

Images