Method and device for dynamically adjusting file system parameters, electronic equipment and storage medium

Abstract

The invention provides a method and device for dynamically adjusting file system parameters, electronic equipment and a storage medium. The method comprises the following steps: determining a relationship between parameters and performance indexes of a file system under the access of different application programs; acquiring sample data, marking the sample data to obtain marked data, and determining the sample data and the marked data as a training data set; training the training data set through multivariate logistic regression to obtain a trained logistic regression model; under the access of different application programs, the parameter values of the file system are dynamically adjusted based on the logistic regression model, the problem of limitation of static configuration of the file system in related technologies is solved, and experimental attempts are not needed for each adjustment.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a method, apparatus, electronic device, and storage medium for dynamically adjusting file system parameters. Background Art

[0002] Different application services have different emphases on file systems. For example, for streaming media applications, their IO access patterns are mainly sequential reads and writes, and generally large files, emphasizing the throughput of the file system; for example, for database management systems, their IO access patterns are mainly random reads and writes, emphasizing the concurrency of IOPS (Input / Output Operations Per Second); for example, for mail servers, they are usually stored in the form of small files, emphasizing the read and write of small files.

[0003] The default configurations of current various application-level distributed file systems emphasize general applicability. However, in specific applications, these configurations cannot achieve an optimized state in terms of I / O performance. And currently, many optimization methods are for the configurations under static specific application systems. When the access pattern of the file system or the read and write of different file sizes change, the previous single static configuration has its limitations. In related technologies, in order to achieve the optimal performance access of the file system, it is necessary to judge whether the parameter configuration can optimize the performance of the file system through experimental attempts each time. Summary of the Invention

[0004] The present disclosure provides a method, apparatus, electronic device, and storage medium for dynamically adjusting file system parameters.

[0005] According to a first aspect of the present disclosure, there is provided a method for dynamically adjusting file system parameters, the method including: determining the relationship between the parameters of the file system and the performance metrics under the access of different application programs; obtaining sample data, marking the sample data to obtain marked data, and determining the sample data and the marked data as a training data set, where the sample data includes the parameters of the file system, the performance metrics related to the parameters of the file system, and the access pattern of the application program, and the marked data is used to mark whether the performance metrics in the sample data reach an optimized state; training the training data set through multiple logistic regression to obtain a trained logistic regression model; dynamically adjusting the parameter values of the file system based on the logistic regression model under the access of different application programs.

[0006] In some embodiments, the parameter values of the file system are dynamically adjusted based on a logistic regression model, including: inputting the parameter values to be optimized into the logistic regression model for processing to obtain predicted values of the performance metrics corresponding to the parameter values; judging whether the values of the performance metrics corresponding to the parameter values reach the optimized state according to the predicted values; in response to the values of the performance metrics corresponding to the parameter values not reaching the optimized state, adjusting the parameter values and inputting the adjusted parameter values into the logistic regression model for processing until the values of the performance metrics corresponding to the adjusted parameter values reach the optimized state.

[0007] In some embodiments, the parameters of the file system include the block size and the buffer size, and the performance metrics include at least one of the number of read / write operations per second, the throughput, and the memory resource utilization rate. Determining the relationship between the parameters of the file system and the performance metrics under the access of different application programs includes: determining the relationship between the block size of the file system and the number of read / write operations per second and / or the throughput under the access of different application programs; determining the relationship between the buffer size of the file system and the memory resource utilization rate under the access of different application programs.

[0008] In some embodiments, obtaining sample data, marking the sample data to obtain marked data, and determining the sample data and the marked data as the training data set, including: for each access of a different application program, determining at least one of the multiple performance metrics related to the parameters of the file system as the performance optimization metric; obtaining sample data, using the parameters of the file system, the performance optimization metric, and the mark as the table headers, and filling the sample data into the corresponding columns according to the table headers to generate an initial table; determining the marked values of the performance optimization metric in the access mode, where the marked values include a first value and a second value, the first value is used to indicate that the value of the performance optimization metric has reached the optimized state, and the second value is used to indicate that the value of the performance optimization metric has not reached the optimized state; filling the marked values into the column corresponding to the mark in the initial table to generate a target table, and determining the target table as the training data set.

[0009] In some embodiments, after training the training data set through multiple logistic regression to obtain a trained logistic regression model, the method further includes: obtaining a validation data set; evaluating the logistic regression model using the validation data set and adjusting the logistic regression model.

[0010] In some embodiments, evaluating the logistic regression model using the validation data set and adjusting the logistic regression model includes: determining the accuracy rate of the logistic regression model according to the validation data set and the prediction results of the logistic regression model; adjusting the hyperparameters of the logistic regression model according to the accuracy rate.

[0011] According to a second aspect of the present disclosure, there is provided an apparatus for dynamically adjusting file system parameters, the apparatus including: a determination module configured to determine the relationship between the parameters of the file system and the performance metrics under the access of different application programs; an acquisition module configured to acquire sample data, label the sample data to obtain labeled data, and determine the sample data and the labeled data as a training data set, wherein the sample data includes the parameters of the file system, the performance metrics related to the parameters of the file system, and the access patterns of application programs, and the labeled data is used to label whether the performance metrics in the sample data reach an optimized state; a training module configured to train the training data set through multiple logistic regression to obtain a trained logistic regression model; and a dynamic adjustment module configured to dynamically adjust the parameter values of the file system based on the logistic regression model under the access of different application programs.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of the foregoing first aspect.

[0013] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method of the foregoing first aspect.

[0014] According to a fifth aspect of the present disclosure, there is provided a computer program product, including a computer program which, when executed by a processor, implements the method of the foregoing first aspect.

[0015] The method, device, electronic device and storage medium for dynamically adjusting file system parameters provided by the present disclosure determine the relationship between the parameters of the file system and the performance metrics under the access of different application programs; obtain sample data, label the sample data to obtain labeled data, and determine the sample data and the labeled data as the training data set, where the sample data includes the parameters of the file system, the performance metrics related to the parameters of the file system, and the access patterns of the application programs, and the labeled data is used to label whether the performance metrics in the sample data reach the optimized state; train the training data set through multiple logistic regression to obtain a trained logistic regression model; under the access of different application programs, dynamically adjust the parameter values of the file system based on the logistic regression model. The method of the present disclosure determines the relationship between the parameters of the file system and the performance metrics according to the influence of the parameters of the file system on the access performance under different application accesses, and establishes a logistic regression model to learn the relationship and rules between the parameters of the file system and the performance metrics. The trained logistic regression model can judge whether the current parameters can optimize the performance of the file system. Therefore, for different application accesses, dynamically adjusting the parameters according to the trained logistic regression model can improve the performance of the file system, solve the limitation problem of static configuration of the file system in the related art, and each adjustment does not need to go through experimental attempts again.

[0016] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0018] Figure 1 is a schematic flowchart of a method for dynamically adjusting file system parameters provided by an embodiment of the present disclosure;

[0019] Figure 2 is a schematic flowchart of another method for dynamically adjusting file system parameters provided by an embodiment of the present disclosure;

[0020] Figure 3 is a schematic structural diagram of a device for dynamically adjusting file system parameters provided by an embodiment of the present disclosure;

[0021] Figure 4 is a schematic block diagram of an exemplary electronic device 400 provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] The exemplary embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0023] The method, device, electronic device, and storage medium for dynamically adjusting file system parameters according to the embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0024] Figure 1 A method for dynamically adjusting file system parameters is provided. This method is applied to a distributed file system, and specifically, it can be applied to various current application-level distributed file systems.

[0025] As Figure 1 shown, the method may include:

[0026] Step 101: Determine the relationship between the parameters of the file system and the performance metrics under the access of different application programs.

[0027] In one embodiment, the access patterns of different application services to the file system and / or the read / write of different file sizes may be different. The access patterns include random read / write and sequential read / write. The read / write of different file sizes can be divided into large file read / write and small file read / write, etc.

[0028] Therefore, different application services have different emphases on the file system. For example, for a streaming media application, its IO access pattern is mainly sequential read / write, and it is generally a large file, emphasizing the throughput of the file system. For example, for a database management system, its IO access pattern is mainly random read / write, emphasizing the concurrent ability of the number of read / write operations per second (hereinafter referred to as IOPS). For example, for a mail server, it is usually stored in the form of small files, emphasizing small file read / write.

[0029] In actual application, according to the characteristics of different application services, the distributed file system is configured specifically. During the operation of these application-level distributed file systems, there are some parameters that support online adjustment, and these adjustable parameters can be selected as variables.

[0030] In one embodiment, the adjustable parameters mainly include: the maximum read / write partition size (i.e., block size) of a single input / output (or write / read, I / O) operation, and the buffer size.

[0031] Taking the JuiceFS distributed file system as an example, the block size of a file is configurable. The default block size is 4MB, which can be adjusted to a minimum of 64K and a maximum of 16MB. In actual applications, a smaller block size can be configured to improve the performance of random reads, reduce latency, and increase IOPS. The read-write buffer is also configurable, with a default size of 300MiB.

[0032] In one embodiment, performance metrics are used to evaluate the performance of a file system. Under the access of different application programs, the performance requirements for the file system are different. Therefore, it is necessary to obtain the relationship between configurable parameters and performance metrics to achieve the required performance metrics by adjusting the parameters.

[0033] In one embodiment, the performance metrics include at least one of the number of read and write operations per second, throughput, and memory resource utilization.

[0034] In one embodiment, when multiple performance metrics need to be considered under a certain access mode, the configurable parameters need to be adjusted to achieve the balanced optimization of multiple performance metrics.

[0035] It should be noted that in the related art, in order to achieve the optimal state performance access of the file system, it is necessary to judge whether the parameter configuration can optimize the performance of the file system through experimental attempts each time. To solve this problem, the present disclosure uses machine learning to learn the relationship between the parameters of the file system and the performance metrics under the access of different application programs, and generates an optimization model based on machine learning. Using this model to achieve guiding and predicting the parameter optimization of the file system, as described in steps 102-104 below.

[0036] Step 102: Obtain sample data, label the sample data to obtain labeled data, and determine the sample data and the labeled data as the training data set. Among them, the sample data includes the parameters of the file system, the performance metrics related to the parameters of the file system, and the access mode of the application program. The labeled data is used to label whether the performance metrics in the sample data reach the optimized state.

[0037] In some embodiments, the sample data can be obtained by simulating the access of various different application programs and testing and recording the performance-related information of the file system. For example, if the performance metric to be optimized in a certain mode is throughput, simulate this mode to adjust the configurable parameters and record the change relationship between the parameters and the throughput.

[0038] In some embodiments, according to the optimization requirements of the performance metrics when different application programs access the file system, the performance metrics are labeled whether they reach the optimized state with data under different parameter values.

[0039] Step 103: Train the training data set through multinomial logistic regression to obtain a trained logistic regression model.

[0040] In some embodiments, a logistic regression model can be established through the Python and TensorFlow machine learning platforms, and the model can be trained using the multiple logistic regression algorithm and the training dataset.

[0041] In some embodiments, the input of the model is the parameter value of the file system, and the output is the predicted value of the performance metric. The predicted value is used to predict the performance under different parameter configurations. When there are multiple performance metrics, a balance can be found among the performance metrics to be optimized, so as to predict the overall I / O performance of the file system.

[0042] Step 104: Dynamically adjust the parameter value of the file system based on the logistic regression model under the access of different applications.

[0043] In some embodiments, under the access of different applications, it is determined whether the parameter configuration can optimize the performance of the file system through the logistic regression model, that is, the performance of the file system under different parameter configurations can be predicted. Therefore, guiding the adjustment of the parameter value based on the logistic regression model can optimize the performance of the file system.

[0044] In summary, the method for dynamically adjusting the parameters of the file system provided by the embodiments of the present disclosure determines the relationship between the parameters of the file system and the performance metrics according to the influence of the parameters of the file system on the access performance under different application accesses, and establishes a logistic regression model to learn the relationship and rules between the parameters of the file system and the performance metrics. The trained logistic regression model can determine whether the current parameters can optimize the performance of the file system. Therefore, for different application accesses, dynamically adjusting the parameters according to the trained logistic regression model can improve the performance of the file system, solve the limitation problem of the static configuration of the file system in the related art, and there is no need to conduct experimental attempts for each adjustment.

[0045] Based on Figure 1 the embodiments shown, Figure 2 Further shown is a schematic flowchart of a method for dynamically adjusting the parameters of a file system provided by the embodiments of the present disclosure. This method is applied to a distributed file system, and specifically, it can be applied to various current application-level distributed file systems.

[0046] As Figure 2 shown, the method may include:

[0047] Step 201: Determine the relationship between the parameters of the file system and the performance metrics under the access of different applications.

[0048] In one embodiment, the access modes include random read / write, sequential read / write, large file read / write, small file read / write, etc.

[0049] In one embodiment, step 201 includes: determining the relationship between the block size of the file system and the number of read / write operations per second and / or throughput under the access of different applications; determining the relationship between the buffer size of the file system and the memory resource utilization rate under the access of an application.

[0050] In one embodiment, for different applications accessing the file system, there is a certain relationship between the block size and at least one of the performance evaluation metrics including IOPS and throughput. For example, for applications that require high throughput, a larger block partition size generally needs to be selected, while for applications that require high IOPS, a smaller block partition size may be required. Therefore, when optimizing the performance of the file system, this parameter needs to be dynamically adjusted according to the needs of different application services to optimize the performance of the file system.

[0051] In one embodiment, for different application services accessing the file system, there is a certain relationship between the buffer size and the memory utilization rate. The buffer size needs to be reasonably set according to the specific disk read / write characteristics and file size. For example, if most of the data operations of a certain application service on the file system are small file operations, then an overly large buffer will waste memory resources and the efficiency improvement will not be obvious. On the contrary, if the data operations are mainly large file reads and writes, appropriately increasing the buffer can reduce the number of I / O operations and improve the efficiency.

[0052] Step 202: Obtain sample data, label the sample data to obtain labeled data, and determine the sample data and the labeled data as the training data set. Among them, the sample data includes the parameters of the file system, the performance metrics related to the parameters of the file system, and the access patterns of the applications. The labeled data is used to label whether the performance metrics in the sample data have reached the optimized state.

[0053] In one embodiment, the IOMeter is used to test the performance of the file system, simulate various IO modes and record performance-related information, and use it as the original data for model training. The training data set is determined by screening from the original data.

[0054] In one embodiment, step 202 includes: for the access of each different application, determining at least one of multiple performance metrics related to the parameters of the file system as the performance optimization metric; obtaining sample data, using the parameters of the file system, the performance optimization metric, and the label as the table headers, and filling the sample data into the corresponding columns according to the table headers to generate an initial table; determining the labeled value of the performance optimization metric in the access mode, where the labeled value includes a first value and a second value. The first value is used to indicate that the value of the performance optimization metric has reached the optimized state, and the second value is used to indicate that the value of the performance optimization metric has not reached the optimized state; filling the labeled value into the column corresponding to the label in the initial table to generate a target table, and determining the target table as the training data set.

[0055] In one embodiment, as shown in Table 1, the table header is the functional characteristics selected for studying the dynamically changing application access pattern. As shown in the table, the selected functional characteristics include the parameters of the file system: block size, buffer size, performance metrics: IOPS, throughput, memory occupancy rate, and a flag.

[0056] Taking throughput as an example, the value in the flag column indicates whether the throughput of the I / O pattern described by a certain row has reached the optimized state, where 1 represents the optimized state and 0 represents that optimization is still required.

[0057] Table 1 Example diagram of the training dataset table

[0058]

[0059] In one embodiment, the performance optimization metrics include a single performance metric or a combination of multiple performance metrics. For example, in a certain access pattern, the access performance of the file system only considers throughput, or it needs to consider the throughput while also considering the impacts of IOPS and memory occupancy rate.

[0060] Step 203: Train the training dataset through multinomial logistic regression to obtain a trained logistic regression model.

[0061] In actual application, after the sample data is prepared and labeled, use Python and TensorFlow to establish a logistic regression model, and adopt the multinomial logistic regression algorithm to train the dataset.

[0062] In some embodiments, after training the training dataset through multinomial logistic regression to obtain a trained logistic regression model, the method further includes: obtaining a validation dataset; using the validation dataset to evaluate the trained logistic regression model and adjusting the trained logistic regression model.

[0063] In some embodiments, using the validation dataset to evaluate the logistic regression model and adjusting the logistic regression model includes: determining the accuracy rate of the logistic regression model according to the validation dataset and the prediction results of the logistic regression model; adjusting the hyperparameters of the logistic regression model according to the accuracy rate.

[0064] In actual application, obtain a randomly allocated validation dataset from the original data, use the validation dataset to evaluate the trained logistic regression model, and for each group of data in the validation dataset, compare the result predicted by the model with the true label to calculate the model accuracy rate.

[0065] In actual application, for a combination of multiple performance metrics, an exemplary model is as follows:

[0066] Y = m1*X1 + m2*X2 + m3*X3 + n

[0067] Wherein, m1, m2, and m3 are the coefficients of the model, representing the influence of independent variables on the dependent variable. The independent variable X1 represents throughput, X2 represents IOPS, X3 represents memory occupancy rate, and n is the error term, representing the random error that the model cannot explain.

[0068] Step 204, input the parameter value to be optimized into the logistic regression model for processing, and obtain the predicted value of the performance metric corresponding to the parameter value.

[0069] In some embodiments, after establishing and validating the logistic regression model through step 203, input the parameter value into the logistic regression model, and the model will output the predicted value of the performance metric. The predicted value can be used to predict the performance of the file system under different parameter configurations.

[0070] Step 205, according to the predicted value, determine whether the value of the performance metric corresponding to the parameter value reaches the optimization state.

[0071] Step 206, in response to the value of the performance metric corresponding to the parameter value not reaching the optimization state, adjust the parameter value, and input the adjusted parameter value into the logistic regression model for processing until the value of the performance metric corresponding to the adjusted parameter value reaches the optimization state.

[0072] In some embodiments, the quality of the file system access performance is reflected by the value of one or more performance metrics. Under different access modes, the performance requirements for the file system are different, and thus the optimization requirements for the performance metrics are different.

[0073] In some embodiments, when there are multiple performance metrics to be optimized, a balance can be found among the performance metrics to be optimized to predict the overall access performance of the file system.

[0074] In some embodiments, according to the predicted value of the performance metric to be optimized, predict whether the access performance of the file system under different parameter configurations is optimal, thereby guiding the adjustment of the parameter value to make the file system performance reach the optimal.

[0075] In summary, the method for dynamically adjusting file system parameters provided by the embodiments of the present disclosure is based on the influence of the access mode, block size, and cache configuration size of the application on the access performance of the application-level distributed file system. Using machine learning to inductively analyze the dynamically changing application access mode, mining the relationship and law between the performance influence factors of configurable parameters and the performance metrics, and generating an optimization model based on machine learning. Using this model to guide the prediction of the parameter optimization of the file system can achieve the optimal performance access of the file system, without the need to judge whether the parameter configuration can optimize the file system performance by experimental attempts every time.

[0076] Corresponding to the above method for dynamically adjusting file system parameters, the present invention also provides an apparatus for dynamically adjusting file system parameters. Since the apparatus embodiment of the present invention corresponds to the above method embodiment, details not disclosed in the apparatus embodiment can be referred to the above method embodiment, and will not be elaborated herein.

[0077] Figure 3 As shown in the structural schematic diagram of an apparatus for dynamically adjusting file system parameters provided by an embodiment of the present disclosure, Figure 3 as shown, the apparatus may include:

[0078] A determination module 310, configured to determine the relationship between the parameters of the file system and the performance metrics under the access of different application programs;

[0079] An acquisition module 320, configured to acquire sample data, mark the sample data to obtain marked data, and determine the sample data and the marked data as a training data set, where the sample data includes the parameters of the file system, the performance metrics related to the parameters of the file system, and the access patterns of application programs, and the marked data is used to mark whether the performance metrics in the sample data reach an optimized state;

[0080] A training module 330, configured to train the training data set through multiple logistic regression to obtain a trained logistic regression model;

[0081] A dynamic adjustment module 340, configured to dynamically adjust the parameter values of the file system based on the logistic regression model under the access of different application programs.

[0082] In one embodiment, the dynamic adjustment module 340 is specifically configured to: input the parameter value to be optimized into the logistic regression model for processing to obtain a predicted value of the performance metric corresponding to the parameter value; determine whether the value of the performance metric corresponding to the parameter value reaches an optimized state according to the predicted value; in response to the value of the performance metric corresponding to the parameter value not reaching the optimized state, adjust the parameter value, and input the adjusted parameter value into the logistic regression model for processing until the value of the performance metric corresponding to the adjusted parameter value reaches the optimized state.

[0083] In one embodiment, the determination module 310 is specifically configured to: determine the relationship between the block size of the file system and the number of read / write operations per second and / or throughput under the access of different application programs; determine the relationship between the buffer size of the file system and the memory resource utilization rate under the access of different application programs.

[0084] In one embodiment, the obtaining module 320 is specifically configured to: for the access of each different application, determine at least one of multiple performance metrics related to the parameters of the file system as a performance optimization metric; obtain sample data, use the parameters of the file system, the performance optimization metric, and the label as table headers, and fill the sample data into the corresponding columns according to the table headers to generate an initial table; determine the label value of the performance optimization metric in the access mode, where the label value includes a first value and a second value, the first value is used to indicate that the value of the performance optimization metric has reached the optimized state, and the second value is used to indicate that the value of the performance optimization metric has not reached the optimized state; fill the label value into the column corresponding to the label in the initial table to generate a target table, and determine the target table as the training data set.

[0085] In one embodiment, it further includes an evaluation module, which obtains a validation data set; uses the validation data set to evaluate the logistic regression model and adjusts the logistic regression model.

[0086] In one embodiment, the evaluation module is specifically configured to: determine the accuracy rate of the logistic regression model according to the validation data set and the prediction result of the logistic regression model; adjust the hyperparameters of the logistic regression model according to the accuracy rate.

[0087] It should be noted that the foregoing explanation of the method embodiment also applies to the device in this embodiment, with the same principle, and will not be limited in this embodiment.

[0088] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0089] Figure 4 FIG. shows a schematic block diagram of an exemplary electronic device 400 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0090] As Figure 4As shown, device 400 includes a computing unit 401, which can execute various appropriate actions and processes according to computer programs stored in a ROM (Read-Only Memory) 402 or computer programs loaded from a storage unit 408 into a RAM (Random Access Memory) 403. In the RAM 403, various programs and data required for the operation of device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0091] Multiple components in device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disc, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0092] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include but are not limited to a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 executes the various methods and processes described above, such as the method for dynamically adjusting file system parameters. For example, in some embodiments, the method for dynamically adjusting file system parameters can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method described above can be executed. Alternatively, in other embodiments, the computing unit 401 can be configured to execute the aforementioned method for dynamically adjusting file system parameters in any other appropriate way (e.g., by means of firmware).

[0093] The various embodiments of the systems and techniques described above in this document may be implemented in digital electronic circuitry, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SoCs (System On Chip), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0094] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0095] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory), or a flash memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0096] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or an LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0097] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a blockchain network.

[0098] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS" for short). The server may also be a server of a distributed system or a server combined with a blockchain.

[0099] Among them, it should be noted that artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and there are both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, technology for dynamically adjusting file system parameters, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0100] It should be understood that various forms of processes shown above can be used, reordering, adding, or deleting steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0101] The above specific implementation manners do not constitute a limitation to the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A method for dynamically adjusting file system parameters, characterized in that Including: Determine the relationship between the parameters of the file system and the performance metrics under the access of different applications; Obtain sample data, label the sample data to obtain labeled data, and determine the sample data and the labeled data as the training dataset, where the sample data includes the parameters of the file system, the performance metrics having the relationship with the parameters of the file system, and the access patterns of the applications, and the labeled data is used to label whether the performance metrics in the sample data reach the optimized state; Train the training dataset through multinomial logistic regression to obtain a trained logistic regression model; Under the access of different applications, dynamically adjust the parameter values of the file system based on the logistic regression model.

2. The method according to claim 1, wherein The dynamically adjusting the parameter values of the file system based on the logistic regression model includes: Input the parameter values to be optimized into the logistic regression model for processing to obtain the predicted values of the performance metrics corresponding to the parameter values; According to the predicted values, determine whether the values of the performance metrics corresponding to the parameter values reach the optimized state; In response to the values of the performance metrics corresponding to the parameter values not reaching the optimized state, adjust the parameter values, and input the adjusted parameter values into the logistic regression model for processing until the values of the performance metrics corresponding to the adjusted parameter values reach the optimized state.

3. The method according to claim 1, characterized in that, The parameters of the file system include the block size and the buffer size, and the performance metrics include at least one of the number of read and write operations per second, the throughput, and the memory resource utilization rate. Determining the relationship between the parameters of the file system and the performance metrics under the access of different applications includes: Determine the relationship between the block size of the file system and the number of read and write operations per second and / or the throughput under the access of different applications; Determine the relationship between the buffer size of the file system and the memory resource utilization rate under different access patterns.

4. The method according to any one of claims 1-3, characterized in that The obtaining sample data, labeling the sample data to obtain labeled data, and determining the sample data and the labeled data as the training dataset includes: For each access of a different application, determine at least one of the multiple performance metrics having the relationship with the parameters of the file system as the performance optimization metric; Obtain sample data, use the parameters of the file system, the performance optimization metric, and the label as the table headers, and fill the sample data into the corresponding columns according to the table headers to generate an initial table; Determine the label values of the performance optimization metric in the access pattern, where the label values include a first value and a second value, the first value is used to indicate that the value of the performance optimization metric has reached the optimized state, and the second value is used to indicate that the value of the performance optimization metric has not reached the optimized state; Fill the label values into the column corresponding to the label in the initial table to generate a target table, and determine the target table as the training dataset.

5. The method according to claim 1, wherein After training the training dataset through multinomial logistic regression to obtain a trained logistic regression model, the method further includes: Obtain a validation dataset; Evaluate the logistic regression model using the validation dataset and adjust the logistic regression model.

6. The method according to claim 5, wherein Evaluating the logistic regression model using the validation dataset and adjusting the logistic regression model includes: Determining the accuracy rate of the logistic regression model according to the validation dataset and the prediction results of the logistic regression model; Adjusting the hyperparameters of the logistic regression model according to the accuracy rate.

7. An apparatus for dynamically adjusting file system parameters, characterized in that, Including: A determining module, configured to determine the relationship between the parameters of the file system and the performance metrics under the access of different applications; An obtaining module, configured to obtain sample data, label the sample data to obtain labeled data, and determine the sample data and the labeled data as a training dataset, where the sample data includes the parameters of the file system, the performance metrics having the relationship with the parameters of the file system, and the access patterns of the applications, and the labeled data is used to label whether the performance metrics in the sample data reach the optimized state; A training module, configured to train the training dataset through multinomial logistic regression to obtain a trained logistic regression model; A dynamic adjustment module, configured to dynamically adjust the parameter values of the file system based on the logistic regression model under the access of different applications.

8. An electronic device, characterized in that, Including: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

Images