Method, apparatus and computer program product for a file system
By monitoring client access times in NFS and updating delegation enable status, NFS performance issues and performance degradation caused by frequent recalls are solved, achieving more efficient I/O operation and user experience.
Patent Information
- Application Number
- CN202310640544.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
Performance issues with existing network file systems (NFS) can lead to inconsistent or loss of data when frequently reading or writing large amounts of data, and frequent recalls of delegations can degrade I/O operation performance.
Automatically enable or disable delegation to improve I/O by monitoring client access to the file system by the delegation analyzer at the server, determining the recall ratio, and updating the delegation enabled status of the file system based on the recall ratio and the enable delegation threshold. Operating performance.
Improves I/O operation performance, reduces the consumption of client and server resources, reduces system latency, and improves user experience.
Smart Images

Figure CN119071309A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data storage, and more particularly, to methods, devices, and computer program products for a file system. Background Art
[0002] Network File System (NFS) is a distributed file system protocol in which client hosts can access server-side files in the same way as accessing local storage. NFS implements data storage, access, and sharing in computer networks, and allows storage resources (e.g., files) to be provided to multiple clients over the network, enabling them to access and manage data in a distributed and shared manner. NFS provides a high-level file access abstraction and supports operations such as reading, writing, creating, and deleting files. NFS is widely used in distributed computing environments, storage systems, cloud computing, and other scenarios.
[0003] Optimizing the performance of the NFS file system can reduce file access latency and response time and increase data transfer speed. This is especially important for applications or users that need to frequently read or write large amounts of data, and can improve work efficiency and productivity. Performance issues may lead to inconsistent or lost data. By improving the performance of the NFS file system, errors and losses during data transfer can be reduced, ensuring data consistency and reliability. Summary of the invention
[0004] The embodiments of the present disclosure provide a method, device and computer program product for a file system. In the embodiments of the present disclosure, the recall ratio within a predetermined time period in the past can be determined, and then the delegation enablement state of the file system can be updated by comparing the recall ratio and the delegation enablement threshold. In this way, the performance of I / O operations can be improved, client resources and server resources can be saved, system delays can be reduced, and user experience can be improved.
[0005] In a first aspect of an embodiment of the present disclosure, a method for a file system is provided. The method includes determining a delegation enablement state of the file system. The method also includes determining the number of multiple access requests from multiple clients to files in the file system within a predetermined time period. The method also includes determining a recall ratio within the predetermined time period based on the delegation enablement state and the number of multiple access requests. In addition, the method also includes updating the delegation enablement state of the file system by comparing the recall ratio with an enablement delegation threshold.
[0006] In a second aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement a method, the method including determining the delegation enablement state of the file system. The method also includes determining the number of multiple access requests to files in the file system by multiple clients within a predetermined time period. The method also includes determining a recall ratio within a predetermined time period based on the delegation enablement state and the number of multiple access requests. In addition, the method also includes updating the delegation enablement state of the file system by comparing the recall ratio and the delegation enablement threshold.
[0007] In a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, a method is implemented, the method comprising determining a delegation enablement state of a file system. The method further comprises determining the number of multiple access requests to files in the file system from multiple clients within a predetermined time period. The method further comprises determining a recall ratio within a predetermined time period based on the delegation enablement state and the number of multiple access requests. In addition, the method further comprises updating the delegation enablement state of the file system by comparing the recall ratio with a delegation enablement threshold.
[0008] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:
[0010] Figure 1 A schematic diagram illustrating an example environment in which various embodiments of the present disclosure may be implemented;
[0011] Figure 2 A schematic diagram showing a process of granting delegation and recalling delegation according to some embodiments of the present disclosure;
[0012] Figure 3 A flowchart of a method for a file system according to some embodiments of the present disclosure is shown;
[0013] Figure 4A A schematic diagram showing an example of a method suitable for initiating delegation when multiple clients access multiple files in a file system when delegation is not enabled according to some embodiments of the present disclosure;
[0014] Figure 4B A schematic diagram showing an example of an inappropriate initiation of delegation when multiple clients access multiple files in a file system when delegation is not enabled according to some embodiments of the present disclosure;
[0015] Figure 5 A schematic diagram showing an example of multiple clients accessing multiple files in a file system when delegation is enabled according to some embodiments of the present disclosure;
[0016] Figure 6 A schematic diagram showing a process of enabling or disabling delegation by dynamically determining a threshold for enabling delegation according to some embodiments of the present disclosure;
[0017] Figure 7 A schematic diagram showing a process of determining a threshold for enabling delegation according to some embodiments of the present disclosure;
[0018] Figure 8 A block diagram of a device in which various embodiments of the present disclosure may be implemented is shown. DETAILED DESCRIPTION
[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.
[0021] In some versions of NFS, the server can use "delegation" to delegate the management of files to the client. For example, the server can grant a read delegation or a write delegation to the client. Read delegation can be granted to multiple clients at the same time because these read delegations do not conflict with each other. Write delegation can only be granted to one client because the write delegation conflicts with any file access by any other client. While holding a write delegation, the client will not send I / O requests to the server because the client can guarantee exclusive access to the file. Similarly, the client will not send I / O requests to the server while holding a read delegation. The reason is that the server guarantees that no client can open the file in write mode. Delegation can greatly reduce the interaction between the server and the client on the delegated file. Therefore, the consumption of network resources is reduced and the performance of the client and the server is improved.
[0022] In some cases, the server can "recall" a delegation. A conflict occurs when a client accesses a file in a way that is inconsistent with the delegation currently granted to the file. For example, if a client holds a write delegation for a file and another client opens the file for read or write access, the server recalls the write delegation of the first client. Similarly, if a client holds a read delegation and another client opens the same file for write, the server recalls the read delegation of the first client. When a conflict occurs, the server uses a callback mechanism to contact the client that currently holds the delegation. After receiving this callback, the client sends the updated status of the file to the server and returns the delegation. The purpose of designing delegation is to reduce the interaction between the server and the client, thereby improving the performance of I / O operations. However, the recall process takes more time than normal I / O operations, so frequent recalls may cause the performance of I / O operations to degrade, or even make the performance of I / O operations lower than that of I / O operations without the delegation function.
[0023] To this end, an embodiment of the present disclosure proposes a scheme for updating the delegation enabling state of a file system. In an embodiment of the present disclosure, a delegation analyzer located at a server can monitor data associated with a given file system in a predetermined time period in the past, for example, the number of accesses of each client requesting to access a file in the file system, and then determine the recall ratio in the predetermined time period based on the monitored data. The scheme updates the delegation enabling state of the file system by comparing the recall ratio and the enabling delegation threshold.
[0024] In this way, the solution provided by the embodiments of the present disclosure can automatically enable or disable delegation for a file system according to different situations in which the client accesses the file system, thereby utilizing delegation to improve I / O operation performance, while also reducing the degradation of I / O operation performance caused by enabling delegation, reducing the consumption of client resources and server resources, reducing system latency, and improving user experience.
[0025] Figure 1 1 is a schematic diagram of an example environment 100 in which various embodiments of the present disclosure may be implemented. Figure 1 As shown, environment 100 includes server 102 and multiple clients, namely clients 110-1, 110-2, ..., 110-N (collectively referred to as client 110). Server 102 includes multiple file systems, namely file systems 104-1, 104-2, ..., 104-N (collectively referred to as file system 104). Multiple files are stored in file system 104. For example, in environment 100, file 106-1, 106-2, ..., 106-N (collectively referred to as file 106) are stored in file system 104-1. Client 110 can send access requests such as reading files, writing files, creating files, deleting files, etc. to file 106 in file system 104 to server 102. The access request usually includes a file path and related operations. When delegation is not enabled, after receiving the access request, server 102 can perform corresponding operations on file 106 in file system 104, and then return the result to client 110. When delegation is enabled, the server 102 can grant delegation to the client 110, so that the client 110 can autonomously process the read and write operations on the file 106 within a certain time range, and synchronize with the server only when the delegation times out or needs to share access with other clients, without frequently communicating with the server 102. This can reduce network latency and bandwidth consumption and improve the performance of I / O operations.
[0026] In some cases, the server 102 may recall the delegation of the file previously granted to the client. For example, after the server 102 grants the client 110-1 the delegation of the file 106-1 high in the file system 104-1, when the client 110-2 also requests to access the file 106-1, there may be a conflict, so the server 102 may recall the delegation of the file 106-1 previously granted to the client 110-1, and then the server 102 may grant the client 110-2 the delegation of the file 106-1.
[0027] like Figure 1As shown, in the environment 100, the server 102 also includes a delegation analyzer 108, which can collect statistical data, such as the number of times the file system 104 is accessed per second, the number of times the file 106 is accessed by each of the multiple clients 110 within a period of time, the number of times the delegation is recalled when the delegation is enabled, etc. The delegation analyzer 108 can also analyze these statistical data to make a decision on whether delegation needs to be enabled or disabled for a certain file system 104. By collecting and analyzing I / O access data over a period of time in the past through the delegation analyzer 108, and automatically making a decision on enabling or disabling delegation for the file system 104, the interaction between the server 102 and the client 110 can be reduced, thereby improving the performance of I / O operations and saving network resources.
[0028] Figure 2 FIG. 2 is a schematic diagram showing a process 200 of granting and recalling delegation according to some embodiments of the present disclosure. Figure 2 As shown, the server 202 (eg, Figure 1 A file 208 is stored in a file system of a server 102 in the process 200, and the server 202 has enabled delegation. Figure 1 The client 110-1 in the example needs to perform an I / O operation on the file 208, so the client 204 can send an access request to the server 202 for the file 208. After receiving the access request, the server 202 can grant the client 204 a delegation for the file 208. Then, the client 204 can save the local copy 210 of the file 208 locally, so that when the client 204 needs to perform an I / O operation on the file 208 in the next period of time, the I / O operation can be performed directly on the local copy 210 without sending an access request to the server 202.
[0029] exist Figure 2 In the illustrated process 200, while client 204 holds a delegation for file 208, client 206 (e.g., Figure 1The client 110-2 in the example needs to perform an I / O operation on the file 208, so the client 206 can send an access request to the file 208 to the server 202. After the server 202 receives the access request, the server 202 finds that the client 204 is holding the delegation for the file 208, so the server 202 can use the callback mechanism to recall the delegation for the file 208 from the client 204. In the process 200, after receiving the recall, the client 204 returns the updated local copy 210 to the server 202, and the server 202 can use the received updated file to replace the currently stored file 208, so that the file 208 is replaced with the updated file 212. Then, the server 202 can delegate the updated file 212 to the client 206. Then, the client 206 can save the updated local copy 214 of the file 212 locally.
[0030] like Figure 2 As shown, in process 200, from client 206 sending an access request for file 208 to server 202 to obtaining updated file 212, a total of four communications are performed between server 202, client 204, and client 206, i.e., client 206 sends an access request to server 202, server 202 sends a recall command to client 204, client 204 returns an updated local copy 210 to server 202, and server 202 sends updated file 212 to client 206. However, if server 202 does not enable delegation, the process only requires two communications between server 202 and client 206, i.e., client 206 sends an access request for file 208 to server 202, and server 202 returns the result of the access request to client 206.
[0031] Figure 3 FIG. 3 is a flow chart of a method 300 for identity authentication according to some embodiments of the present disclosure. Figure 3 As shown, at block 302, method 300 may determine a delegation enabling state of a file system. For example, the delegation enabling state of the file system may be enabled or disabled. At block 304, method 300 may determine a number of access requests by multiple clients to a file in the file system within a predetermined time period. For example, Figure 1 In the environment 100 shown, the method 300 can determine the number of times the client 110-1 requested access to the file 106-1 in the file system 104-1, the number of times the client 110-1 requested access to the file 106-2, the number of times the client 110-2 requested access to the file 106-1, the number of times the client 110-2 requested access to the file 106-2, and so on, within a predetermined time period in the past.
[0032] At block 306, the method 300 may determine a recall ratio within a predetermined time period based on the delegation enablement state and the number of multiple access requests. For example, if the delegation enablement state of the file system is enabled, the method 300 may determine a ratio of the number of recalls that actually occurred to the total number of access requests within the past predetermined time period. If the delegation enablement state of the file system is disabled, the method 300 may predict a ratio of the number of recalls that may have occurred to the total number of accesses within the past predetermined time period if delegation was enabled based on the number of times that files in the file system were requested to be accessed by various clients.
[0033] In box 308, method 300 can update the delegation enablement state of the file system by comparing the recall ratio and the delegation enablement threshold. For example, when the recall ratio is greater than the delegation enablement threshold, it means that the number of recalls is large, resulting in a decrease in the I / O operation performance of the entire environment. Therefore, when the delegation enablement state of the file system is enabled, it can be updated to disabled, or when the delegation enablement state is disabled, it can be kept in a disabled state. On the contrary, when the recall ratio is less than or equal to the delegation enablement threshold, it means that the number of recalls is small, so when the delegation enablement state of the file system is disabled, it can be updated to enabled, thereby improving the I / O operation performance, or when the delegation enablement state is enabled, it can be kept in an enabled state.
[0034] In this way, method 300 can automatically enable or disable delegation for the file system according to different situations in which the client accesses the file system, thereby utilizing delegation to improve the performance of I / O operations, while also reducing the degradation of I / O operation performance caused by enabling delegation, reducing the consumption of client resources and server resources, reducing system latency, and improving user experience.
[0035] In some embodiments, if the delegation enablement state of the file system is disabled, then in order to estimate the predicted recall ratio of enabling delegation within a predetermined time period, the predicted total recall number within the predetermined time period can be estimated based on the number of access requests from multiple clients to multiple files in a given file system within the predetermined time period. In these embodiments, the total number of access requests within the predetermined time period can also be determined. Then, the predicted recall ratio can be determined based on the predicted total recall number and the total number of access requests. In some embodiments, in response to the recall ratio being less than or equal to the delegation enablement threshold, the delegation enablement state of the file system can be updated to enabled.
[0036] In some embodiments, in order to estimate the predicted total recall times within a predetermined time period, for each file in a given file system, the discrete degree of the number of access requests of each client to the file can be determined, and the predicted recall times for the file can be determined based on the discrete degree. Then, the predicted total recall times can be determined by determining multiple predicted recall times for multiple files.
[0037] In some embodiments, the degree of dispersion of the number of access requests by each client to a given file can be determined by calculating the standard deviation of the number of access requests by each client to the given file. In these embodiments, the average number of access requests by multiple clients to the file can be determined. Then, the standard deviation of the number of access requests by each client to the file is determined based on the average number of access requests.
[0038] To estimate the recall ratio when delegation is enabled when delegation is not enabled, a delegation analyzer (e.g. Figure 1 The delegate analyzer 108 in the storage system needs to collect data related to the file system, such as the number of clients, the number of I / O requests per second of the file system, etc. Assume that there are K servers in the storage system, each server k includes N file systems, and for each file system fs i , the client connected to the file system is C i,j , then connect to the file system fs i The number of clients is NC i , where 1≤j≤NC i . Client j has access to the file system fs i The number of access requests is CIO i,j , file system fs i The number of access requests per second is f si_iops , then the following equation (1) can be used to obtain the file system fs in the past predetermined time period i Total number of access requests FIO i :
[0039]
[0040] M represents the file system fs i The number of files on the im represents each file, where 1≤m≤M. For each file f im , whose access requests come from different clients C i,j , then for file f im The number of access requests can be expressed as a sequence The following equation (2) can be used to determine the file f im Total number of access requests IOf im_total :
[0041]
[0042] In some embodiments, in order to determine the multiple clients' requests for the file f im The distribution of the number of access requests can be calculated by the following equation (3): im The standard deviation σ im :
[0043]
[0044] where f im_average Indicates that multiple clients have access to file f im The average number of access requests.
[0045] Standard deviation im It can represent multiple clients for file f im The discrete degree of the number of access requests. im Afterwards, the predicted recall ratio R when delegation is enabled within the predetermined time period can be calculated by the following equation (4): i :
[0046]
[0047] If the file system fs i If delegation is enabled, different clients will request to access files on the file system alternately. im Therefore, multiple clients may have more recalls for file f im The standard deviation of the number of access requests σ im The smaller it is, the better the file f is. im The more evenly the number of access requests is distributed among multiple clients C i,j The higher the recall rate, the higher the recall rate may be. In this case, for the file system fs i Disabling delegation can achieve better I / O performance. im The standard deviation of the number of access requests σ im The larger the value, the more im The more unevenly the number of access requests is distributed among multiple clients C i,j The higher the recall rate, the lower the recall rate may be. In this case, for the file system fs i Enabling delegation can improve the performance of I / O operations.
[0048] FIG. 4A to FIG. 4B A schematic diagram showing an example of multiple clients accessing multiple files in a file system when delegation is not enabled according to some embodiments of the present disclosure. Figure 4AA schematic diagram of an example 400 suitable for initiating delegation is shown for multiple clients accessing multiple files in a file system when delegation is not enabled according to some embodiments of the present disclosure. Figure 4B A schematic diagram is shown of an example 420 of multiple clients accessing multiple files in a file system that is not suitable for initiating delegation when delegation is not enabled according to some embodiments of the present disclosure.
[0049] like Figure 4A As shown, example 400 includes file system 402, and clients 404, 406, and 408, wherein file system 402 includes files 410 and 412. Within a predetermined period of time in the past, three access requests to file 410 (i.e., access requests 413, 414, and 415) are all from client 404, and three access requests to file 412 (i.e., access requests 416, 417, and 418) are all from client 406. In example 400, client 408 may have requested access to other files in file system 402, but has not requested access to files 410 and 412. It should be understood that for the sake of simplicity, in Figure 4A Other files in the file system 402 and other clients accessing the file system 402 are not shown, but it is not intended to limit the number of files included in the file system, the number of clients, or the number of access requests.
[0050] exist Figure 4AIn the example 400 shown, for file 410, the delegation analyzer 108 can calculate the average number of access requests from clients 404, 406 and 408 to file 410 based on the number of clients (i.e., 3 clients, namely clients 404, 406 and 408), the number of access requests from each client to file 410 (i.e., client 404 is 3 times, requesting 413, 414 and 415 respectively, and clients 406 and 408 are 0), and then use the above equation (3) to calculate the standard deviation of the number of access requests from clients 404, 406 and 408 to file 410. Similarly, in example 400, for file 412, delegation analyzer 108 may calculate the average number of access requests by clients 404, 406, and 408 to file 412 based on the number of clients (i.e., 3 clients, namely, clients 404, 406, and 408), the number of access requests by each client to file 412 (i.e., client 406 has 3 times, namely, requests 416, 417, and 418, and clients 404 and 408 have 0 times), and then use equation (3) above to calculate the standard deviation of the number of access requests by clients 404, 406, and 408 to file 412. Then, delegation analyzer 108 may use equation (4) above to calculate the predicted recall ratio if delegation is enabled for file system 402 within the past predetermined period of time based on the number of access requests by clients 404, 406, and 408 to files 410 and 412 and the standard deviation. If the predicted recall ratio is less than or equal to the delegation enablement threshold, the delegation enablement state of the file system 402 may be updated to enabled.
[0051] exist Figure 4A In the example 400 shown, it can be seen that the access requests for file 410 are not uniformly all from client 404, in which case the standard deviation of the number of access requests for file 410 from clients 404, 406, and 408 is large. Similarly, in example 400, the standard deviation of the number of access requests for file 412 from clients 404, 406, and 408 is also large. If delegation is enabled for file system 402, fewer recalls will occur, so enabling delegation can improve the I / O operation performance of file system 402.
[0052] Figure 4B A schematic diagram of an example 420 of an inappropriate initiation of delegation when multiple clients access multiple files in a file system when delegation is not enabled according to some embodiments of the present disclosure is shown. Figure 4BAs shown, example 420 includes a file system 422 and clients 424, 426, and 428, wherein file system 422 includes files 430 and 432. Within a predetermined period of time in the past, three access requests to file 430 (i.e., access requests 433, 434, and 435) are sequentially from clients 424, 426, and 428, and three access requests to file 432 (i.e., access requests 436, 437, and 438) are also sequentially from clients 424, 426, and 428. It should be understood that for the sake of simplicity, in Figure 4B Other files in the file system 422 and other clients accessing the file system 422 are not shown, but it is not intended to limit the number of files included in the file system, the number of clients, or the number of access requests.
[0053] exist Figure 4B In the example 420 shown, for file 430, the delegation analyzer 108 can calculate the average number of access requests from clients 424, 426, and 428 to file 430 based on the number of clients (i.e., 3 clients, namely clients 424, 426, and 428), the number of access requests from each client to file 430 (i.e., 1 time for clients 424, 426, and 428), and then use the above equation (3) to calculate the standard deviation of the number of access requests from clients 424, 426, and 428 to file 430. Similarly, in example 420, for file 432, the delegation analyzer 108 may calculate the average number of access requests by clients 424, 426, and 428 to file 432 based on the number of clients (i.e., 3 clients, namely, clients 424, 426, and 428), the number of access requests by each client to file 432 (i.e., clients 424, 426, and 428 are all 1 time), and then use the above equation (3) to calculate the standard deviation of the number of access requests by clients 424, 426, and 428 to file 432. Then, the delegation analyzer 108 may calculate the predicted recall ratio if delegation is enabled for file system 422 within the past predetermined time period based on the number of access requests by clients 424, 426, and 428 to files 430 and 432 and the standard deviation, using the above equation (4). If the predicted recall ratio is less than or equal to the delegation enabling threshold, the delegation enabling state of file system 422 may be updated to enabled.
[0054] exist Figure 4BIn the example 420 shown, it can be seen that the access requests for file 430 are uniformly from clients 424, 426, and 428. In this case, the standard deviation of the number of access requests from clients 424, 426, and 428 to file 430 is small. Similarly, in example 400, the standard deviation of the number of access requests from clients 424, 426, and 428 to file 432 is also small. If delegation is enabled for file system 402, more recalls will occur, so disabling delegation can reduce the degradation of the I / O operation performance of file system 402.
[0055] In this way, the delegation analyzer can automatically determine whether enabling delegation will improve or reduce the I / O operation performance of the file system, and thus can improve the I / O operation performance of the file system or reduce the I / O operation performance of the file system by dynamically adjusting the delegation enabling status of the file system, thereby saving network resources, reducing latency, and improving user experience.
[0056] In some embodiments, if it is determined that the delegation enabling state of the file system is enabled, the number of multiple access requests from multiple clients to a file in the file system within a predetermined time period can be determined, as well as the number of recalls associated with the file within the predetermined time period. Then, the actual recall ratio can be determined based on the number of multiple access requests and the number of recalls. In some embodiments, in response to the recall ratio being greater than the enabling delegation threshold, the delegation enabling state of the file system can be updated to disabled.
[0057] In some embodiments, the total number of access requests to files in the file system by multiple clients within a predetermined time period may be determined. total , and the total number of recalls IO recall Then, the file system f when delegation is enabled can be calculated by the following equation (5): si The actual recall ratio R i :
[0058]
[0059] Figure 5 A schematic diagram of an example 500 of multiple clients accessing multiple files in a file system when delegation is enabled according to some embodiments of the present disclosure. Figure 5 As shown, example 500 includes a file system 502, and clients 504 and 506, wherein file system 502 includes file 1 508 and file 2 510. Within a predetermined period of time in the past, client 504 requested access to file 1 508, and then the server (e.g., Figure 1In the example 500, the server 102 in the file system 502 may grant the client 504 a delegation of file 1 508, thereby generating a local copy 512 of file 1 508 at the client 504. Then, the client 506 also requests access to file 1 508, so the server sends a command to the client 504 to recall the delegation of file 1. After receiving the recall command, the client 504 returns the updated local copy 512 to the server, so the file 1 508 at the file system 502 is updated to file 1 514. Then, the server may grant the client 506 a delegation of file 1 514. In the example 500, the client 1 may also request access to file 2 510 from the server, and the server grants the client 504 a delegation of file 2 510, and generates a local copy 516 of file 2 510 at the client 504. After the delegations granted by the server to clients 504 and 506 time out, client 506 returns updated file 1 518 , and client 504 returns updated file 2 520 .
[0060] exist Figure 5 In the illustrated example 500, a delegate analyzer (e.g., Figure 1 The delegation analyzer 108 in the example can determine the total number of access requests by clients 504 and 506 to file 1 508 and file 2 510 in the file system 502 during the time period (i.e., a total of 3 requests, of which client 504 requested 2 times and client 506 requested 1 time), as well as the total number of recalls (i.e., a total of 1 recall). Then, the delegation analyzer can determine the actual recall ratio of the file system 502 based on the total number of access requests and the total number of recalls using the above equation (5). If the actual recall ratio is greater than the delegation enabling threshold, the delegation analyzer can update the delegation enabling state of the file system 502 to disabled, thereby improving the I / O operation performance of the file system 502.
[0061] In this way, the delegation analyzer can automatically determine whether disabling delegation will improve or reduce the I / O operation performance of the file system, thereby improving the I / O operation performance of the file system or reducing the reduction of the I / O operation performance of the file system by dynamically adjusting the delegation enabling state of the file system.
[0062] In some embodiments, the delegation analyzer may dynamically determine the enable delegation threshold based on data associated with the file system over a past predetermined period of time. Figure 6 FIG. 6 is a schematic diagram of a process 600 for enabling or disabling delegation by dynamically determining a threshold for enabling delegation according to some embodiments of the present disclosure. Figure 6As shown, at block 602, process 600 determines whether the delegation enabling state of the file system is enabled or disabled. At block 604, if process 600 determines that the delegation enabling state of the file system is disabled, it moves to block 606.
[0063] At block 606, process 600 monitors data associated with the file system in the past predetermined time period when delegation is disabled. For example, the delegation analyzer may monitor the number of files in the file system, the number of clients connected to the file system, the number of access requests from each client to access each file in the file system, the number of access requests per second of the file system, etc. At block 608, the delegation analyzer may calculate the predicted recall ratio if delegation is enabled for the file system in the past predetermined time period based on the data monitored at block 608 using equation (4) above.
[0064] Returning to box 604, if the process 600 determines that the delegation enabling state of the file system is enabled, it moves to box 618. In box 618, the process 600 monitors the data associated with the file system in the past predetermined time period when delegation is enabled. For example, the delegation analyzer can monitor the number of access requests for files in the file system sent by each client to the server in the past predetermined time period, and the number of recalls sent by the server to each client, etc. In box 620, the process 600 can determine the actual recall ratio associated with the file system in the past predetermined time period based on the number of multiple access requests and the number of recalls collected in box 618. For example, the delegation analyzer can calculate the total number of access requests sent by all clients based on the number of access requests for files in the file system sent by each monitored client to the server. In addition, the delegation analyzer can also calculate the total number of recalls based on the number of high recalls sent by the monitored server to each client. Then, the delegation analyzer can calculate the actual recall ratio associated with the file system based on the total number of access requests and the total number of recalls using the above equation (5).
[0065] At block 610, process 600 may determine a threshold for enabling delegation based on a predicted recall rate or an actual recall rate (collectively referred to as a recall rate) over a predetermined period of time in the past and monitored data associated with the file system. The detailed process of determining the threshold for enabling delegation will be described below in conjunction with Figure 7Detailed description is given. At box 612, process 600 can compare the recall ratio and the delegation enablement threshold. If the recall ratio is greater than the delegation enablement threshold, it means that enabling delegation may result in more recalls, so that the I / O operation performance of the file system may be reduced, so at box 614, process 600 can update the delegation enablement state to disabled or remain disabled. On the contrary, if the recall ratio is less than or equal to the delegation enablement threshold at box 612, it means that enabling delegation may result in fewer recalls, so that the I / O operation performance of the file system may be improved, so at box 616, process 600 can update the delegation enablement state of the file system to enabled or remain enabled.
[0066] In this way, the delegation enabling threshold can be dynamically determined based on data within a predetermined time period in the past, so that the delegation analyzer can make appropriate decisions based on the latest data, thereby improving the accuracy of updating the delegation enabling status of the file system, improving I / O operation performance or reducing I / O operation performance.
[0067] In order to determine the threshold value for enabling delegation, in some embodiments, the threshold value for enabling delegation can be determined based on the request time consumed by sending the access request, the response time consumed by receiving the request response, the number of total access requests of the file system within a predetermined time period, the number of requests per second of the file system within a predetermined time period, and the delegation timeout period. In some embodiments, the total time consumed by multiple access requests to files in the file system when the delegation of the file system is not enabled within the predetermined time period (also referred to as the first consumption time) can be determined. Then, the total time consumed by multiple access requests to files in the file system when the delegation of the file system is enabled within the predetermined time period (also referred to as the second consumption time) can be determined. Then, the threshold value for enabling delegation can be determined based on the consumption time in the above two situations.
[0068] In some embodiments, the total time consumed by all access requests when delegation is disabled may be determined based on the request time consumed to send the access request, the response time consumed to receive the request response, and the number of total access requests within a predetermined time period.
[0069] In some embodiments, in order to determine the total time consumed by all access requests when delegation is enabled, the total time consumed by access requests that are recalled (also called the third consumed time) and the total time consumed by access requests that are not recalled (also called the fourth consumed time) can be determined, and then the total time consumed by all access requests when delegation is enabled can be determined based on the time consumed by these two parts of the access requests.
[0070] In some embodiments, to determine the total time consumed by access requests that are recalled, the third consumed time may be determined based on the time consumed by the request, the time consumed by the response, and the number of access requests that are recalled. In some embodiments, to determine the total time consumed by access requests that are not recalled, the total time consumed by access requests that are not recalled may be determined based on the time consumed by the request, the time consumed by the response, the number of requests per second, the delegation timeout, and the number of access requests that are not recalled.
[0071] Figure 7 A schematic diagram of process 700 for determining a threshold for enabling delegation according to some embodiments of the present disclosure is shown. In box 702, process 700 can determine the total time consumed by access requests to files in the file system without enabling delegation. For example, without enabling delegation, the time consumed by the client to send an access request to the file to the server is t1, and the time consumed by the client to receive a response to the access request from the server is t2. Usually t1 is close to t2, so in order to make the equation simple, it is assumed in this article that t1 and t2 are both equal to t, so the time consumed by the client to obtain file permissions is 2t, and the total consumed time t of the file system can be calculated by the following equation (6): wo :
[0072] t wo =FIO i *2t (6)
[0073] Among them FIO i Is the total number of access requests to the file system.
[0074] It should be understood that in the event that t1 is not equal to t2, the equations listed herein may continue to be utilized by replacing 2t with t1 + t2.
[0075] At block 704, process 700 may determine the time consumed by the access request that is recalled from all access requests when delegation is enabled. For example, the total time consumed by the access request that is recalled may be determined by equation (7): w_r :
[0076] t w_r =4t*FIO i *R i (7)
[0077] Where R i is the recall ratio.
[0078] At block 706, process 700 may determine the time consumed by access requests that have not been recalled among all access requests when delegation is enabled. For example, the total time t consumed by access requests that have not been recalled may be determined by the following equation (8): w_wor :
[0079]
[0080] Where T is the delegation timeout. The delegation timeout refers to the time that the client can maintain the delegation after obtaining the file delegation. When the client maintains the delegation for more than the delegation timeout T, the client needs to request access to the file from the server again. si_iops Is the file system f si The number of access requests per second.
[0081] At block 708, process 700 may calculate the total time t consumed by all access requests without enabling delegation. wo , the total time t consumed by access requests that are recalled when delegation is enabled w_r , and the total time t consumed by access requests without recall when delegation is enabled w_wor To determine the threshold for enabling delegation. For example, when the recall ratio R i Make the total time consumed by all access requests when delegation is enabled equal to the total time consumed by all access requests when delegation is not enabled (that is, (t w_r +t w_wor ) = t wo ), the I / O operation performance with delegation enabled is the same as the I / O operation performance with delegation disabled. Based on this condition, the recall ratio at the inflection point can be determined by the following equation (9), i.e., the delegation enabling threshold R i_tp :
[0082]
[0083] From the above equations (6), (7) and (8), we can see that when the recall ratio R i Greater than R i_tp When the recall ratio R is , the total time consumed by all access requests when delegation is enabled is greater than the total time consumed by all access requests when delegation is not enabled. Therefore, enabling delegation will lead to a decrease in I / O operation performance. On the contrary, when the recall ratio R i Less than R i_tp When delegation is enabled, the total time consumed by all access requests is less than the total time consumed by all access requests when delegation is not enabled, so enabling delegation will improve I / O operation performance.
[0084] In this way, the threshold for enabling delegation can be accurately determined, thereby improving the accuracy of the delegation analyzer's decision to enable or disable delegation, thereby improving I / O operation performance, saving network resources, reducing latency, and improving user experience.
[0085] Figure 8 A schematic block diagram of an example device 800 that can be used to implement an embodiment of the present disclosure is shown. As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0086] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0087] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as method 300. For example, in some embodiments, the method 300 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the method 300 described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the method 300 in any other appropriate manner (e.g., by means of firmware).
[0088] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.
[0089] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0090] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In addition, although each operation is depicted in a specific order, this should be understood as requiring such operations to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination.
[0091] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.
Claims
1. A method for a file system, comprising: Determine the delegation-enabled state of the file system; Determining the number of multiple access requests to files in the file system by multiple clients within a predetermined time period; determining a recall ratio within the predetermined time period based on the delegation enablement state and the number of the plurality of access requests; The delegation enabling state of the file system is updated by comparing the recall ratio with a delegation enabling threshold.
2. The method according to claim 1, wherein determining the recall ratio within the predetermined time period comprises: In response to determining that the delegation enablement state of the file system is disabled: Determining a predicted total number of recalls within the predetermined time period based on the number of the plurality of access requests; determining a total number of access requests within the predetermined time period; as well as Based on the predicted total number of recalls and the total number of access requests, a predicted recall ratio is determined.
3. The method of claim 2, wherein determining the number of predicted total recalls within the predetermined time period comprises: For the file in the file system, determining a discrete degree of the number of multiple access requests to the file by the multiple clients; Based on the discreteness, determining a predicted recall number for the file; as well as The predicted total recall number is determined by determining one or more predicted recall numbers for one or more files in the file system.
4. The method according to claim 3, wherein determining the discrete degree of the number of times of the plurality of access requests of the plurality of clients to the file comprises: Determine an average number of access requests to the file by the plurality of clients; determining a standard deviation of the number of access requests from the plurality of clients to the file based on the average number of access requests, and The standard deviation is determined as the degree of dispersion.
5. The method of claim 4, wherein updating the delegation enabled state of the file system comprises: In response to the recall ratio being less than or equal to the delegation enabling threshold, updating the delegation enabling state of the file system to enabled.
6. The method of claim 1, wherein determining the recall ratio within the predetermined time period further comprises: In response to determining that the delegation enablement state of the file system is enabled: Determining the number of multiple access requests from the multiple clients to the file in the file system within a predetermined time period; determining a number of recalls associated with the document within the predetermined time period; as well as The recall ratio is determined based on the number of the multiple access requests and the number of recalls.
7. The method of claim 6, wherein updating the delegation enabled state of the file system comprises: In response to the recall ratio being greater than the delegation enabling threshold, updating the delegation enabling state of the file system to disabled.
8. The method according to claim 1, further comprising: The delegation enabling threshold is determined based on the request time consumed for sending the access request, the response time consumed for receiving the request response, the total number of access requests to the file system within the predetermined time period, the number of requests per second to the file system within the predetermined time period, and the delegation timeout.
9. The method of claim 8, wherein determining the enabling delegation threshold comprises: determining a first consumption time, the first consumption time being a total time consumed by a plurality of access requests to the file in the file system without enabling the delegation of the file system within the predetermined time period; determining a second consumed time, the second consumed time being a total time consumed by a plurality of access requests to the file in the file system when the delegation of the file system is enabled within the predetermined time period; as well as The enabling delegation threshold is determined based on the first elapsed time and the second elapsed time.
10. The method of claim 9, wherein determining the first elapsed time comprises: The first consumption time is determined based on the request time, the response time and the total number of access requests.
11. The method of claim 9, wherein determining the second elapsed time comprises: determining a third consumed time, the third consumed time being a total time consumed by the access request in which the recall occurs; determining a fourth consumed time, the fourth consumed time being a total time consumed by access requests for which no recall occurs; as well as The second elapsed time is determined based on the third elapsed time and the fourth elapsed time.
12. The method of claim 11, wherein determining the third elapsed time comprises: The third elapsed time is determined based on the request time, the response time, and the number of access requests for which recall occurs.
13. The method of claim 11, wherein determining the fourth elapsed time comprises: The fourth consumed time is determined based on the request time, the response time, the number of requests per second, the delegation timeout time, and the number of access requests without recall.
14. An electronic device comprising: at least one processor; as well as coupled to the at least one processor and having instructions stored thereon, the instructions, when executed by the at least one processor, causing the electronic device to perform actions, the actions comprising: Determine the delegation-enabled state of the file system; Determining the number of multiple access requests to files in the file system by multiple clients within a predetermined time period; determining a recall ratio within the predetermined time period based on the delegation enablement state and the number of the plurality of access requests; The delegation enabling state of the file system is updated by comparing the recall ratio with a delegation enabling threshold.
15. The electronic device of claim 14, wherein determining the recall ratio within the predetermined time period comprises: In response to determining that the delegation enablement state of the file system is disabled: Determining a predicted total number of recalls within the predetermined time period based on the number of the plurality of access requests; determining a total number of access requests within the predetermined time period; as well as Based on the predicted total number of recalls and the total number of access requests, a predicted recall ratio is determined.
16. The electronic device of claim 15, wherein determining the number of predicted total recalls within the predetermined time period comprises: For the file in the file system, determining a discrete degree of the number of multiple access requests to the file by the multiple clients; Based on the discreteness, determining a predicted recall number for the file; as well as The predicted total recall number is determined by determining one or more predicted recall numbers for one or more files in the file system.
17. The electronic device according to claim 16, wherein determining the discrete degree of the number of times of the plurality of access requests of the plurality of clients to the file comprises: Determine an average number of access requests to the file by the plurality of clients; determining a standard deviation of the number of access requests from the plurality of clients to the file based on the average number of access requests, and The standard deviation is determined as the degree of dispersion.
18. The electronic device of claim 17, wherein updating the delegation enabled state of the file system comprises: In response to the recall ratio being less than or equal to the delegation enabling threshold, updating the delegation enabling state of the file system to enabled.
19. The electronic device of claim 14, wherein determining the recall ratio within the predetermined time period further comprises: In response to determining that the delegation enablement state of the file system is enabled: Determining the number of multiple access requests from the multiple clients to the file in the file system within a predetermined time period; determining a number of recalls associated with the document within the predetermined time period; as well as The recall ratio is determined based on the number of the multiple access requests and the number of recalls.
20. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions that when executed cause a machine to perform actions comprising: Determine the delegation-enabled state of the file system; Determining the number of multiple access requests to files in the file system by multiple clients within a predetermined time period; determining a recall ratio within the predetermined time period based on the delegation enablement state and the number of the plurality of access requests; The delegation enabling state of the file system is updated by comparing the recall ratio with a delegation enabling threshold.
