Performance data acquisition method and device, computer equipment and storage medium

Abstract

The invention relates to a performance data acquisition method and device, computer equipment and a storage product. The method is applied to a server side in a distributed file system and comprises the steps that a performance data obtaining request sent by a manufacturer management node is responded, and the performance data obtaining request carries corresponding subdirectory information lists mounted under the server side by multiple different tenants in the distributed file system; and according to the subdirectory information list, obtaining performance data of corresponding subdirectories mounted by different tenants under each data storage server in the server, and sending the performance data of each subdirectory to the manufacturer management node. By adopting the method, the performance data of the distributed file system for providing the storage service for different tenants can be obtained from the fine granularity, namely, the performance data of the subdirectories of the different tenants can be obtained, and on the basis, reliable reference information can be provided for various storage service applications.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a performance data acquisition method, device, computer equipment and storage product. Background Art

[0002] With the development of cloud computing technology, cluster systems are being used more and more widely.

[0003] Taking the distributed file system as an example, in actual applications, each user usually obtains the distributed file system by leasing. Correspondingly, cloud storage vendors will allocate subdirectories to multiple different tenants based on the distributed file system to provide storage services. Therefore, it is particularly important to obtain the performance data of the subdirectories of different tenants.

[0004] However, the related art can only obtain the performance data of the read / write process, the client device or the same read / write tasks during the read / write operation requested by the client device, but cannot obtain the performance data of the sub-directories of different tenants. Summary of the invention

[0005] Based on this, it is necessary to provide a performance data acquisition method, device, computer equipment and storage product to address the above technical problems, which can obtain the performance data of the distributed file system providing storage services to different tenants in a fine-grained manner, that is, to obtain the performance data of the sub-directories of different tenants.

[0006] In a first aspect, an embodiment of the present application provides a method for obtaining performance data, which is applied to a server in a distributed file system, and the method includes:

[0007] Respond to the performance data acquisition request sent by the manufacturer management node; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system;

[0008] The performance data of the corresponding subdirectories mounted on each data storage server of different tenants on the server are obtained according to the subdirectory information list; the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read / write request sent by each client device during the process of requesting a read / write operation;

[0009] Send the performance data of each sub-directory to the manufacturer management node.

[0010] In one embodiment, the sub-directory information list includes a sub-directory identifier corresponding to each tenant; and obtaining performance data of corresponding sub-directories mounted on each data storage server in the server by different tenants according to the sub-directory information list includes:

[0011] For any sub-directory identifier, obtain the sub-directory mounted by the tenant corresponding to the sub-directory identifier on each data storage server;

[0012] The performance data of the corresponding subdirectories mounted on each data storage server is determined according to the subdirectories.

[0013] In one embodiment, determining performance data of corresponding subdirectories mounted on each data storage server according to the subdirectories includes:

[0014] Find subdirectories in the association relationship between different subdirectories and different performance data;

[0015] The performance data corresponding to the sub-directory in the association relationship is determined as the performance data of the sub-directory mounted by the tenant under each data storage server.

[0016] In one embodiment, the performance data includes read / write bandwidth and / or read / write throughput; and the process of obtaining the performance data of each subdirectory includes:

[0017] For any subdirectory, obtain the current duration between the subdirectory creation time and the current time, and the historical duration between the subdirectory creation time and the historical time;

[0018] Determine the read and write bandwidth of the subdirectory on each data storage server according to the current duration, the current read and write data volume processed within the current duration, the historical duration, and the historical read and write data volume processed within the historical duration;

[0019] The read and write throughput of the subdirectory on each data storage server is determined based on the current duration, the current read and write request volume processed within the current duration, the historical duration, and the historical read and write request volume processed within the historical duration.

[0020] In a second aspect, an embodiment of the present application provides a method for obtaining performance data, which is applied to a manufacturer management node. The method includes:

[0021] Send a performance data acquisition request to the server in the distributed file system; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system;

[0022] Obtain the performance data of the corresponding sub-directories mounted on each data storage server in the server by each tenant sent by the server; the performance data of each sub-directory is obtained by the server based on the sub-directory information list, and the storage performance data of each sub-directory is determined based on the sub-directory identifier carried in the corresponding read and write request sent by each client device during the process of requesting read and write operations.

[0023] In one embodiment, the method comprises:

[0024] For any tenant, the performance data of the corresponding subdirectories mounted by the tenant on each data storage server are summed to obtain the performance data of the subdirectory.

[0025] In a third aspect, an embodiment of the present application provides a performance data acquisition device, the device comprising:

[0026] The request response module is used to respond to the performance data acquisition request sent by the manufacturer management node; the performance data acquisition request carries a list of corresponding sub-directory information mounted on the server by multiple different tenants in the distributed file system;

[0027] The data acquisition module is used to obtain the performance data of the corresponding subdirectories mounted by different tenants under each data storage server in the server of the distributed file system according to the subdirectory information list; the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read and write request sent by each client device in the process of requesting a read and write operation;

[0028] The data sending module is used to send the performance data of each sub-directory to the manufacturer management node.

[0029] In a fourth aspect, an embodiment of the present application provides a performance data acquisition device, the device comprising:

[0030] A request sending module is used to send a performance data acquisition request to a server in a distributed file system; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system;

[0031] The data acquisition module is used to obtain the performance data of the corresponding sub-directories mounted on each data storage server in the server by each tenant sent by the server; the performance data of each sub-directory is obtained by the server based on the sub-directory information list, and the storage performance data of each sub-directory is determined based on the sub-directory identifier carried in the corresponding read and write request sent by each client device during the process of requesting read and write operations.

[0032] In a fifth aspect, an embodiment of the present application further provides a computer device, comprising a transceiver, a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method of any one of the embodiments of the first and second aspects above are implemented.

[0033] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method of any embodiment of the first aspect and the second aspect above are implemented.

[0034] In a seventh aspect, an embodiment of the present application further provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method of any one of the embodiments of the first and second aspects above.

[0035] The embodiment of the present application provides a performance data acquisition method, device, computer equipment and storage product, which is applied to the server in a distributed file system, including: responding to a performance data acquisition request sent by a manufacturer management node, the performance data acquisition request carries a corresponding sub-directory information list of multiple different tenants in the distributed file system mounted on the server, acquiring performance data of corresponding sub-directories mounted on each data storage server in the server by different tenants according to the sub-directory information list, and sending the performance data of each sub-directory to the manufacturer management node; the above method can obtain the performance data of the distributed file system providing storage services for different tenants in a fine-grained manner, that is, obtain the performance data of sub-directories of different tenants, and on this basis, can also provide reliability reference information for multiple storage service applications; at the same time, the above method does not require manual processing, can reduce the errors generated in the process of obtaining performance data, improve the accuracy of the performance data of sub-directories of different tenants obtained, and can also speed up the speed of obtaining performance data of sub-directories of different tenants, thereby improving the efficiency of obtaining performance data. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 An application environment diagram of a performance data acquisition method in an embodiment;

[0037] Figure 2 is a flow chart of a method for acquiring performance data in one embodiment;

[0038] Figure 3 is a flow chart of a method for acquiring performance data in another embodiment;

[0039] Figure 4 is a flow chart of a method for acquiring performance data in another embodiment;

[0040] Figure 5 is a flow chart of a method for acquiring performance data in another embodiment;

[0041] Figure 6 is a flow chart of a method for acquiring performance data in one embodiment;

[0042] Figure 7 is a structural block diagram of a performance data acquisition device in one embodiment;

[0043] Figure 8 is a structural block diagram of a performance data acquisition device in another embodiment;

[0044] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In the field of cloud computing, cluster systems are increasingly used. Taking a distributed file system as an example, in actual applications, each user usually obtains a distributed file system by leasing. Correspondingly, cloud storage vendors can divide part of the storage capacity in the distributed file system into multiple tenants (or users), that is, allocate subdirectories (i.e., fileset) to multiple different tenants to provide storage services for each tenant. Therefore, it is particularly important to obtain the performance data of subdirectories of different tenants. However, the related art can only obtain the performance data of the read and write process, the client device, or the same read and write tasks during the process of the client device requesting a read and write operation, and cannot obtain the performance data of the subdirectories of different tenants. Based on this, an embodiment of the present application provides a performance data acquisition method, which can obtain the performance data of the distributed file system providing storage services for different tenants in a fine-grained manner, that is, obtain the performance data of the subdirectories of different tenants.

[0047] The performance data acquisition method provided in the embodiment of the present application can be applied to Figure 1The application environment diagram shown. The application environment includes a performance data acquisition system, which may include a distributed file system and a vendor management node (i.e., a CM node), and the distributed file system may be a high-performance parallel file system (HighPerformance File System, HPFS). Optionally, the distributed file system includes at least one client device and a server corresponding to at least one tenant (wherein each tenant corresponds to a client device). It should be noted here that a client provided by a cloud storage vendor is deployed on each client device. In actual applications, the client on the client device is used to mount the corresponding subdirectory under the server for the corresponding tenant; in an embodiment of the present application, the client is presented in the form of an application. Optionally, the above-mentioned server may include a metadata server node (i.e., an MGS node), a metadata principle node (i.e., an MDS node), and at least one data storage server node (i.e., an OSS node), the MGS node includes a configuration information management node (i.e., an MGT node), the MDS node includes a metadata storage node (i.e., an MDT node), and the OSS node includes at least one data storage node (i.e., an OST node). It should be noted here that the distributed storage system can aggregate the bandwidth and storage resources of each data storage node to achieve TB-level bandwidth per second and EB-level storage capacity.

[0048] like Figure 1 The figure shows a structure diagram of a performance data acquisition system composed of a distributed file system and a manufacturer management node, wherein adjacent nodes can be connected by communication, and the communication method can be Wi-Fi, mobile network, optical fiber connection or Bluetooth connection, etc., and the performance data acquisition request or read and write request transmitted between adjacent nodes are all requests of the Remote Procedure Call (RPC) protocol. Optionally, each node in the performance data acquisition system can be implemented by, but is not limited to, an independent server or computer device, and this embodiment does not limit the specific form of each node. The following embodiment of the present application uses the server and the manufacturer management node in the distributed file system as the execution body of the performance data acquisition method to introduce the specific process of the performance data acquisition method.

[0049] like Figure 2 FIG. 1 is a flow chart of a method for obtaining performance data provided in an embodiment of the present application. The method is applied to a server in a distributed file system. The method can be implemented by the following steps:

[0050] S101, responding to a performance data acquisition request sent by a manufacturer management node, wherein the performance data acquisition request carries a corresponding sub-directory information list mounted on a server by multiple different tenants in a distributed file system.

[0051] Among them, when the cloud storage vendor needs to obtain the storage service performance of the subdirectories allocated to each tenant to prevent and diagnose system performance bottlenecks, optimize the storage service performance of the subdirectories allocated to each tenant, improve the utilization rate of storage resources in the distributed file system, or needs to obtain performance data of the subdirectories allocated to each tenant by the cloud storage vendor for each tenant to view, the vendor management node can send a performance data acquisition request to the server in the distributed file system. Correspondingly, the server in the distributed file system can receive and respond to the performance data acquisition request sent by the vendor management node. In an embodiment of the present application, the storage resources between different subdirectories are independent of each other.

[0052] It should be noted that each tenant can mount at least one subdirectory on the server. In the embodiment of the present application, each tenant mounts a subdirectory on the server as an example. Specifically, the cloud storage vendor allocates subdirectories to multiple different tenants based on the distributed file system, which actually refers to the subdirectories allocated to different tenants by the cloud storage vendor from each storage node under each data storage server on the server. It can also be simply understood as the storage area allocated to different tenants in each storage node.

[0053] Among them, for any tenant, the tenant mounts one subdirectory on the server; the subdirectory can be created based on at least one storage node under the same data storage server, or it can be created based on different storage nodes under multiple data storage servers, which is not limited to this embodiment of the present application.

[0054] In practical applications, the sub-directory information list may include sub-directory information corresponding to different tenants. Optionally, the sub-directory information list may include information such as size, creation time and / or service type of a sub-directory corresponding to at least one tenant.

[0055] Optionally, at least one tenant can concurrently send corresponding performance data acquisition requests to the manufacturer management node through the same client device, and obtain the performance data of subdirectories corresponding to multiple tenants at one time. Compared with traditional technologies, the use of a concurrent method can greatly reduce the time consumption of obtaining performance data of subdirectories corresponding to multiple tenants. In the embodiment of the present application, a tenant sending a performance data acquisition request to the manufacturer management node is used as an example for explanation.

[0056] S102, obtaining performance data of corresponding subdirectories mounted on each data storage server of different tenants in the server according to the subdirectory information list, wherein the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read / write request sent by each client device during the process of requesting a read / write operation.

[0057] Specifically, the server can construct performance data reading commands for each subdirectory based on the subdirectory information list, and trigger each performance data reading command to obtain pre-stored performance data of corresponding subdirectories mounted on each data storage server in the server by different tenants.

[0058] In addition, the server can also call a performance data acquisition tool, and after inputting the sub-directory information list into the performance data acquisition tool, obtain the pre-stored performance data of the corresponding sub-directories mounted under each data storage server in the server by different tenants.

[0059] In one implementation, the server can pre-train an algorithm model, and then input at least one read and write request sent by each client device at the current moment and before the current moment in the process of requesting read and write operations into the algorithm model in sequence. The algorithm model outputs the performance data of the corresponding sub-directories mounted by each tenant under each data storage server in the server.

[0060] S103: Send the performance data of each sub-directory to the manufacturer management node.

[0061] Furthermore, the server may send the performance data of the corresponding subdirectories mounted by each tenant under each data storage server in the server to the manufacturer management node.

[0062] The technical solution in the embodiment of the present application responds to a performance data acquisition request sent by a manufacturer's management node, the performance data acquisition request carries a corresponding sub-directory information list mounted on the server side by multiple different tenants in the distributed file system, obtains the performance data of the corresponding sub-directories mounted on each data storage server in the server side by different tenants according to the sub-directory information list, and sends the performance data of each sub-directory to the manufacturer's management node; the above method can obtain the performance data of the distributed file system providing storage services for different tenants in a fine-grained manner, that is, obtains the performance data of the sub-directories of different tenants, and on this basis, can also provide reliability reference information for a variety of storage service applications; at the same time, the above method does not require manual processing, can reduce the errors generated in the process of obtaining performance data, improve the accuracy of the performance data of the sub-directories of different tenants obtained, and can also speed up the speed of obtaining the performance data of the sub-directories of different tenants, thereby improving the efficiency of obtaining performance data.

[0063] The following is an explanation of the process of obtaining the performance data of the corresponding subdirectories mounted by different tenants on each data storage server in the server according to the subdirectory information list. In one embodiment, the subdirectory information list includes the subdirectory identifier corresponding to each tenant; Figure 3 As shown, the steps in S102 above can be implemented in the following manner:

[0064] S112: For any sub-directory identifier, obtain the sub-directory mounted by the tenant corresponding to the sub-directory identifier on each data storage server.

[0065] The sub-directory identifier corresponding to the tenant may be understood as the identifier of the sub-directory corresponding to the tenant.

[0066] In one embodiment, for any sub-directory identifier, the server can search for the sub-directory identifier corresponding to the tenant in the mapping relationship between different sub-directory identifiers and different sub-directories, and determine the sub-directory corresponding to the sub-directory identifier that matches the sub-directory identifier corresponding to the tenant in the mapping relationship as the sub-directory mounted by the tenant corresponding to the sub-directory identifier on each data storage server.

[0067] In another embodiment, the server may also perform character matching, comparison and other operations on the sub-directory identifier corresponding to each tenant, and determine the sub-directory mounted by the tenant corresponding to the sub-directory identifier on each data storage server according to the processing operation results.

[0068] S122. Determine performance data of corresponding subdirectories mounted on each data storage server according to the subdirectories.

[0069] Specifically, the server can determine the corresponding performance data storage location according to the sub-directory, and then read the performance data of the sub-directory mounted under each data storage server from the performance data storage location. In addition, the server can call a performance data reading tool, input the sub-directory into the performance data reading tool, and the performance data reading tool outputs the performance data of the corresponding sub-directory mounted under each data storage server.

[0070] In an embodiment of the present application, the server may trigger a performance data set acquisition instruction corresponding to a sub-directory to obtain performance data of the sub-directory mounted on each data storage server.

[0071] The technical solution in the embodiment of the present application, for any sub-directory identifier, obtains the sub-directory mounted on each data storage server by the tenant corresponding to the sub-directory identifier, and determines the performance data of the corresponding sub-directory mounted on each data storage server based on the sub-directory; the above method can obtain the performance data of the distributed file system providing storage services for different tenants in a fine-grained manner, that is, obtains the performance data of the sub-directories of different tenants, and the processing process is relatively simple and does not require the participation of complex algorithms, thereby improving the wide applicability of the performance data acquisition method.

[0072] The following is an explanation of the process of determining the performance data of the corresponding subdirectories mounted on each data storage server according to the subdirectories. Figure 4 As shown, the steps in the above S122 may include:

[0073] S1221. Search for a subdirectory in the association relationship between different subdirectories and different performance data.

[0074] The above association relationship is pre-created by the server; the different performance data in the association relationship can be determined in real time by the server in the process of providing storage services according to the corresponding sub-directories mounted on each data storage server by different tenants obtained in advance. It should be noted here that the storage service provided by any sub-directory can be a read operation and / or a write operation.

[0075] S1222. Determine the performance data corresponding to the sub-directory in the association relationship as the performance data of the sub-directory mounted by the tenant on each data storage server.

[0076] Specifically, the server may determine the performance data corresponding to the sub-directory that matches the sub-directory in the association relationship as the performance data of the sub-directory mounted by the corresponding tenant under each data storage server.

[0077] In one embodiment, the above performance data includes read / write bandwidth and / or read / write throughput; before executing the step in S221 above, Figure 5 As shown, the performance data acquisition process of each sub-directory can be achieved through the following steps:

[0078] S1223. For any subdirectory, obtain the current duration corresponding to the subdirectory creation time and the current time, and the historical duration corresponding to the subdirectory creation time and the historical time.

[0079] In the embodiment of the present application, the sub-directory creation time refers to the time when the sub-directory creation is completed. The current time and the historical time are two different times corresponding to the process of periodically counting the amount of read and write data, and these two statistical times are adjacent.

[0080] S1224. Determine the read and write bandwidth of the subdirectory on each data storage server according to the current duration, the current read and write data volume processed within the current duration, the historical duration, and the historical read and write data volume processed within the historical duration.

[0081] In actual applications, for any sub-directory, the server can count the total read and write data volume corresponding to all read and write requests responded to by the sub-directory within the time period corresponding to the current duration, that is, the current read and write data volume; and the server can also count the total read and write data volume corresponding to all read and write requests responded to by the sub-directory within the historical time period corresponding to the historical duration, that is, the historical read and write data volume.

[0082] It should be noted here that before the client device sends each read / write request to the server, it can modify the task identifier job_id in the read / write request to the sub-directory identifier, so that the server can count the amount of read / write data of the sub-directory on each data storage server according to the sub-directory identifier carried in the read / write request, and the amount of read / write data is accumulated over time. In the embodiment of the present application, the amount of read / write data is stored in json format for easy reading and storage.

[0083] Specifically, the server can obtain the current read and write data volume processed in the current duration and the historical read and write data volume processed in the historical duration, and perform a first arithmetic operation on the current duration, the current read and write data volume, the historical duration, and the historical read and write data volume to obtain the read and write bandwidth of the subdirectory on each data storage server. Optionally, the first arithmetic operation can be implemented by at least one of addition, subtraction, multiplication, logarithm, exponential, etc.

[0084] In addition, the server can also pre-train a read-write bandwidth acquisition model, and then input the current duration, current read-write data volume, historical duration and historical read-write data volume into the read-write bandwidth acquisition model, and the read-write bandwidth acquisition model outputs the read-write bandwidth of the subdirectory on each data storage server. Optionally, the above-mentioned read-write bandwidth acquisition model can be implemented by at least one of a convolutional neural network model, a fully connected neural network model, a long short-term memory neural network model, a residual neural network model, a recurrent neural network model, etc.

[0085] In practical applications, the above read and write requests include read requests and write requests, and correspondingly, the above read and write bandwidths may include read bandwidths and write bandwidths. In an embodiment of the present application, the read bandwidth of a subdirectory on any data storage server may be determined by the following formula (1), namely:

[0086] (1)

[0087] in, Indicates the read bandwidth of the subdirectory on any data storage server. Indicates the current duration The current amount of read and write data processed internally, Indicates historical duration The amount of historical read and write data processed internally, Indicates the total number of data storage nodes under the data storage server;

[0088] At the same time, the write bandwidth of the subdirectory on any data storage server can be determined by the following formula (2), namely:

[0089] (2)

[0090] in, Indicates that the subdirectory is on any data storage server bandwidth, Indicates the current duration The current amount of read and write data processed internally, Indicates historical duration The amount of historical read and write data processed internally.

[0091] S1225. Determine the read and write throughput of the subdirectory on each data storage server according to the current duration, the current read and write request volume processed within the current duration, the historical duration, and the historical read and write request volume processed within the historical duration.

[0092] Specifically, the server can obtain the current read and write request volume processed in the current duration and the historical read and write request volume processed in the historical duration, and perform a second arithmetic operation on the current duration, the current read and write request volume, the historical duration, and the historical read and write request volume to obtain the read and write throughput of the subdirectory on each data storage server. The second arithmetic operation is different from the first arithmetic operation.

[0093] In addition, the server can also pre-train a throughput acquisition model, and then input the current duration, current read and write request volume, historical duration and historical read and write request volume into the throughput acquisition model, and the throughput acquisition model outputs the read and write throughput of the subdirectory on each data storage server. Optionally, the throughput acquisition model can be implemented by at least one of a deep learning network model, a convolutional neural network model, a fully connected neural network model, a long short-term memory neural network model, a residual neural network model, a recurrent neural network model, etc.

[0094] In practical applications, the above read and write throughput may include read throughput and write throughput. In the embodiment of the present application, the read throughput of the subdirectory on any data storage server can be determined by the following formula (3), namely:

[0095] (3)

[0096] in, Indicates the read throughput of the subdirectory on any data storage server. Indicates the current duration The current number of read requests processed within Indicates historical duration The historical number of read requests processed internally;

[0097] At the same time, the write throughput of the subdirectory on any data storage server can be determined by the following formula (4), namely:

[0098] (4)

[0099] in, Indicates that the subdirectory is on any data storage server. Throughput, Indicates the current duration The current number of write requests being processed within Indicates historical duration The historical number of write requests processed within the system.

[0100] In actual applications, for any client device, if the read / write throughput of the subdirectory corresponding to the client device on each data storage server changes, the read / write throughput of the subdirectory corresponding to the client device on each data storage server may be updated.

[0101] At the same time, it should be noted that before sending each read / write request, the client device can modify the task identifier job_id in the read / write request to the sub-directory identifier, so that the server can count the read / write throughput of the sub-directory on each data storage server according to the sub-directory identifier carried in the read / write request, and the read / write throughput is cumulatively counted over time. In the embodiment of the present application, the read / write throughput is stored in json format for easy reading and storage.

[0102] The technical solution in the embodiment of the present application searches for sub-directories in the association relationship between different sub-directories and different performance data, and determines the performance data corresponding to the sub-directory in the association relationship as the performance data of the sub-directory mounted by the tenant under each data storage server; the above method does not need to perform complex processing, and can directly obtain the performance data of the sub-directory mounted by each tenant under each data storage server through a pre-built association relationship. The processing process is relatively simple, thereby speeding up the acquisition of performance data and improving the efficiency of acquiring performance data.

[0103] In one embodiment, the present application also provides a method for obtaining performance data, which is applied to a server in a distributed file system. The method includes the following process:

[0104] (1) Responding to a performance data acquisition request sent by a vendor management node; the performance data acquisition request carries a corresponding sub-directory information list of multiple different tenants in the distributed file system mounted on the server; the sub-directory information list includes a sub-directory identifier corresponding to each tenant;

[0105] (2) For any sub-directory identifier, obtain the sub-directory mounted by the tenant corresponding to the sub-directory identifier on each data storage server;

[0106] (3) For any subdirectory, obtain the current duration between the subdirectory creation time and the current time, and the historical duration between the subdirectory creation time and the historical time;

[0107] (4) Determine the read and write bandwidth of the subdirectory on each data storage server based on the current duration, the current read and write data volume processed within the current duration, the historical duration, and the historical read and write data volume processed within the historical duration;

[0108] (5) Determine the read and write throughput of the subdirectory on each data storage server based on the current duration, current read and write request volume, historical duration, and historical read and write request volume;

[0109] (6) searching for a subdirectory in the association relationship between different subdirectories and different performance data; the performance data includes read / write bandwidth and / or read / write throughput;

[0110] (7) Determine the performance data corresponding to the subdirectory in the association relationship as the performance data of the subdirectory mounted by the tenant on each data storage server; the storage performance data of each subdirectory is determined based on the subdirectory identifier carried in the corresponding read / write request sent by each client device during the process of requesting a read / write operation;

[0111] (8) Send the performance data of each sub-directory to the manufacturer management node.

[0112] The execution process of the above (1) to (8) can be specifically referred to the description of the above embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0113] like Figure 6 FIG. 1 is a flow chart of a method for obtaining performance data provided in an embodiment of the present application. The method is applied to a manufacturer management node. The method can be implemented by the following steps:

[0114] S201: Send a performance data acquisition request to a server in a distributed file system, wherein the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system.

[0115] In actual applications, when a cloud storage vendor needs to obtain the storage service performance of a subdirectory allocated to each tenant to prevent system performance bottlenecks, optimize the storage service performance of a subdirectory allocated to each tenant, improve the utilization of storage resources in a distributed file system, or needs to obtain performance data of a subdirectory allocated to each tenant by the cloud storage vendor for each tenant to view, the vendor management node can send a performance data acquisition request to a server in the distributed file system. In an embodiment of the present application, storage resources between different subdirectories are independent of each other.

[0116] It should be noted here that when cloud storage vendors allocate sub-directories to multiple different tenants based on a distributed file system, this actually refers to the sub-directories that cloud storage vendors allocate to different tenants in each storage node under each data storage server on the server side. It can also be simply understood as the storage area allocated to different tenants in each storage node.

[0117] Among them, for any tenant, the tenant mounts one subdirectory on the server; the subdirectory can be created based on at least one storage node under the same data storage server, or it can be created based on different storage nodes under multiple data storage servers, which is not limited to this embodiment of the present application.

[0118] In practical applications, the sub-directory information list may include sub-directory information corresponding to different tenants. Optionally, the sub-directory information list may include information such as size, creation time and / or service type of a sub-directory corresponding to at least one tenant.

[0119] In an embodiment of the present application, for any tenant, when the tenant needs to obtain a distributed storage system, the tenant can send a sub-directory creation request to the manufacturer's management node through the corresponding client device. Correspondingly, the manufacturer's management node receives and responds to the sub-directory creation request. In order to improve the storage service quality Qos of the distributed file system, the manufacturer's management node reasonably allocates the corresponding sub-directory from at least one storage node to the tenant based on the storage capacity carried in the sub-directory creation request and the data reading and writing speed of each storage node in the distributed storage system. Generate a sub-directory identifier, and then send both the storage capacity and the sub-directory identifier to the MGS node, instructing the MGS node to allocate the sub-directory from at least one corresponding storage node.

[0120] Afterwards, the MGS node sends the subdirectory and the storage address of the subdirectory on at least one storage node to the client device through the manufacturer management node, so that the client device can mount at least one storage node for the corresponding tenant according to the storage address to create a subdirectory; the storage address here can be understood as the mount address where the tenant mounts the subdirectory. At the same time, the manufacturer management node can store relevant information such as the subdirectory and the storage address of the subdirectory on the corresponding storage node in the back-end database for subsequent tracing during fault handling. It should be noted here that no matter how large or small the number of tenants who concurrently apply for storage capacity is, the total storage capacity of all storage nodes in the distributed file system is fixed and can be flexibly adjusted during allocation.

[0121] S202, obtaining the performance data of the corresponding subdirectories mounted on each data storage server in the server by each tenant sent by the server. The performance data of each subdirectory is obtained by the server according to the subdirectory information list, and the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read / write request sent by each client device during the process of requesting a read / write operation.

[0122] Furthermore, the server in the distributed file system can receive and respond to the performance data acquisition request sent by the manufacturer management node, obtain the performance data of the corresponding subdirectories mounted by different tenants under each data storage server in the server according to the subdirectory information list, and send the performance data of each subdirectory to the manufacturer management node. Correspondingly, the manufacturer management node can receive the performance data of the corresponding subdirectories mounted by each tenant under each data storage server in the server sent by the server.

[0123] Specifically, the server can construct performance data reading commands for each subdirectory based on the subdirectory information list, and trigger each performance data reading command to obtain pre-stored performance data of corresponding subdirectories mounted on each data storage server in the server by different tenants.

[0124] In addition, the server can also call a performance data acquisition tool, and after inputting the sub-directory information list into the performance data acquisition tool, obtain the pre-stored performance data of the corresponding sub-directories mounted under each data storage server in the server by different tenants.

[0125] In one implementation, the server can pre-train an algorithm model, and then input at least one read and write request sent by each client device at the current moment and before the current moment in the process of requesting read and write operations into the algorithm model in sequence. The algorithm model outputs the performance data of the corresponding sub-directories mounted by each tenant under each data storage server in the server.

[0126] The technical solution in the embodiment of the present application is to send a performance data acquisition request to a server in a distributed file system, wherein the performance data acquisition request carries a corresponding sub-directory information list of multiple different tenants in the distributed file system mounted on the server, and obtain the performance data of the corresponding sub-directories of each tenant mounted on each data storage server in the server sent by the server, wherein the performance data of each sub-directory is obtained by the server according to the sub-directory information list, and the storage performance data of each sub-directory is determined according to the sub-directory identifier carried in the corresponding read-write request sent by each client device in the process of requesting a read-write operation; the above method can obtain the performance data of the distributed file system providing storage services for different tenants in a fine-grained manner, that is, obtain the performance data of the sub-directories of different tenants, and on this basis, can also provide reliability reference information for a variety of storage service applications; at the same time, the above method does not require manual participation in processing, can reduce the errors generated in the process of obtaining performance data, improve the accuracy of the performance data of the sub-directories of different tenants obtained, and can also speed up the speed of obtaining the performance data of the sub-directories of different tenants, thereby improving the efficiency of obtaining performance data.

[0127] The following is an explanation of the process of the manufacturer management node obtaining the total performance data of each sub-directory. In one embodiment, after executing the step in S202 above, the method may include: for any tenant, summing the performance data of the corresponding sub-directories mounted by the tenant under each data storage server to obtain the performance data of the sub-directory.

[0128] Specifically, for any tenant, the manufacturer management node can sum the performance data of the same subdirectory mounted by the tenant on each data storage server to obtain the performance data of the corresponding subdirectory of the tenant in the distributed file system.

[0129] In the embodiment of the present application, the performance data may include read bandwidth, write bandwidth, read throughput and write throughput. Correspondingly, for any tenant, the obtained read bandwidth, write bandwidth, read throughput and write throughput of the tenant's subdirectory in the distributed file system can be expressed by formulas (5)-(8), that is:

[0130] (5)

[0131] (6)

[0132] (7)

[0133] (8)

[0134] in, Indicates the read bandwidth of any tenant in the subdirectory of the distributed file system. Indicates the write bandwidth of any tenant in the subdirectory of the distributed file system. Indicates the read throughput of any tenant in the subdirectory of the distributed file system. Indicates the write throughput of any tenant in the subdirectory of the distributed file system. Indicates the total number of data storage servers (OSS nodes) in the distributed file system.

[0135] The technical solution in the embodiment of the present application is that, for any tenant, the performance data of the corresponding sub-directories mounted by the tenant on each data storage server are summed to obtain the performance data of the sub-directory. This processing process is relatively simple, thereby speeding up the acquisition of performance data of different sub-directories and improving the efficiency of acquiring performance data of different sub-directories. This process can be implemented without learning professional knowledge, and can greatly reduce the complexity of obtaining performance data of different sub-directories to a certain extent.

[0136] In one embodiment, the present application also provides a method for obtaining performance data, which is applied to a manufacturer management node. The method includes the following process:

[0137] (1) Sending a performance data acquisition request to the server in the distributed file system; the performance data acquisition request carries a list of corresponding sub-directory information mounted on the server by multiple different tenants in the distributed file system;

[0138] (2) Obtaining the performance data of the corresponding subdirectories mounted on each data storage server in the server by each tenant sent by the server; the performance data of each subdirectory is obtained by the server according to the subdirectory information list, and the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read and write request sent by each client device during the process of requesting a read and write operation;

[0139] (3) For any tenant, the performance data of the corresponding sub-directories mounted by the tenant on each data storage server are summed to obtain the performance data of the sub-directory.

[0140] The execution process of the above (1) to (3) can be specifically referred to the description of the above embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0141] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0142] Based on the same inventive concept, the embodiment of the present application also provides a performance data acquisition device for implementing the performance data acquisition method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more performance data acquisition device embodiments provided below can refer to the limitations on the performance data acquisition method above, and will not be repeated here.

[0143] In one embodiment, Figure 7 FIG. 1 is a schematic diagram of the structure of a performance data acquisition device in an embodiment of the present application. The performance data acquisition device provided in the embodiment of the present application can be applied to the server in a distributed file system. Figure 7 As shown, the performance data acquisition device of the embodiment of the present application may include: a request response module 11, a data acquisition module 12 and a data sending module 13, wherein:

[0144] The request response module 11 is used to respond to the performance data acquisition request sent by the manufacturer management node; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system;

[0145] The data acquisition module 12 is used to obtain the performance data of the corresponding subdirectories mounted on each data storage server of different tenants in the server according to the subdirectory information list; the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read and write request sent by each client device in the process of requesting a read and write operation;

[0146] The data sending module 13 is used to send the performance data of each sub-directory to the manufacturer management node.

[0147] The performance data acquisition device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned performance data acquisition method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0148] In one embodiment, the sub-directory information list includes sub-directory identifiers corresponding to each tenant; the data acquisition module 12 includes: an acquisition unit and a determination unit, wherein:

[0149] An acquisition unit, used for acquiring, for any sub-directory identifier, a sub-directory mounted on each data storage server by a tenant corresponding to the sub-directory identifier;

[0150] The determination unit is used to determine the performance data of the corresponding subdirectories mounted under each data storage server according to the subdirectories.

[0151] The performance data acquisition device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned performance data acquisition method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0152] In one embodiment, the determining unit is specifically configured to:

[0153] Find subdirectories in the association relationship between different subdirectories and different performance data;

[0154] The performance data corresponding to the sub-directory in the association relationship is determined as the performance data of the sub-directory mounted by the tenant under each data storage server.

[0155] The performance data acquisition device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned performance data acquisition method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0156] In one embodiment, the performance data includes read / write bandwidth and / or read / write throughput; the performance data acquisition device further includes: a duration acquisition module, a bandwidth determination module and a throughput determination module, wherein:

[0157] The duration acquisition module is used to acquire, for any subdirectory, the current duration corresponding to the subdirectory creation time and the current time, and the historical duration corresponding to the subdirectory creation time and the historical time;

[0158] The bandwidth determination module is used to determine the read and write bandwidth of the subdirectory on each data storage server according to the current duration, the current read and write data volume processed within the current duration, the historical duration, and the historical read and write data volume processed within the historical duration;

[0159] The throughput determination module is used to determine the read and write throughput of the subdirectory on each data storage server according to the current duration, the current read and write request volume, the historical duration and the historical read and write request volume.

[0160] The performance data acquisition device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned performance data acquisition method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0161] In another embodiment, Figure 8 This is a schematic diagram of the structure of a performance data acquisition device in one embodiment of the present application. The performance data acquisition device provided in the embodiment of the present application can be applied to a manufacturer management node. Figure 8 As shown, the performance data acquisition device of the embodiment of the present application may include: a request sending module 21 and a data acquisition module 22, wherein:

[0162] The request sending module 21 is used to send a performance data acquisition request to the server in the distributed file system; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system;

[0163] The data acquisition module 22 is used to obtain the performance data of the corresponding sub-directories mounted on each data storage server in the server by each tenant sent by the server; the performance data of each sub-directory is obtained by the server based on the sub-directory information list, and the storage performance data of each sub-directory is determined based on the sub-directory identifier carried in the corresponding read and write request sent by each client device during the process of requesting a read and write operation.

[0164] The performance data acquisition device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned performance data acquisition method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0165] In one embodiment, the performance data acquisition device further includes: a summing module, wherein:

[0166] The summing module is used to sum the performance data of the corresponding subdirectories mounted by any tenant on each data storage server to obtain the performance data of the subdirectories.

[0167] The performance data acquisition device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned performance data acquisition method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0168] For the specific definition of the performance data acquisition device, please refer to the definition of the performance data acquisition method above, which will not be repeated here. Each module in the above-mentioned performance data acquisition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0169] In one embodiment, a computer device is also provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9As shown. The computer device includes a transceiver, a processor connected through a system bus, a memory and a network interface. Among them, the processor of the computer device is used to provide processing power. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store backup data of the logical disk. The network interface of the computer device is used to communicate with an external endpoint through a network connection. When the computer program is executed by the processor, a performance data acquisition method is implemented.

[0170] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0171] In one embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the technical solution in the above-mentioned performance data acquisition method embodiment of the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0172] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the technical solution of the above-mentioned performance data acquisition method of the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0173] In one embodiment, a computer program product is also provided, including a computer program. When the computer program is executed by a processor, the technical solution of the above-mentioned performance data acquisition method of the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0174] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for acquiring performance data, characterized in that: Applied to a server in a distributed file system, the method comprises: Responding to a performance data acquisition request sent by a manufacturer management node; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system; Acquire performance data of corresponding subdirectories mounted on each data storage server in the server by different tenants according to the subdirectory information list; the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read / write request sent by each client device in the distributed file system during the process of requesting a read / write operation; The performance data of each of the sub-directories is sent to the manufacturer management node.

2. The method according to claim 1, characterized in that The sub-directory information list includes sub-directory identifiers corresponding to each of the tenants; and obtaining performance data of corresponding sub-directories mounted on each data storage server in the server by different tenants according to the sub-directory information list includes: For any sub-directory identifier, obtain the sub-directory mounted by the tenant on each of the data storage servers corresponding to the sub-directory identifier; The performance data of the corresponding subdirectories mounted on each of the data storage servers is determined according to the subdirectories.

3. The method according to claim 2, characterized in that The determining, according to the sub-directories, the performance data of the corresponding sub-directories mounted on each of the data storage servers includes: Searching for the subdirectory in the association relationship between different subdirectories and different performance data; The performance data corresponding to the sub-directory in the association relationship is determined as the performance data of the sub-directory mounted by the tenant under each of the data storage servers.

4. The method according to any one of claims 1 to 3, characterized in that The performance data includes read / write bandwidth and / or read / write throughput; the performance data acquisition process of each sub-directory includes: For any subdirectory, obtain the current duration corresponding to the subdirectory creation time and the current time, and the historical duration corresponding to the subdirectory creation time and the historical time; Determine the read and write bandwidth of the subdirectory on each of the data storage servers according to the current duration, the current read and write data volume processed within the current duration, the historical duration, and the historical read and write data volume processed within the historical duration; The read and write throughput of the subdirectory on each of the data storage servers is determined based on the current duration, the current read and write request volume processed within the current duration, the historical duration, and the historical read and write request volume processed within the historical duration.

5. A method for acquiring performance data, characterized in that: Applied to a manufacturer management node, the method includes: Sending a performance data acquisition request to a server in a distributed file system; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system; Obtain the performance data of the corresponding subdirectories mounted under each data storage server in the server by each tenant and sent by the server; the performance data of each subdirectory is obtained by the server according to the subdirectory information list, and the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read and write request sent by each client device in the distributed file system during the process of requesting read and write operations.

6. The method according to claim 5, characterized in that The method comprises: For any tenant, the performance data of the corresponding subdirectories mounted by the tenant on each of the data storage servers are summed to obtain the performance data of the subdirectory.

7. A performance data acquisition device, characterized in that: The device comprises: A request response module, used to respond to a performance data acquisition request sent by a manufacturer management node; the performance data acquisition request carries a corresponding sub-directory information list mounted on a server in the distributed file system by multiple different tenants in the distributed file system; A data acquisition module is used to obtain performance data of corresponding subdirectories mounted by different tenants under each data storage server in the server of the distributed file system according to the subdirectory information list; the storage performance data of each subdirectory is determined according to the subdirectory identifier carried in the corresponding read / write request sent by each client device in the distributed file system during the process of requesting a read / write operation; The data sending module is used to send the performance data of each sub-directory to the manufacturer management node.

8. A performance data acquisition device, characterized in that: The device comprises: A request sending module, used to send a performance data acquisition request to a server in a distributed file system; the performance data acquisition request carries a corresponding sub-directory information list mounted on the server by multiple different tenants in the distributed file system; A data acquisition module is used to obtain the performance data of the corresponding sub-directories mounted on each data storage server in the server by each tenant and sent by the server; the performance data of each sub-directory is obtained by the server according to the sub-directory information list, and the storage performance data of each sub-directory is determined according to the sub-directory identifier carried in the corresponding read and write request sent by each client device in the distributed file system during the process of requesting read and write operations.

9. A computer device comprising a transceiver, a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images