Performance testing method, device and equipment for distributed object storage cluster

Abstract

The present disclosure relates to the field of distributed storage technology, and proposes a performance test method, device and equipment for a distributed object storage cluster, the method comprising: obtaining target node information of a target storage cluster and data pool information of the target storage cluster, and determining the estimated performance index of the target storage cluster; creating one or more object test users according to the data pool information, and generating a test configuration file of a preset test tool for each object test user; updating a client configuration file of the test configuration file based on the target node information, and driving the preset test tool to perform a performance test on the target storage cluster using the updated test configuration file; judging whether the performance test result meets expectations according to the performance test result and the estimated performance index. The technical solution provided by one or more embodiments of the present disclosure can effectively obtain and evaluate the baseline performance index of a distributed object storage cluster.

Description

Technical Field

[0001] The present disclosure relates to the field of distributed storage technology, and in particular to a performance testing method, device and equipment for a distributed object storage cluster. Background Art

[0002] Faced with massive amounts of unstructured data of varying sizes, distributed object storage has gradually become a preferred solution for storing massive data due to its flexible metadata management model and object-level access control function.

[0003] However, in the field of media storage, the read and write performance of large-scale objects such as pictures, texts, videos, logs, etc. puts a test on the performance of backend distributed object storage.

[0004] Therefore, how to accurately grasp the baseline performance data of the distributed object storage cluster delivered to customers is of great significance to operation and maintenance personnel. Using the baseline performance data of the distributed object storage cluster, operation and maintenance personnel can adjust business strategies according to actual business pressure. Summary of the invention

[0005] In view of this, one or more embodiments of the present disclosure provide a performance testing method, apparatus and device for a distributed object storage cluster, which can accurately obtain baseline performance indicators of the distributed object storage cluster and effectively evaluate the baseline performance indicators.

[0006] On the one hand, the present disclosure provides a performance testing method for a distributed object storage cluster, the method comprising: obtaining target node information of a target storage cluster and data pool information of the target storage cluster, and determining an estimated performance indicator of the target storage cluster based on the target node information and the data pool information, the target node information comprising proxy server node information, gateway node information, data storage node information and management node information; creating one or more object test users according to the data pool information, and generating at least one test configuration file of a preset test tool for each of the object test users based on the proxy server node information; updating a client configuration file of the test configuration file based on the gateway node information and the data storage node information, and driving the preset test tool to perform a performance test on the target storage cluster using the updated test configuration file; judging whether the performance test result meets expectations according to the performance test result and the estimated performance indicator.

[0007] On the other hand, the present disclosure further provides a performance testing device for a distributed object storage cluster, the device comprising: an information acquisition unit, used to acquire target node information of a target storage cluster and data pool information of the target storage cluster, and determine an estimated performance indicator of the target storage cluster based on the target node information and the data pool information, wherein the target node information comprises proxy server node information, gateway node information, data storage node information and management node information; a configuration parsing unit, used to create one or more object test users according to the data pool information, and generate at least one test configuration file of a preset test tool for each object test user based on the proxy server node information; a performance testing unit, used to update a client configuration file of the test configuration file based on the gateway node information and the data storage node information, and drive the preset test tool to perform a performance test on the target storage cluster using the updated test configuration file; a performance comparison unit, used to determine whether the performance test result meets expectations based on the performance test result and the estimated performance indicator.

[0008] On the other hand, the present disclosure further provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the performance testing method of the distributed object storage cluster is implemented.

[0009] On the other hand, the present disclosure further provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, the performance testing method of the distributed object storage cluster is implemented.

[0010] The technical solution provided by one or more embodiments of the present disclosure can obtain the target node information and data pool information of the target storage cluster, and then first determine the estimated performance indicators of the target storage cluster. Subsequently, using the obtained target node information and data pool information, the steps of generating a test configuration file, updating the test configuration file, and driving the preset test tool can be automatically completed to achieve performance testing. Finally, the estimated performance indicators can also be used to judge the actual performance test results. This method helps to improve the continuity of cluster deployment and testing, and improve the efficiency of performance testing. At the same time, by combining the performance test results with the estimated performance indicators for comparative analysis, the baseline performance of the cluster can be grasped more quickly and performance optimization can be performed.

[0011] The technical solution provided by one or more embodiments of the present disclosure can coordinate the test server, client, and test resources to immediately carry out performance testing after a large-scale distributed object storage cluster is deployed or a version upgrade is completed, and accurately obtain cluster baseline performance indicators to effectively evaluate the performance test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The features and advantages of the embodiments of the present disclosure will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0013] Figure 1 A schematic diagram showing the steps of a performance testing method for a distributed object storage cluster in one embodiment of the present disclosure is shown;

[0014] Figure 2 A schematic diagram showing a flow chart of information acquisition steps in one embodiment of the present disclosure is shown;

[0015] Figure 3 A schematic diagram of a flow chart of a configuration parsing step in one embodiment of the present disclosure is shown;

[0016] Figure 4 A schematic diagram showing a flow chart of performance testing steps in one embodiment of the present disclosure is shown;

[0017] Figure 5 A schematic flow chart of a performance comparison step in one embodiment of the present disclosure is shown;

[0018] Figure 6 A schematic diagram of a process flow of a performance testing system for a distributed object storage cluster in one embodiment of the present disclosure is shown;

[0019] Figure 7 A schematic diagram of functional units of a performance testing device for a distributed object storage cluster in one embodiment of the present disclosure is shown;

[0020] Figure 8 A schematic structural diagram of an electronic device in one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0022] In the related art, the scale of the cluster to be delivered to the customer is usually many times higher than the scale of the conventional 3-node or 6-node test environment. Before the performance test of the conventional test cluster begins, it is necessary to make more configuration modifications to the test tool, such as defining the server and client, defining the input and output (IO) model to be tested, and the amount of data to be tested and the test time, and then start the test. The final performance index value is difficult to become a necessary and sufficient condition for evaluating the performance of a large-scale cluster because it does not take into account the impact of hardware, network or other processes on resource usage. Moreover, when facing a large-scale media storage cluster and requiring to obtain the baseline performance of the entire cluster within a limited time, repeated configuration modifications and manual data statistics increase the time cost. Complicated manual operations can also easily lead to inaccurate test data and reduce test efficiency.

[0023] In view of this, a performance testing method for a distributed object storage cluster provided in an embodiment of the present disclosure can efficiently and accurately obtain a baseline performance indicator of the distributed object storage cluster and effectively evaluate the baseline performance indicator.

[0024] See also Figure 1 A performance testing method for a distributed object storage cluster provided by an embodiment of the present disclosure may include the following steps.

[0025] S1: Obtain target node information of a target storage cluster and data pool information of the target storage cluster, and determine estimated performance indicators of the target storage cluster based on the target node information and the data pool information, wherein the target node information includes proxy server node information, gateway node information, data storage node information and management node information.

[0026] In this embodiment, the target storage cluster can be a distributed object storage cluster. The target nodes can be some common nodes of the distributed object storage cluster, such as: a proxy server node (Proxy, PXY), which is mainly used in the storage cluster to access the upper-layer service request and provide domain name access or IP address access; a gateway node (RadosGateway, RGW), which is mainly used in the storage cluster to access the object service request, convert the HTTP request of the front-end service into the semantics that the back-end application can recognize, and after the request is authenticated, call the application program interface (API) to perform different object processing operations; a data storage node (Data Node, DN), which is used in the storage cluster to deploy the cluster, create a storage pool to divide the storage space, and provide data storage services; a management node (Monitor Node, Mon), which is used in the storage cluster to maintain the status of cluster components including the monitor table (MonMap) and the cluster management table (MGRMap), to ensure the normal operation of the cluster.

[0027] In this embodiment, after obtaining the target node information and data pool information of the target storage cluster, the estimated performance index of the target storage cluster can be determined using a preset calculation formula or a preset performance evaluation model, so as to facilitate subsequent comparative analysis. The estimated performance index may include IOPS (Input / Output Operations Per Second), bandwidth representing the data transmission rate, delay time of read / write operations, etc.

[0028] In some embodiments, obtaining the target node information of the target storage cluster and the data pool information of the target storage cluster includes: in response to a trigger instruction of the application front-end page, obtaining the target cluster node prefix from the application front-end page; determining the target storage cluster based on the target cluster node prefix; obtaining the target node information in the target storage cluster; and obtaining the data pool information of the target storage cluster based on the data storage node information.

[0029] After the distributed storage cluster is deployed, it only needs a host name prefix to automatically call the test platform, automatically identify the test client and server, automatically generate the test configuration file, and then automatically distribute it to the corresponding node, and then automatically execute the performance test. This avoids the unknown risks caused by repeated manual modification of test scripts and test configurations, improves the efficiency of performance testing, and ensures the accuracy of the test.

[0030] See also Figure 2 In an actual application example, after the cluster management platform completes the deployment of the distributed object storage cluster, the performance test for the target storage cluster can be directly triggered on the cluster management page. Before the test starts, the complete test toolkit can be downloaded from the cluster management platform's affiliated tool library or network channel to the specified directory of the execution machine (jump server node) of the target storage cluster.

[0031] Specifically, through the pre-edited script tool, the deployed cluster node prefix, such as R0x-P0x, can be obtained from the front-end page of the cluster management platform. Based on the cluster node prefix, the node server information of all different types in the target storage cluster can be obtained, such as Mon, DN, PXY, RGW, etc. Based on the DN node information of the target storage cluster, the data pool information of the target storage cluster can be obtained, including the number of data pools and the type of data pools. Afterwards, based on the number of data pools, the number of DN nodes, the number of SSD disks and HDD disks corresponding to each DN node, and the redundancy ratio of the data pool, the preset performance estimation formula or preset performance estimation model can be combined to output the performance estimation results of the target storage cluster for different objects and save the performance estimation results.

[0032] In some embodiments, the data pool information includes the number of data pools, the data pool type (including mirror type and erasure code type) and the data pool redundancy ratio, the data storage node information includes the number of data storage nodes and the number of disks corresponding to each data storage node, and determining the estimated performance indicators of the target storage cluster based on the target node information and the data pool information includes: determining the estimated performance indicators of the target storage cluster based on the number of data pools, the data pool redundancy ratio, the number of data storage nodes, and the number of disks.

[0033] Before performing a performance test, based on the number of cluster data nodes, the number of disks per node, the storage pool redundancy ratio, and the input / output model to be tested, combined with the performance estimation calculation model summarized in practice, the cluster's limit performance under the specified input / output model can be estimated. The estimated performance indicators can be compared and analyzed with the actual test results to better determine whether the current cluster can meet the needs of the business launch.

[0034] In an actual application example, for a target storage cluster whose data pool type is erasure coding, the performance estimation method can be:

[0035] Small object write IOPS = avg_hdd_write_iops*hdd_osd_num / (EC_K+EC_M);

[0036] Small object read IOPS = avg_hdd_read_iops*hdd_osd_num / EC_K;

[0037] Large object write bandwidth

[0038] =avg_hdd_write_bw*hdd_osd_num*EC_K / (EC_K+EC_M);

[0039] Large object read bandwidth = avg_hdd_read_bw*hdd_osd_num;

[0040] Among them, large objects usually refer to large files, such as pictures, videos, etc. In object storage, large files are usually split into multiple blocks of equal size to improve read and write performance and facilitate management. Small objects refer to small files, such as configuration files, small pictures, etc. Compared with large objects, the read and write operations of small objects may be more frequent, but the data volume of a single file is smaller. EC_K can represent the value of the original data block in the erasure code configuration, EC_M can represent the value of the check data block in the erasure code configuration, hdd_osd_num can represent the number of object storage (OSD) in the entire cluster, and the avg_hdd_write parameter represents the performance baseline of each HDD hard disk. Depending on the number of RGWs in the target storage cluster, the value range of the avg_hdd_write parameter may be different.

[0041] S2: creating one or more object test users according to the data pool information, and generating at least one test configuration file of a preset test tool for each object test user based on the proxy server node information.

[0042] In this embodiment, the target node information and data pool information obtained can be used to automatically generate a test configuration file, which can improve the efficiency of performance testing and avoid errors caused by manual operations. The preset test tool can be an open source object storage performance test tool, such as Cosbench, or a dedicated test tool developed by the user.

[0043] In some implementations, the data pool information includes the number of data pools and the type of data pool. Creating one or more object test users according to the data pool information may include: determining the object storage type of the target storage cluster according to the data pool type; if the object storage type is the first object storage, creating one object test user; if the object storage type is the second object storage and there is only one data pool, creating one object test user; if the object storage type is the second object storage and there is more than one data pool, creating the same number of object test users as the number of data pools.

[0044] See also Figure 3 ,In a practical application example, the test client Cosbench ,configuration file can be automatically parsed and generated according to the ,node information and storage pool information automatically acquired.

[0045] Specifically, based on the data pool type, it can be determined whether the target storage cluster is a new type of object storage. Due to different architectures, new object storage only requires one object test user. If it is traditional object storage, you need to create one or more RGW object test users equal to the number of storage pools based on the number of data pools. For new object storage, one user can connect to all PXY nodes and all RGW nodes. For traditional object storage, each user corresponds to different PXY nodes and different RGW nodes, so the obtained PXY, RGW, and DN node information needs to be divided and saved according to the number of users.

[0046] The obtained PXY node information can be updated to the client Cosbench configuration file in the test tool. If there are multiple object user scenarios, multiple Cosbench configuration files can be generated in batches.

[0047] It is worth noting that in order to adapt to the situation where the target storage cluster does not have a PXY node, in some actual application scenarios, you can view the current client Cosbench configuration on the deployment page of the target storage cluster and return the Controller and Driver information of the client Cosbench tool according to the template. If the client Cosbench configuration is specified on the front-end page, the client Cosbench configuration file in the Cosbench tool can be parsed and updated when the performance test method is executed.

[0048] Driver is the load generator in Cosbench, which is mainly responsible for workload generation, initiating operations in the distributed object storage system, and collecting performance statistics. Controller is the controller of Cosbench, which can coordinate various drivers to jointly execute workloads and collect benchmark performance test results from drivers.

[0049] S3: Based on the gateway node information and the data storage node information, the client configuration file of the test configuration file is updated, and the preset test tool is driven to perform a performance test on the target storage cluster using the updated test configuration file.

[0050] In this embodiment, the obtained target node information and data pool information can be used to automatically update the test configuration file and drive the preset test tool to achieve performance testing. This helps to improve the continuity of cluster deployment and testing and improve the efficiency of performance testing.

[0051] In some embodiments, updating the client configuration file of the test configuration file based on the gateway node information and the data storage node information includes: if the object storage type is a first object storage, or a second object storage with only one data pool, updating the client configuration file based on the proxy server node information, the gateway node information, and the data storage node information; if the object storage type is a second object storage with multiple data pools, then according to the number of the object test users, cyclically reading the configuration file, the gateway node information, and the data storage node information corresponding to each object test user, and updating the client configuration file corresponding to each object test user.

[0052] See also Figure 4 In a practical application example, after reading the client Cosbench configuration, RGW node information and DN node information, you can update the PXY_Nodes, DN_Nodes, RGW_Nodes and other fields in the clients.yml file in the Cosbench tool. Start to execute the performance test.

[0053] If the target storage cluster is a new storage, or a common storage with only one data pool, before the test execution starts, you can use all the PXY, RGW, and DN node information to update the clients.yml file. If the target storage cluster is a common storage with multiple data pools, you can read the corresponding Cosbench configuration, RGW node information, and DN node information in a loop according to the number of users, and update the clients.yml file.

[0054] After completing the update of the clients.yml file, the Cosbench tool can automatically perform performance testing based on the pre-specified test model and test parameters.

[0055] S4: According to the performance test result and the estimated performance indicator, determine whether the performance test result meets expectations.

[0056] In this embodiment, by combining the performance test results with the estimated performance indicators for comparative analysis, the baseline performance of the cluster can be grasped more quickly and performance optimization can be performed. Based on the comparative analysis of the actual test results and the estimated performance indicators, it can be better judged whether the current cluster can meet the requirements of the business launch.

[0057] In some embodiments, judging whether the performance test result meets expectations based on the performance test result and the estimated performance indicator includes: determining a warning interval for the estimated performance indicator; judging whether the performance test result meets expectations based on whether the performance test result exceeds the warning interval; and feeding back warning information if the performance test result exceeds the warning interval.

[0058] Specifically, the performance test results may include the test process log and the final test results. Warning information can be fed back to the user in various forms such as text, pictures, videos, voice, indicator lights, etc. Warning information can be a summary report (for example, whether the current performance test results meet expectations), or a detailed analysis report (for example, showing in detail which local data in the test process log and the final test results differ from the estimated performance indicators).

[0059] In some embodiments, determining the warning interval of the estimated performance indicator includes: reducing the estimated performance indicator by a first preset value, determining it as the lower limit of the interval; increasing the estimated performance indicator by a second preset value, determining it as the upper limit of the interval; and determining the warning interval of the estimated performance indicator based on the upper limit of the interval and the lower limit of the interval.

[0060] Specifically, the warning interval can be determined by the estimated performance index. For example, the estimated performance index can be increased or decreased by a fixed value (some pre-trained reference values) to form the upper and lower limits of the warning interval; the estimated performance index can also be increased or decreased by a dynamic value matching the estimated performance index (for example, some proportional values ​​of the estimated performance index) to form the upper and lower limits of the warning interval.

[0061] See also Figure 5 In a practical application example, an interval range can be defined around the estimated performance index. Using this interval range, the measured performance value obtained by each test model can be compared with the estimated performance index value to determine whether the measured performance value meets expectations.

[0062] Specifically, the interval range can be defined as [-20%, 20%], which means that if the measured performance value falls within the [-20%, 20%] interval of the estimated performance index, the measured performance value meets expectations. Considering the influence of factors such as actual server hardware and network, the span of the interval range can be appropriately adjusted. If the comparison result between the measured performance value and the estimated performance index is not within the specified range of the estimated performance index, all test process data and test result data are collected and fed back to the user for performance tuning analysis.

[0063] See also Figure 6A performance testing system for a distributed object storage cluster provided by an embodiment of the present disclosure may include an information acquisition module, a configuration parsing module, a test execution module, and a performance comparison module. The workflow of the system may be as follows.

[0064] After the distributed object storage cluster is deployed on the deployment management platform, performance testing can be carried out. On the management platform page, there is only one "Start Test" button that needs to be added. Click the button, the page responds, obtains and passes the cluster node prefix name information to the performance test entry script, such as R03-P03. Then, with the cooperation of various modules, the entire system starts the entire performance test work.

[0065] The main function of the information acquisition module is to obtain all node information of different types with the same prefix according to the cluster node prefix passed from the front-end page, such as the IP address and hostname of the PXY node / DN node / RGW node.

[0066] The information acquisition module can obtain the data pool information in the cluster corresponding to the specified prefix, including the number of data pools and storage type. The data pool ends with .data and uses the cluster name as the prefix.

[0067] The information acquisition module can also estimate the performance of the cluster based on the number of cluster DN nodes, storage pool redundancy ratio and other configuration parameters, combined with the performance estimation model calculation formula or the defined performance estimation model, to facilitate comparative analysis after the test is completed. The estimated performance value can be saved in the specified directory of the script execution node.

[0068] Since the distributed object storage cluster is a converged architecture that supports two types of object storage, the configuration parsing module can determine whether the target storage cluster is a new type of object storage based on the data pool type of the storage cluster. Then, based on the judgment result, one or more test object users can be created.

[0069] If it is ordinary object storage and there are multiple data pools, the configuration parsing module can classify node information such as DN, PXY, RGW according to the number of data pools to ensure that the relationship between users and storage pools corresponds one to one, and also to ensure that the read and write requests executed by the client through each object user can fall on the corresponding node.

[0070] If it is a new type of object storage or there is only one data pool, the configuration parsing module can use all PXY nodes in the cluster as test clients and all RGW nodes as object request access nodes.

[0071] According to the obtained PXY node IP address and the object type, the configuration parsing module can automatically generate the Cosbench configuration file of the client in the performance testing tool.

[0072] The test execution module can drive the test tool to perform best practices based on different types of object storage. In addition, multiple different test resource parameters have been pre-fixed in the test tool code, including the number of concurrent clients, the number of test objects, and the test time.

[0073] For scenarios with multiple test users, the test execution module can cyclically read the corresponding client Cosbench configuration file, DN node information, and RGW node information according to the number of users, and update them to the client clients.yml file.

[0074] The test execution module can record all test process logs and final test results after the test script starts executing. If an exception occurs during the test and the test fails, the test execution module can automatically end the script process and the client Cosbench related process, and give an error prompt on the front-end page. After all the specified input and output models are executed normally, the test execution module can display the information returned by the test tool on the front-end page, so that users can easily identify the running status of the test tool.

[0075] The performance comparison module can use the estimated performance indicators determined by the information acquisition module to set the result comparison range. Based on the test results executed by each test model and the performance estimation indicator values ​​generated in the information acquisition module, the performance comparison module can determine whether the comparison results fall within the result comparison range. If they fall within the result comparison range, it means that the test results of the test model meet expectations. Otherwise, it means that the test results of the test model do not meet expectations and an alarm needs to be prompted on the front-end page.

[0076] For performance data that does not meet expectations, the performance comparison module can mark the results of real-time printing on the page with special colors, and submit relevant logs to users for performance optimization analysis after the test is completed.

[0077] It is necessary to explain that, considering the situation that the target storage cluster does not have PXY nodes, the performance test system of the distributed object storage cluster supports viewing only, or directly specifying the Controller and Driver configuration information of the Cosbench tool on the test client according to the template. In this way, the target storage cluster can be automatically tested and the test results can be obtained.

[0078] In this embodiment, the performance test system of the distributed object storage cluster can be integrated into the cluster deployment management platform, and a complete distributed storage cluster performance test can be implemented with just one button. The front-end page of the cluster deployment management platform can output the test process and comparison results in real time, which helps users to grasp the performance data of the cluster more quickly and better judge whether the performance of the cluster meets the online requirements.

[0079] In this embodiment, only a prefix needs to be specified to automatically identify all different types of nodes, automatically read cluster configuration information, and generate configuration files that meet test requirements, thereby improving the accuracy and efficiency of the test. Combined with the existing cluster performance estimation model, it can better and faster discover the performance weaknesses of the cluster in certain business scenarios, help R&D to optimize and improve performance in advance, and ensure the stability and reliability of the cluster after going online. The performance testing tool can be expanded to support all types of distributed storage, and then integrated into the deployment platform, so that it can face larger-scale clusters and significantly improve test efficiency. Fully automated testing eliminates the unknown testing risks brought by manual operations, and can also reduce the time and resource costs caused by performing tedious test configuration modifications.

[0080] The technical solution provided by one or more embodiments of the present disclosure can obtain the target node information and data pool information of the target storage cluster, and then first determine the estimated performance indicators of the target storage cluster. Subsequently, using the obtained target node information and data pool information, the steps of generating a test configuration file, updating the test configuration file, and driving the preset test tool can be automatically completed to achieve performance testing. Finally, the estimated performance indicators can also be used to judge the actual performance test results. This method helps to improve the continuity of cluster deployment and testing, and improve the efficiency of performance testing. At the same time, by combining the performance test results with the estimated performance indicators for comparative analysis, the baseline performance of the cluster can be grasped more quickly and performance optimization can be performed.

[0081] The technical solution provided by one or more embodiments of the present disclosure can coordinate the test server, client, and test resources to immediately carry out performance testing after a large-scale distributed object storage cluster is deployed or a version upgrade is completed, and accurately obtain cluster baseline performance indicators to effectively evaluate the performance test results.

[0082] For video storage (basically massive amounts of unstructured data of varying sizes), the technical solutions provided by one or more embodiments of the present disclosure can well simulate actual business scenarios and accurately obtain performance data before the storage cluster is put into production, which helps the R&D and operation and maintenance teams evaluate the stability of the cluster. Through fully automated performance testing and evaluation, R&D and operation and maintenance personnel can obtain more accurate performance data to provide data support for subsequent expansion and upgrade of the cluster.

[0083] See also Figure 7 The present disclosure also provides a performance testing device for a distributed object storage cluster, the device comprising:

[0084] The information acquisition unit 100 is used to acquire target node information of a target storage cluster and data pool information of the target storage cluster, and determine an estimated performance indicator of the target storage cluster based on the target node information and the data pool information, wherein the target node information includes proxy server node information, gateway node information, data storage node information and management node information;

[0085] The configuration parsing unit 200 is used to create one or more object test users according to the data pool information, and generate at least one test configuration file of a preset test tool for each object test user based on the proxy server node information;

[0086] The performance testing unit 300 is used to update the client configuration file of the test configuration file based on the gateway node information and the data storage node information, and drive the preset test tool to perform a performance test on the target storage cluster using the updated test configuration file;

[0087] The performance comparison unit 400 is used to determine whether the performance test result meets expectations based on the performance test result and the estimated performance index.

[0088] In one embodiment, the information acquisition unit 100 is specifically used to obtain the target cluster node prefix from the application front-end page in response to a trigger instruction of the application front-end page; determine the target storage cluster based on the target cluster node prefix; obtain the target node information in the target storage cluster; and obtain the data pool information of the target storage cluster based on the data storage node information.

[0089] In one embodiment, the data pool information includes the number of data pools, the data pool type and the data pool redundancy ratio, the data storage node information includes the number of data storage nodes and the number of disks corresponding to each data storage node, and the information acquisition unit 100 is specifically used to determine the estimated performance indicator of the target storage cluster based on the number of data pools, the data pool redundancy ratio, the number of data storage nodes, and the number of disks.

[0090] In one embodiment, the configuration parsing unit 200 is specifically used to determine the object storage type of the target storage cluster according to the data pool type; if the object storage type is the first object storage, create an object test user; if the object storage type is the second object storage and there is only one data pool, create an object test user; if the object storage type is the second object storage and the number of data pools is greater than one, create the same number of object test users as the number of data pools.

[0091] In one embodiment, the performance testing unit 300 is specifically used to update the client configuration file based on the proxy server node information, the gateway node information and the data storage node information if the object storage type is the first object storage, or the second object storage with only one data pool; if the object storage type is the second object storage with multiple data pools, then according to the number of the object test users, the configuration file, the gateway node information and the data storage node information corresponding to each object test user are cyclically read to update the client configuration file corresponding to each object test user.

[0092] In one embodiment, the performance comparison unit 400 is specifically used to determine the warning interval of the estimated performance indicator; based on whether the performance test result exceeds the warning interval, determine whether the performance test result meets expectations; if the performance test result exceeds the warning interval, feedback warning information.

[0093] In one embodiment, the performance comparison unit 400 includes an interval determination subunit 401, which is used to reduce the estimated performance indicator by a first preset value to determine it as the lower limit of the interval; increase the estimated performance indicator by a second preset value to determine it as the upper limit of the interval; and determine the warning interval of the estimated performance indicator based on the upper limit of the interval and the lower limit of the interval.

[0094] Each unit described in the above embodiments may be implemented by a computer chip or a product having a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0095] For the convenience of description, the above devices are described in terms of functions and are described separately in various units. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0096] See also Figure 8 The present disclosure also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the performance testing method of the distributed object storage cluster is implemented.

[0097] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the performance testing method of the distributed object storage cluster is implemented.

[0098] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0099] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0100] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0102] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0103] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0104] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A performance testing method for a distributed object storage cluster, characterized in that: The method comprises: Obtain target node information of a target storage cluster and data pool information of the target storage cluster, and determine estimated performance indicators of the target storage cluster based on the target node information and the data pool information, wherein the target node information includes proxy server node information, gateway node information, data storage node information, and management node information; Creating one or more object test users according to the data pool information, and generating at least one test configuration file of a preset test tool for each object test user based on the proxy server node information; Based on the gateway node information and the data storage node information, updating the client configuration file of the test configuration file, and driving the preset test tool to perform a performance test on the target storage cluster using the updated test configuration file; According to the performance test results and the estimated performance indicators, determine whether the performance test results meet expectations.

2. The method according to claim 1, characterized in that The acquiring target node information of the target storage cluster and data pool information of the target storage cluster includes: In response to a trigger instruction of an application front-end page, obtaining a target cluster node prefix from the application front-end page; Determining the target storage cluster according to the target cluster node prefix; Obtaining the target node information in the target storage cluster; The data pool information of the target storage cluster is acquired according to the data storage node information.

3. The method according to claim 1, characterized in that The data pool information includes the number of data pools, the type of data pool and the redundancy ratio of the data pool; the data storage node information includes the number of data storage nodes and the number of disks corresponding to each data storage node; and the estimated performance index of the target storage cluster is determined based on the target node information and the data pool information, including: The estimated performance index of the target storage cluster is determined according to the number of data pools, the redundancy ratio of the data pools, the number of data storage nodes, and the number of disks.

4. The method according to claim 1, characterized in that: The data pool information includes the number of data pools and the type of data pools, and the step of creating one or more object test users according to the data pool information includes: Determining the object storage type of the target storage cluster according to the data pool type; If the object storage type is the first object storage, creating an object test user; If the object storage type is the second object storage and there is only one data pool, create an object test user; If the object storage type is the second object storage and the number of data pools is greater than one, then create the same number of object test users as the number of data pools.

5. The method according to claim 1, characterized in that The updating of the client configuration file of the test configuration file based on the gateway node information and the data storage node information includes: If the object storage type is a first object storage, or a second object storage with only one data pool, updating the client configuration file based on the proxy server node information, the gateway node information and the data storage node information; If the object storage type is a second object storage with multiple data pools, then according to the number of the object test users, the configuration file, the gateway node information and the data storage node information corresponding to each object test user are cyclically read, and the client configuration file corresponding to each object test user is updated.

6. The method according to any one of claims 1 to 5, characterized in that: The step of judging whether the performance test result meets expectations based on the performance test result and the estimated performance indicator includes: Determining an early warning interval for the estimated performance indicator; Based on whether the performance test result exceeds the warning interval, judging whether the performance test result meets expectations; If the performance test result exceeds the warning interval, a warning message is fed back.

7. The method according to claim 6, characterized in that Determining the warning interval of the estimated performance indicator includes: The estimated performance indicator is reduced by a first preset value to determine the lower limit of the interval; The estimated performance indicator is increased by a second preset value to be determined as an upper limit of the interval; The warning interval of the estimated performance indicator is determined according to the interval upper limit and the interval lower limit.

8. A performance testing device for a distributed object storage cluster, characterized in that: The device comprises: An information acquisition unit, used to acquire target node information of a target storage cluster and data pool information of the target storage cluster, and determine an estimated performance indicator of the target storage cluster based on the target node information and the data pool information, wherein the target node information includes proxy server node information, gateway node information, data storage node information, and management node information; A configuration parsing unit, configured to create one or more object test users according to the data pool information, and generate at least one test configuration file of a preset test tool for each object test user based on the proxy server node information; A performance testing unit, configured to update a client configuration file of the test configuration file based on the gateway node information and the data storage node information, and drive the preset test tool to perform a performance test on the target storage cluster using the updated test configuration file; The performance comparison unit is used to determine whether the performance test result meets expectations based on the performance test result and the estimated performance index.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images