Performance test method and device for distributed storage system, equipment and medium

By sending correlated target requests to each node in a distributed storage system and determining the test result based on collective node responses, the method enhances the accuracy of performance testing to align with the system's operating logic.

CN119088629APending Publication Date: 2024-12-06JINAN INSPUR DATA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411231135.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-06

Smart Images

  • Figure CN119088629A_ABST
    Figure CN119088629A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of performance testing, and discloses a performance testing method and device for a distributed storage system, equipment and a medium. To-be-tested data are acquired, and a plurality of target requests corresponding to the to-be-tested data are generated; respectively issuing the plurality of target requests to each storage node in the distributed storage system to enable each storage node to respond to the received target requests, and returning request completion information after the response is completed; and obtaining request completion information returned by each storage node, and determining a test result of the to-be-tested data based on the request completion information. Therefore, when the performance test is carried out on the distributed storage system, the input and output requests received by the storage nodes are associated, so that the final test result is associated with the response conditions of the storage nodes, and the performance test on the distributed storage system is closer to the operation logic of the distributed storage system; therefore, the accuracy of performance testing of the distributed storage system is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of performance testing, and in particular to a performance testing method, device, equipment and medium for a distributed storage system. Background Art

[0002] When optimizing the performance of a distributed storage system, the performance testing tool of the upper layer interface of the storage system is often used to perform performance testing on the storage pool of the distributed storage system. The performance testing tool adds input and output channels and sends input and output requests to each device in the distributed storage system. There is no correlation between the input and output requests of each device. This is equivalent to starting multiple performance testing tool processes at the same time to perform performance testing on each device separately, rather than testing the overall performance of the distributed storage system. This leads to a large difference between the test results and the actual performance of the distributed storage system, and the test results are less accurate. Summary of the invention

[0003] In view of this, the present invention provides a performance test method, apparatus, device and medium for a distributed storage system to solve the problem of low accuracy in performance testing of the distributed storage system.

[0004] In a first aspect, the present invention provides a performance testing method for a distributed storage system, which is applied to a client of the distributed storage system, and the method comprises:

[0005] Acquire the data to be tested, and generate multiple target requests corresponding to the data to be tested; the multiple target requests are used to perform performance testing on the distributed storage system based on the data to be tested;

[0006] Sending multiple target requests to each storage node in the distributed storage system respectively, so that each storage node responds to the received target request and returns request completion information after the response is completed;

[0007] Obtain the request completion information returned by each storage node, and determine the test result of the data to be tested based on the request completion information.

[0008] In this way, the target requests corresponding to the data to be tested are respectively sent to the storage nodes in the distributed storage system, so that each storage node responds to the received target request, so that when the distributed storage system is performance tested, the input and output requests received by each storage node can be associated, and at the same time, the test results of the data to be tested are determined according to the response of each storage node to the target request, so that the final test result is associated with the response of each storage node, so that the performance test of the distributed storage system is closer to the operating logic of the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0009] In an optional implementation, generating multiple target requests corresponding to the data to be tested includes:

[0010] Based on the data to be tested, generate a preset number of data copies;

[0011] Input and output requests corresponding to a preset number of data copies are generated respectively to obtain multiple target requests.

[0012] In this way, input and output requests are generated based on the data copy of the data to be tested, thereby obtaining multiple interrelated input and output requests, and sending them to the storage nodes as target requests respectively, so that the target requests received by each storage node are interrelated, making the operating logic of the performance test closer to the operating logic of the distributed storage system, thereby improving the accuracy of the performance test.

[0013] In an optional implementation, generating multiple target requests corresponding to the data to be tested includes:

[0014] Splitting the data to be tested into a preset number of data segments;

[0015] Based on the erasure algorithm, a preset number of data fragments are calculated and processed to obtain multiple target requests.

[0016] In this way, the data to be tested is divided into data segments, and the data segments are processed using an erasure algorithm to obtain target requests, so that the target requests received by the storage nodes are correlated with each other, making the operating logic of the performance test closer to the operating logic of the distributed storage system, thereby improving the accuracy of the performance test.

[0017] In an optional implementation, a network device and a plurality of storage devices are deployed in each storage node of the distributed storage system; a network-based non-volatile memory storage service is deployed in the network device, and each network device is respectively connected to a client of the distributed storage system; in each storage node, the network device is connected to the plurality of storage devices;

[0018] Send multiple target requests to each storage node in the distributed storage system respectively, and each storage node responds to the received target request and returns request completion information after the response is completed, including:

[0019] The client sends multiple target requests to the network devices connected to the client of the distributed storage system respectively;

[0020] The network device responds to the target request and writes the data corresponding to the target request into each storage device connected to the network device;

[0021] If the storage device completes writing the data corresponding to the target request, it returns a response message to the connected network device;

[0022] If the network device receives response information returned by each connected storage device, it returns request completion information to the client.

[0023] In this way, by connecting the storage nodes and clients of the distributed storage system in series through network devices and deploying network-based non-volatile memory storage services on the network devices, cross-node access to storage devices can be achieved, thereby generating sufficient pressure on the network and storage of the distributed storage system to reach the performance ceiling, thereby improving the accuracy of performance testing of the distributed storage system.

[0024] In an optional implementation, determining the test result of the to-be-tested data based on the request completion information includes:

[0025] If the number of request completion messages received within the preset time is greater than the preset number, the test result is determined to be a test success;

[0026] If the number of request completion messages received within the preset time is less than the preset number, the test result is determined to be a test failure.

[0027] In this way, the test results of the data to be tested are determined based on the response completion status of each storage node to the target request within a preset time, thereby associating the response completion status of each storage node, so that the test results can reflect the actual response status of each storage node in the distributed storage system, thereby improving the accuracy of performance testing of the distributed storage system.

[0028] In an optional implementation, determining the test result of the to-be-tested data based on the request completion information further includes:

[0029] If the number of request completion messages received within the preset time is greater than the preset number, and the time range of receiving the request completion messages is within the preset time range, then the test result is determined to be a test success;

[0030] If the number of request completion messages received within the preset time is greater than the preset number, and the time range of receiving the request completion messages exceeds the preset time range, the test result is determined to be a test failure.

[0031] In this way, based on the response completion status of each storage node to the target request within the preset time, combined with the response time range of each storage node, the response completion status and response completion rate of each storage are comprehensively considered, so that the test results can reflect the actual response status of each storage node in the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0032] In an optional implementation, if there are multiple groups of data to be tested, the method further includes:

[0033] Generate and issue multiple target requests corresponding to multiple groups of test data respectively, so as to perform performance test on the distributed storage system and obtain test results of the multiple groups of test data;

[0034] Based on the test results of multiple groups of data to be tested, the performance test results of the distributed storage system are determined.

[0035] In this way, the performance test result of the distributed storage system is obtained by aggregating the test results of multiple groups of data to be tested, thereby improving the accuracy of the performance test of the distributed storage system.

[0036] In a second aspect, the present invention provides a performance testing device for a distributed storage system, which is applied to a client of the distributed storage system, and the device includes:

[0037] A target request generation module is used to obtain the data to be tested and generate multiple target requests corresponding to the data to be tested; the multiple target requests are used to perform performance testing on the distributed storage system based on the data to be tested;

[0038] The target request response module is used to send multiple target requests to each storage node in the distributed storage system respectively, so that each storage node responds to the received target request and returns request completion information after the response is completed;

[0039] The test result determination module is used to obtain the request completion information returned by each storage node, and determine the test result of the data to be tested based on the request completion information.

[0040] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the performance testing method of a distributed storage system according to the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0041] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the performance testing method for a distributed storage system of the first aspect or any corresponding embodiment thereof.

[0042] In a fifth aspect, the present invention provides a computer program product, including computer instructions, and the computer instructions are used to enable a computer to execute the performance testing method of a distributed storage system of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a schematic diagram of a performance test of a distributed storage system in the related art;

[0045] Figure 2 It is a schematic diagram of a process of performing performance testing on a distributed storage system in the related art;

[0046] Figure 3 is a flow chart of a performance testing method for a distributed storage system according to an embodiment of the present invention;

[0047] Figure 4 is a system architecture diagram of a distributed storage system in a performance testing method of a distributed storage system provided by an embodiment of the present invention;

[0048] Figure 5 is a flow chart of another distributed storage system performance testing method according to an embodiment of the present invention;

[0049] Figure 6 is a schematic diagram of a specific implementation of a performance testing method for a distributed storage system according to an embodiment of the present invention;

[0050] Figure 7 is a structural block diagram of a performance testing device for a distributed storage system according to an embodiment of the present invention;

[0051] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0053] When optimizing the performance of a distributed storage system, a performance testing tool of the upper layer interface of the storage system is often used to perform a performance test on the storage pool of the distributed storage system. Figure 1It is a schematic diagram of the performance test of the distributed storage system in the related technology, such as Figure 1 As shown, the performance test tool increases the input and output channels and uses multiple queues to send input and output requests to each device in the distributed storage system. There is no correlation between the input and output requests of each device. This is equivalent to starting multiple performance test tool processes at the same time to perform performance tests on each device separately, rather than testing the overall performance of the distributed storage system. This leads to a large difference between the test results and the actual performance of the distributed storage system, and the test results are less accurate. In addition, Figure 2 It is a flowchart of the performance test of the distributed storage system in the related technology, such as Figure 2 As shown, in the related technology, input and output requests are submitted to each storage device in the distributed storage system separately. Since there is no correlation between the input and output requests of each device, when any storage device completes the response to the input and output request, the input and output request can be marked as completed, and the next round of input and output request issuance and testing can be restarted. This causes the performance test of the distributed storage system to deviate from the operating logic of the distributed storage system itself, resulting in low accuracy of the test results.

[0054] Based on this, an embodiment of the present invention provides a performance testing method for a distributed storage system, which is applied to a client of a distributed storage system, and the method includes: obtaining data to be tested, and generating multiple target requests corresponding to the data to be tested; the multiple target requests are used to perform performance testing on the distributed storage system based on the data to be tested; the multiple target requests are respectively sent to each storage node in the distributed storage system, so that each storage node responds to the received target request, and returns request completion information after the response is completed; the request completion information returned by each storage node is obtained, and the test result of the data to be tested is determined based on the request completion information. In this way, the target requests corresponding to the data to be tested are respectively sent to the storage nodes in the distributed storage system, so that each storage node responds to the received target request, so that when the distributed storage system is subjected to performance testing, the input and output requests received by each storage node can be associated, and the test result of the data to be tested is determined according to the response of each storage node to the target request, so that the final test result is associated with the response of each storage node, so that the performance test of the distributed storage system is closer to the running logic of the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0055] According to an embodiment of the present invention, an embodiment of a performance testing method for a distributed storage system is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0056] In this embodiment, a performance test method for a distributed storage system is provided, which can be used in the above-mentioned distributed storage system, and specifically, can be applied to a client of the distributed storage system. Figure 3 FIG. 1 is a flow chart of a performance testing method for a distributed storage system according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0057] Step S301, obtaining data to be tested, and generating multiple target requests corresponding to the data to be tested.

[0058] In the embodiment of the present invention, the target request is an input / output request (IO), and multiple target requests are used to perform performance testing on a distributed storage system based on the data to be tested. Since multiple target requests correspond to the data to be tested, multiple target requests are related to each other and have a certain degree of correlation in data, so when the performance test of the distributed storage system is performed based on the target request, the associated input / output request can be responded to, so that the test process is more in line with the operation logic of the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0059] In an optional implementation, a performance testing tool is built into the client of the distributed storage system, and the data to be tested can be randomly generated by the performance testing tool.

[0060] In an optional implementation, in order to make the generated multiple target requests correlated, the data to be tested may be copied or erased, and input and output requests may be generated on this basis to obtain multiple target requests.

[0061] In an optional implementation, obtaining a target request through copy processing may include the following operations: first, based on the data to be tested, a preset number of data copies is generated; wherein the number of data copies, that is, the preset number, may be consistent with the number of storage nodes or storage devices in the distributed storage system, and may also be adjusted according to the test requirements; then, input and output requests corresponding to the preset number of data copies are generated respectively to obtain multiple target requests. Since the data corresponding to the multiple target requests are data copies of the data to be tested, the multiple target requests are interrelated, and the target requests are sent to the storage nodes respectively. The target requests received and responded to by each storage node are also interrelated, and the operating logic of the performance test based on the target request is closer to the operating logic of the distributed storage system, thereby improving the accuracy of the performance test.

[0062] In an optional implementation, obtaining a target request through erasure processing may include the following operations: first, dividing the data to be tested into a preset number of data fragments; wherein the number of data copies, that is, the preset number, may be consistent with the number of storage nodes or storage devices in the distributed storage system, and may also be adjusted according to test requirements; then, based on an erasure algorithm, calculating and processing the preset number of data fragments to obtain multiple target requests, wherein the erasure algorithm used may be adjusted according to detection requirements, for example, erasure algorithms such as Reed-Solomon code (RS code), Hamming code (Hamming Code), and parity check code (Parity Check Code) may be used, and no specific restrictions are made here. Since the data to be tested is divided into data fragments and the data fragments are processed using an erasure algorithm to obtain target requests, the data corresponding to multiple target requests constitute a whole of the data to be tested, and the target requests are interrelated. At the same time, in terms of data integrity, only the data corresponding to multiple target requests can constitute complete data. Moreover, since distributed storage systems often split data into multiple fragments when storing data, and store the multiple fragments on different storage devices respectively, the method of obtaining target requests using erasure processing is closer to the operating logic of the distributed storage system, thereby improving the accuracy of performance testing.

[0063] In an optional implementation, when a target request is generated based on the data to be tested, only the data to be tested may be processed as a copy, only the data to be tested may be processed as an erasure, or both the data to be tested may be processed as a copy and as an erasure. That is, a part of the target request is generated based on the data copy of the data to be tested, the data to be tested is divided into multiple data segments for erasure, and then a part of the target request is generated, and then the generated target request is sent to different storage nodes or storage devices.

[0064] In an optional implementation, when generating a target request corresponding to the data to be tested, the processing method for the data to be tested can be set according to the test requirements, that is, whether the data to be tested is to be subjected to copy processing, erasure processing, or both copy processing and erasure processing. If copy processing and erasure processing are performed simultaneously, the target request generated based on the data copy obtained by copy processing and the target request generated based on the data fragment obtained by erasure processing are respectively sent to the storage node or storage device, so that different types of target requests are used to perform different performance tests on different storage nodes or storage devices in the distributed storage system, thereby testing the distributed storage system under a variety of storage conditions or storage environments, and improving the accuracy of the performance test of the distributed storage system.

[0065] Step S302: Send multiple target requests to each storage node in the distributed storage system respectively, so that each storage node responds to the received target request and returns request completion information after the response is completed.

[0066] In an embodiment of the present invention, multiple target requests are respectively sent to each storage node in the distributed storage system. Each storage node generates a response after receiving the target request, writes the data corresponding to the target request into the storage device to complete the response to the target request, and returns request completion information to the client of the distributed storage system after the response is completed to indicate that the storage node has completed the response to the sent target request.

[0067] In an optional implementation, when generating target requests corresponding to the data to be tested, the storage nodes or storage devices corresponding to these target requests can be set, that is, the storage nodes or storage devices to which each target request is issued can be set, thereby realizing the performance test of specific storage nodes or storage devices.

[0068] Step S303: Obtain request completion information returned by each storage node, and determine the test result of the data to be tested based on the request completion information.

[0069] In the embodiment of the present invention, the request completion information returned by each storage node is obtained, and the test result of the data to be tested is determined according to the request completion information returned by each storage node. Specifically, the test result of the data to be tested is determined according to the comparison between the number of the issued target requests and the number of the request completion information returned by each storage node.

[0070] In an optional embodiment, if the client receives request completion information that is greater than a preset number, or receives request completion information that is equal to the target request number without setting a preset number, that is, when all storage nodes or storage devices return request completion information, it is determined that the input and output requests corresponding to the data to be tested are completed, and the test result of the distributed storage system based on the data to be tested is successful, that is, the performance of the distributed storage system can meet the current data transmission requirements.

[0071] In an optional implementation, if there is a need for further performance testing of the response time of the distributed storage system, a preset time can also be set to limit the return time of the request completion information. Specifically, if within the preset time, the number of request completion information received is greater than the preset number, it indicates that at this time, each storage node or storage device can complete the response to the received target request within the required response time, so the test result is determined to be a test success, indicating that the performance of the distributed storage system can meet the current data transmission and response time requirements. If within the preset time, the number of request completion information received is less than the preset number, it indicates that at this time, there is a storage node or storage device that cannot complete the response to the received target request within the required response time, so the test result is determined to be a test failure, indicating that the performance of the distributed storage system cannot meet the current response time requirements. Thus, through the above method, according to the response completion status of each storage node to the target request within the preset time, the test result of the data to be tested is determined, thereby associating the response completion status and response time of each storage node, so that the test result can reflect the actual response status and response time of each storage node in the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0072] In an optional implementation, if there is a need for further performance testing of the synchronization rate of the response of each storage node or storage device to a request in the distributed storage system, that is, there is a need for testing the time deviation between the response times of each storage node or storage device, a preset time range can also be set to limit the time deviation range of the response time of each storage node or storage device. Specifically, if within the preset time, the number of request completion information received is greater than the preset number, and the receiving time range of the request completion information is within the preset time range, it indicates that the response time of each storage node or storage device is within the required time deviation range, so the test result is determined to be a successful test, indicating that the performance of the distributed storage system can meet the current data transmission, response time and synchronization rate requirements of the response time. If within the preset time, the number of request completion information received is greater than the preset number, and the receiving time range of the request completion information exceeds the preset time range, it indicates that the response time of each storage node or storage device is not within the required time deviation range, so the test result is determined to be a test failure, indicating that the performance of the distributed storage system cannot meet the synchronization rate requirements of the response time. Therefore, through the above method, on the basis of the response completion status of each storage node to the target request within the preset time, combined with the response time range of each storage node, the response completion status, response time and response synchronization rate of each storage are comprehensively considered, so that the test results can reflect the actual response status of each storage node in the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0073] In an optional implementation, if there are multiple groups of data to be tested, multiple target requests corresponding to the multiple groups of data to be tested are generated and issued respectively to perform performance tests on the distributed storage system and obtain test results of the multiple groups of data to be tested; the distributed storage system is sequentially tested based on the multiple groups of data to be tested to obtain test results of the multiple groups of data to be tested, wherein the data processing methods of the multiple groups of data to be tested, that is, the methods of copy processing or erasure calculation, can be different, the number of target requests generated, the storage nodes or storage devices issued can also be different, and the test content, that is, the data storage capacity, response time, response synchronization rate and other test contents of the distributed storage system can be tested respectively. One or more of them. After obtaining the test results of the multiple groups of data to be tested, based on the test results of the multiple groups of data to be tested, all the test results are combined to determine the performance test results of the distributed storage system. Thus, the performance test results of the distributed storage system are obtained according to the test results of the multiple groups of data to be tested, thereby improving the accuracy of the performance test of the distributed storage system.

[0074] The performance testing method of a distributed storage system provided by an embodiment of the present invention sends target requests corresponding to the data to be tested to the storage nodes in the distributed storage system respectively, so that each storage node responds to the received target request, so that when the performance test of the distributed storage system is performed, the input and output requests received by each storage node can be associated, and at the same time, the test result of the data to be tested is determined according to the response of each storage node to the target request, so that the final test result is associated with the response of each storage node, so that the performance test of the distributed storage system is closer to the operating logic of the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0075] In an optional embodiment, Figure 4 is a system architecture diagram of a distributed storage system in a performance testing method of a distributed storage system provided by an embodiment of the present invention, such as Figure 4 As shown, network devices and multiple storage devices are deployed in each storage node of the distributed storage system. Among them, a network-based non-volatile memory storage (NVM Express over Fabrics, NVMf) service is deployed in the network device; NVMf, as a network protocol, can enable storage devices to communicate with remote hosts through the network, which can extend the high performance and low latency characteristics of NVMe (NVM Express, non-volatile memory storage) storage to a wider network environment, thereby realizing remote access and sharing of storage resources, and realizing cross-node access to storage devices through NVMf services, which can generate enough pressure on the network and storage of the distributed storage system to reach the performance ceiling, and test the hardware performance of the distributed storage system through the network device deployed with the NVMf service, and the test results obtained can be closer to the performance ceiling of the distributed storage system. Each network device of each storage node is connected to the client of the distributed storage system respectively, and a performance testing tool is deployed in the client of the distributed storage system. The performance testing tool generates the data to be tested and submits it to the queue, performs copy processing or erasure calculation on it, and obtains multiple target requests. In each storage node, a network device is connected to multiple storage devices, that is, each storage node may include multiple storage devices, which are composed of 1 network device and at least 1 storage device. The multiple storage devices are connected to the client of the distributed storage system through the network device.

[0076] This embodiment provides a performance testing method for a distributed storage system, which can be used for the above Figure 4 In the distributed storage system shown in , specifically, it can be applied to the client of the distributed storage system, Figure 5 FIG. 1 is a flow chart of another distributed storage system performance testing method according to an embodiment of the present invention. Figure 5As shown, the process includes the following steps:

[0077] Step S501, obtain the data to be tested, and generate multiple target requests corresponding to the data to be tested. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0078] Step S502: Send multiple target requests to each storage node in the distributed storage system respectively, so that each storage node responds to the received target request and returns request completion information after the response is completed.

[0079] Specifically, step S502 is implemented in the following manner:

[0080] Step S5021: The client sends multiple target requests to network devices connected to the client of the distributed storage system respectively.

[0081] In an embodiment of the present invention, the client sends multiple target requests to the network device connected to the corresponding storage nodes or storage devices according to the storage nodes or storage devices corresponding to the set target requests, so as to send the target requests to the corresponding storage nodes or storage devices through the network device.

[0082] Step S5022: The network device responds to the target request and writes the data corresponding to the target request into each storage device connected to the network device.

[0083] In the embodiment of the present invention, after receiving the target request, the network device generates a response to the target request and writes the data corresponding to the target request into the storage device corresponding to the target request, so that the storage device responds to the target request to complete the target request.

[0084] Step S5023: If the storage device completes writing the data corresponding to the target request, it returns response information to the connected network device.

[0085] In the embodiment of the present invention, if the storage device completes writing the data corresponding to the target request, it indicates that the storage device has completed the response to the target request, and the storage device returns response information to the network device connected thereto to indicate that the target request has been completed.

[0086] Step S5024: If the network device receives response information returned by each connected storage device, it returns request completion information to the client.

[0087] In the embodiment of the present invention, if the network device receives the response information returned by each connected storage device, it returns the request completion information to the client to indicate that the storage node has completed the corresponding target request. Optionally, if the target request is issued corresponding to the storage node, after all storage devices in the storage node return the response information to the connected network device, the network device returns the request completion information to the client; if the target request is issued corresponding to the storage device, after the storage device returns the response information to the connected network device, the network device directly returns the request completion information to the client.

[0088] Through the above method, the storage nodes and clients of the distributed storage system are connected in series through network devices, and a network-based non-volatile memory storage service is deployed on the network devices, so that cross-node access to storage devices can be achieved, thereby generating sufficient pressure on the network and storage of the distributed storage system to reach the performance ceiling, thereby improving the accuracy of performance testing of the distributed storage system.

[0089] Step S503: Obtain the request completion information returned by each storage node, and determine the test result of the data to be tested based on the request completion information. Figure 1 Step S403 of the illustrated embodiment will not be described in detail here.

[0090] Figure 6 is a schematic diagram of a specific implementation of a performance testing method for a distributed storage system according to an embodiment of the present invention. Figure 6 The performance testing method of the distributed storage system provided by the embodiment of the present invention is further described. Figure 6 As shown, after generating the data to be tested, the performance testing tool submits the input and output requests corresponding to the data to be tested to initiate the performance test of the distributed storage system based on the data to be tested, generates multiple request fragments, i.e., multiple target requests, through copy processing or erasure calculation, and sends them to the storage devices in the corresponding storage nodes through the network device connected to the distributed storage system and deployed with NVMf, the storage device responds to the received target request, writes the data corresponding to the target request, and then returns the response information, the network device receives the response information returned by the connected storage device, and returns the request completion information to the client. When the client receives more than the preset number of request completion information, i.e., each storage device returns the request completion information or the number of returned request completion information reaches the requirement, it indicates that the input and output requests of the group of data to be tested are completed, and the input and output requests of the next group of data to be tested can be responded to, thereby testing the performance of the distributed storage system based on the next group of data to be tested. Thus, in the above manner, the performance of the distributed storage system is tested, and the performance of the distributed storage system is determined according to the completion of the issued target request.

[0091] The performance test method of the distributed storage system provided by the embodiment of the present invention is applied to the client of the distributed storage system, and the method includes: obtaining data to be tested, and generating multiple target requests corresponding to the data to be tested; the multiple target requests are used to perform performance test on the distributed storage system based on the data to be tested; the multiple target requests are respectively sent to each storage node in the distributed storage system, so that each storage node responds to the received target request, and returns request completion information after the response is completed; the request completion information returned by each storage node is obtained, and the test result of the data to be tested is determined based on the request completion information. In this way, the target requests corresponding to the data to be tested are respectively sent to the storage nodes in the distributed storage system, so that each storage node responds to the received target request, so that when the performance test of the distributed storage system is performed, the input and output requests received by each storage node can be associated, and the test result of the data to be tested is determined according to the response of each storage node to the target request, so that the final test result is associated with the response of each storage node, so that the performance test of the distributed storage system is closer to the operation logic of the distributed storage system, thereby improving the accuracy of the performance test of the distributed storage system.

[0092] In this embodiment, a performance test device for a distributed storage system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0093] This embodiment provides a performance testing device for a distributed storage system, which can be applied to a client of the distributed storage system, such as Figure 7 As shown, including:

[0094] The target request generation module 710 is used to obtain the data to be tested and generate multiple target requests corresponding to the data to be tested; the multiple target requests are used to perform performance testing on the distributed storage system based on the data to be tested;

[0095] The target request response module 720 is used to send multiple target requests to each storage node in the distributed storage system respectively, so that each storage node responds to the received target request and returns request completion information after the response is completed;

[0096] The test result determination module 730 is used to obtain the request completion information returned by each storage node, and determine the test result of the data to be tested based on the request completion information.

[0097] In an optional implementation, the target request generation module 710 includes:

[0098] A data copy generating unit, used for generating a preset number of data copies based on the data to be tested;

[0099] The target request generating unit is used to respectively generate input and output requests corresponding to a preset number of data copies to obtain multiple target requests.

[0100] In an optional implementation, the target request generation module 710 includes:

[0101] The test data segmentation unit is used to segment the test data into a preset number of data segments;

[0102] The target request generation unit is used to calculate and process a preset number of data fragments based on an erasure algorithm to obtain multiple target requests.

[0103] In an optional implementation, a network device and a plurality of storage devices are deployed in each storage node of the distributed storage system; a network-based non-volatile memory storage service is deployed in the network device, and each network device is respectively connected to a client of the distributed storage system; in each storage node, the network device is connected to the plurality of storage devices;

[0104] The target request response module 720 includes:

[0105] A target request sending unit, used for the client to send multiple target requests to the network device connected to the client of the distributed storage system respectively;

[0106] The data writing unit is used for the network device to respond to the target request and write the data corresponding to the target request into each storage device connected to the network device;

[0107] A response information returning unit, configured to return response information to the connected network device if the storage device completes writing the data corresponding to the target request;

[0108] The request completion information returning unit is used to return the request completion information to the client if the network device receives the response information returned by each connected storage device.

[0109] In an optional implementation, the test result determination module 730 includes:

[0110] A test success unit, configured to determine that the test result is a test success if the number of request completion messages received within a preset time is greater than a preset number;

[0111] The test failure unit is used to determine that the test result is a test failure if the number of request completion information received within a preset time is less than a preset number.

[0112] In an optional implementation, the test result determination module 730 further includes:

[0113] The test success unit is further used to determine that the test result is a test success if the number of request completion information received within a preset time is greater than a preset number and the time range of receiving the request completion information is within a preset time range;

[0114] The test failure unit is also used to determine that the test result is a test failure if the number of request completion information received within a preset time is greater than a preset number, and the receiving time range of the request completion information exceeds the preset time range.

[0115] In an optional implementation, if there are multiple groups of data to be tested, the device further includes:

[0116] A test result determination module is used to respectively generate and issue multiple target requests corresponding to multiple groups of data to be tested, so as to perform performance testing on the distributed storage system and obtain test results of the multiple groups of data to be tested;

[0117] The performance test result determination module is used to determine the performance test result of the distributed storage system based on the test results of multiple groups of data to be tested.

[0118] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0119] The performance testing device of the distributed storage system in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0120] The embodiment of the present invention also provides a computer device having the above Figure 7 The performance test device of the distributed storage system is shown.

[0121] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0122] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0123] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0124] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0125] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0126] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0127] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0128] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0129] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the present invention.

Claims

1. A performance testing method for a distributed storage system, characterized in that: Applied in a client of a distributed storage system, the method includes: Acquire data to be tested, and generate multiple target requests corresponding to the data to be tested; the multiple target requests are used to perform performance testing on the distributed storage system based on the data to be tested; Sending the plurality of target requests to the respective storage nodes in the distributed storage system respectively, so that each storage node responds to the received target request and returns request completion information after the response is completed; The request completion information returned by each of the storage nodes is obtained, and based on the request completion information, a test result of the data to be tested is determined.

2. The method according to claim 1, characterized in that The generating a plurality of target requests corresponding to the data to be tested includes: Based on the data to be tested, generating a preset number of data copies; Input and output requests corresponding to the preset number of data copies are generated respectively to obtain a plurality of target requests.

3. The method according to claim 1, characterized in that The generating a plurality of target requests corresponding to the data to be tested includes: Dividing the to-be-tested data into a preset number of data segments; Based on an erasure correction algorithm, the preset number of data fragments are calculated and processed to obtain a plurality of the target requests.

4. The method according to claim 1, characterized in that: A network device and a plurality of storage devices are deployed in each storage node of the distributed storage system; a network-based non-volatile memory storage service is deployed in the network device, and each of the network devices is respectively connected to a client of the distributed storage system; in each of the storage nodes, the network device is connected to a plurality of the storage devices; The sending of the plurality of target requests to the respective storage nodes in the distributed storage system, each storage node responding to the received target request, and returning request completion information after the response is completed, includes: The client sends the plurality of target requests to the network device connected to the client of the distributed storage system respectively; The network device writes the data corresponding to the target request into each of the storage devices connected to the network device in response to the target request; If the storage device completes writing the data corresponding to the target request, it returns response information to the connected network device; If the network device receives the response information returned by each of the connected storage devices, it returns the request completion information to the client.

5. The method according to claim 1, characterized in that: The step of determining the test result of the data to be tested based on the request completion information includes: If the number of the request completion information received within the preset time is greater than the preset number, the test result is determined to be a test success; If the number of the request completion information received within the preset time is less than the preset number, the test result is determined to be a test failure.

6. The method according to claim 5, characterized in that The step of determining the test result of the data to be tested based on the request completion information further includes: If within the preset time, the number of the request completion information received is greater than the preset number, and the receiving time range of the request completion information is within the preset time range, then the test result is determined to be a test success; If the number of the request completion information received within the preset time is greater than the preset number, and the receiving time range of the request completion information exceeds the preset time range, the test result is determined to be a test failure.

7. The method according to claim 1, characterized in that If there are multiple groups of the data to be tested, the method further includes: Respectively generate and issue a plurality of target requests corresponding to a plurality of groups of the data to be tested, so as to perform a performance test on the distributed storage system and obtain test results of the plurality of groups of the data to be tested; Based on the test results of the multiple groups of data to be tested, the performance test results of the distributed storage system are determined.

8. A performance testing device for a distributed storage system, characterized in that: Applied in a client of a distributed storage system, the device comprises: A target request generation module, used to obtain data to be tested and generate multiple target requests corresponding to the data to be tested; the multiple target requests are used to perform performance testing on the distributed storage system based on the data to be tested; A target request response module, used to send the multiple target requests to each storage node in the distributed storage system respectively, so that each storage node responds to the received target request and returns request completion information after the response is completed; The test result determination module is used to obtain the request completion information returned by each storage node, and determine the test result of the data to be tested based on the request completion information.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the performance storage method of the distributed storage system according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the performance storage method of the distributed storage system according to any one of claims 1 to 7.