A performance test method, device and equipment of a distributed storage engine and a medium
By initializing the storage engine and worker threads on the local node, performing read and write operations, and collecting performance information, the interference problem in storage engine performance testing in existing technologies is solved, achieving highly accurate and controllable performance testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JINAN INSPUR DATA TECH CO LTD
- Filing Date
- 2022-10-21
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies make it difficult to conduct detailed performance analysis without introducing interference when testing the performance of Ceph storage engines. Cluster testing solutions have unnecessary impacts, while unit testing solutions cannot meet the needs of long-term, highly complex testing scenarios.
The storage engine, worker threads, and test objects are initialized on the local node. Read and write operations are performed through the worker threads to collect performance information, and the test ends when preset conditions are met. This avoids OSD layer logic processing in cluster mode and reduces the impact on upper-layer applications.
It enables high-accuracy and controllable performance testing of storage engines in non-cluster mode, and improves the verification of storage engine support for hard drives.
Smart Images
Figure CN115454806B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a performance testing method, apparatus, device, and medium for a distributed storage engine. Background Technology
[0002] With the development of Ceph (distributed file system) and the contributions of the community, objectstore, as the storage engine of Ceph osd (Object-based Storage Device), is also constantly iterating and optimizing. So far, there are storage engines such as filestore and bluestore, and the next-generation storage engine seastore has entered the testing phase.
[0003] Because of the different hard drives used, every storage engine has its limitations, failing to fully meet the needs of the hard drives or fully utilize their advantages. Therefore, optimization and modification of the storage engine are often necessary to improve its storage performance. However, most current testing methods require building a cluster and using Rados (ReliableAutonomic Distributed Object Store) tools to test the overall performance of Ceph. See [link to relevant documentation]. Figure 1 As shown, cluster testing can introduce unnecessary interference, making it difficult to perform more detailed analysis and testing of the storage engine.
[0004] While Ceph's native testing tools do provide unit tests for storage engines such as Bluestore, these unit tests primarily focus on testing the functionality of the storage engines. They only offer some simple and single test scenarios, making them unsuitable for long-term, highly complex testing scenarios and unable to meet the need for comprehensive performance testing of the storage engines.
[0005] Therefore, how to design a simple and effective tool that can test storage engines independently and meet the need for overall performance testing of storage engines is an urgent problem to be solved in this field. Summary of the Invention
[0006] In view of this, the purpose of this invention is to provide a performance testing method, apparatus, device, and medium for distributed storage engines, which can avoid introducing unnecessary interference and meet the need for overall performance testing of storage engines. The specific solution is as follows:
[0007] Firstly, this application discloses a performance testing method for a distributed storage engine, including:
[0008] Initialize global public information, and create a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters;
[0009] The test objects are evenly distributed to the worker threads, and the worker threads are started. Then, the read and write operations of the storage engine are completed by executing the worker threads.
[0010] The performance information of the storage engine is collected after each worker thread completes a read / write operation. When any worker thread meets the preset test termination condition, the execution process of the worker thread is terminated to complete the performance test.
[0011] Optionally, the performance testing method for the distributed storage engine includes:
[0012] A number of sets are evenly distributed to a number of worker threads; wherein each set includes a number of test objects.
[0013] Optionally, the step of creating a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters includes:
[0014] Based on preset configuration parameters, a storage engine, a number of worker threads, a number of test objects, and a number of collections are created on the local node.
[0015] Optionally, the collection of storage engine performance information statistically analyzed by each worker thread after completing read / write operations includes:
[0016] At preset time intervals, the performance information of the storage engine is collected by each worker thread after completing read and write operations.
[0017] Optionally, the step of executing the worker thread to complete the read and write operations of the storage engine includes:
[0018] The operation type is determined by executing the worker thread, and the operation offset position and length are calculated; wherein, the operation type includes random read, random write, sequential read, and sequential write;
[0019] The interface of the storage engine is determined based on the operation type, and then the read and write operations of the storage engine are completed by calling the interface and based on the operation offset position and length.
[0020] Optionally, before determining the operation type by executing the worker thread, the method further includes:
[0021] Determine whether any of the worker threads satisfies the preset test termination condition;
[0022] If it exists, the execution process of the worker thread is terminated to complete the performance test; if it does not exist, the operation type is determined by executing the worker thread.
[0023] Optionally, the performance testing method for the distributed local storage engine further includes:
[0024] After completing the read and write operations of the storage engine, the performance information of the storage engine is statistically analyzed.
[0025] Secondly, this application discloses a performance testing device for a distributed storage engine, comprising:
[0026] The engine, object, and thread creation module is used to initialize global public information and create a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters.
[0027] The thread execution module is used to evenly distribute a number of test objects to a number of worker threads, start the worker threads, and then complete the read and write operations of the storage engine by executing the worker threads.
[0028] The performance information collection module is used to collect the performance information of the storage engine after each worker thread completes the read and write operations. When any worker thread meets the preset test termination condition, the execution process of the worker thread is terminated to complete the performance test.
[0029] Thirdly, this application discloses an electronic device, including:
[0030] Memory, used to store computer programs;
[0031] A processor is used to execute the computer program to implement the aforementioned performance testing method for the distributed storage engine.
[0032] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned performance testing method for the distributed storage engine.
[0033] As can be seen, this application proposes a performance testing method for a distributed storage engine, including: initializing global public information, and creating a storage engine, a number of worker threads, and a number of test objects on a local node based on preset configuration parameters; evenly distributing the number of test objects to the number of worker threads, starting the worker threads, and then executing the worker threads to complete read and write operations on the storage engine; collecting the performance information of the storage engine after each worker thread completes the read and write operations, and ending the execution process of the worker thread when any worker thread meets a preset test termination condition, thereby completing the performance test. It is evident that this application tests the performance of the storage engine independently in a non-cluster mode, abandoning the logical processing of the OSD layer, reducing the impact of upper-layer applications on the storage engine performance test, and fully verifying the storage engine's support for hard disks, thus greatly improving the accuracy and controllability of the storage engine performance test. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0035] Figure 1 A flowchart of an existing storage engine performance testing method;
[0036] Figure 2 This is a flowchart of a performance testing method for a distributed storage engine disclosed in this application;
[0037] Figure 3 This is a flowchart of the performance test of the improved storage engine disclosed in this application;
[0038] Figure 4 This application discloses a flowchart of a specific performance testing method for a distributed storage engine.
[0039] Figure 5 This application discloses a flowchart of a specific performance testing method for a distributed storage engine.
[0040] Figure 6 This is a flowchart illustrating a worker thread disclosed in this application;
[0041] Figure 7 This is a schematic diagram of the performance testing device for a distributed storage engine disclosed in this application;
[0042] Figure 8This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0044] Because of the different hard drives used, every storage engine has its limitations, failing to fully meet the needs of the hard drives or fully utilize their advantages. Therefore, optimization and modification of the storage engine are often necessary to improve its performance. However, most current testing solutions require building a cluster and using Rados tools to test the overall Ceph performance. Cluster testing introduces unnecessary interference, making it difficult to perform more detailed analysis and testing of the storage engines. While Ceph's native testing tools provide unit tests for individual storage engines like Bluestore, these tests primarily focus on functional testing, offering only simple, single-mode test scenarios. They are unsuitable for long-term, highly complex testing scenarios and cannot meet the need for comprehensive performance testing of the storage engines.
[0045] Therefore, this application proposes a performance testing scheme for a distributed storage engine, which can avoid introducing unnecessary interference and meet the need for overall performance testing of the storage engine.
[0046] This application discloses a performance testing method for a distributed storage engine. (See also...) Figure 2 As shown, the method includes:
[0047] Step S11: Initialize global public information and create a storage engine, a number of worker threads, and a number of test objects on the local node based on the preset configuration parameters.
[0048] In this embodiment, firstly, global public information and the distributed file system environment are initialized. After initializing the global public information, a storage engine, a number of worker threads, and a number of test objects are created on the local node based on the preset configuration parameters in the configuration file. Specifically, a storage engine is created based on the storage type configured in the configuration file. The storage engine includes, but is not limited to, bluestore and filestore. It should be noted that the bluestore and filestore storage engines are types of Objectstore. A number of test objects and worker threads are created based on the number of test objects and the number of threads specified in the configuration file.
[0049] Step S12: Distribute a number of test objects evenly to a number of worker threads, start the worker threads, and then complete the read and write operations of the storage engine by executing the worker threads.
[0050] In one specific implementation, after determining a certain number of test objects and worker threads, the certain number of test objects are evenly distributed to a certain number of worker threads, and the worker threads are started. Then, the read and write operations of the storage engine are completed by executing the worker threads.
[0051] In another specific implementation, this embodiment can also distribute a number of sets evenly to a number of worker threads; wherein each set includes a number of test objects. It should be noted that before distributing the number of sets evenly to the number of worker threads, it is necessary to create a storage engine, a number of worker threads, a number of test objects, and a number of sets on the local node based on preset configuration parameter information.
[0052] Step S13: Collect the performance information of the storage engine after each worker thread completes the read and write operations, and terminate the execution process of the worker thread when any worker thread meets the preset test termination condition, so as to complete the performance test.
[0053] In this embodiment, collecting the performance information of the storage engine after each worker thread completes a read / write operation specifically includes: collecting the performance information of the storage engine after each worker thread completes a read / write operation at preset time intervals, and ending the execution process of the worker thread when any worker thread meets a preset test termination condition to complete the performance test. For example, the preset test termination condition includes, but is not limited to, setting the test time to half an hour; further, ending the execution process of the worker thread after half an hour of testing to complete the performance test; the performance information includes, but is not limited to, performance data such as latency, bandwidth, and IOPS (Input / Output Operations Per Second, i.e., the number of read / write operations per second).
[0054] See Figure 3 As shown, Figure 3 This is a flowchart of the storage engine performance test disclosed in this application. The performance test tool of this application is used to perform performance testing on the Objectstore storage engine. For details, see [link to relevant documentation]. Figure 4 As shown, firstly, global public information is initialized and a storage engine is created. Then, various configuration parameters are loaded into the configuration file. After loading the configuration parameters, the storage engine is initialized, and test objects, sets, and worker threads are created according to the configuration parameters. Then, the test objects or sets are evenly distributed to the worker threads according to the number of test objects or sets, and the worker threads are executed. It is determined whether the preset stopping condition is met. If not, the performance information of each worker thread is collected at certain time intervals and the collected performance information is saved. If the condition is met, the test ends.
[0055] In summary, compared to the traditional method of building a cluster and testing the overall performance of Ceph through Rados, this application tests the performance of the storage engine separately in a non-cluster mode (local node), abandons the logical processing of the OSD layer, reduces the impact of upper-layer applications on the performance testing of the storage engine, fully verifies the storage engine's support for hard disks, and greatly improves the accuracy and controllability of the storage engine performance test.
[0056] As can be seen, this application proposes a performance testing method for a distributed storage engine, including: initializing global public information, and creating a storage engine, a number of worker threads, and a number of test objects on a local node based on preset configuration parameters; evenly distributing the number of test objects to the number of worker threads, starting the worker threads, and then executing the worker threads to complete read and write operations on the storage engine; collecting the performance information of the storage engine after each worker thread completes the read and write operations, and ending the execution process of the worker thread when any worker thread meets a preset test termination condition, thereby completing the performance test. It is evident that this application tests the performance of the storage engine independently in a non-cluster mode, abandoning the logical processing of the OSD layer, reducing the impact of upper-layer applications on the storage engine performance test, and fully verifying the storage engine's support for hard disks, thus greatly improving the accuracy and controllability of the storage engine performance test.
[0057] This application discloses a specific performance testing method for a distributed local storage engine. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. See also... Figure 5 as well as Figure 6 As shown, it specifically includes:
[0058] Step S21: Initialize global public information and create a storage engine, a number of worker threads, and a number of test objects on the local node based on the preset configuration parameters.
[0059] In this embodiment, firstly, global public information and the distributed file system environment are initialized. After initializing the global public information, a storage engine, a number of worker threads, and a number of test objects are created on the local node based on the preset configuration parameters in the configuration file. Specifically, this includes: creating a storage engine based on the configured storage type in the configuration file, wherein the storage engine includes, but is not limited to, bluestore, filestore, etc.; and creating a number of test objects and worker threads based on the number of test objects and the number of threads in the configuration file.
[0060] Step S22: Distribute a number of test objects evenly to a number of worker threads, start the worker threads, and then determine the operation type by executing the worker threads, and calculate the operation offset position and length; wherein, the operation type includes random read, random write, sequential read and sequential write.
[0061] See Figure 6As shown, this embodiment first needs to determine whether any of the worker threads meet the preset test termination condition; if so, the execution process of the worker thread is terminated to complete the performance test; if not, the operation type is determined by executing the worker thread, wherein the operation type includes random read, random write, sequential read and sequential write. After determining the corresponding operation type, the operation offset position and length are calculated.
[0062] It should be noted that the following processing methods exist for the above four operation types:
[0063] For sequential reads and sequential writes, there are two processing methods based on the length of the data being processed: writing objects in fixed-length or variable-length ways. Regardless of whether the writing is fixed-length or variable-length, the entire object space must eventually be filled before moving on to the next object.
[0064] For random reads and random writes, in addition to the two data lengths mentioned above, there are two randomization methods: true randomness and pseudo-randomness. True randomness uses the current time as the seed for the random number generator, and the objects processed and the intervals of operations on those objects in each worker thread are randomized. Pseudo-randomness uses a fixed value as the seed for the random number generator, and the objects processed and the intervals of operations on those objects in each worker thread are also randomized. The difference between pseudo-randomness and true randomness is that pseudo-randomness follows a fixed randomization method each time it starts. In certain scenarios, pseudo-randomness is very helpful for troubleshooting.
[0065] Step S23: Determine the interface of the storage engine based on the operation type, and then complete the read and write operation of the storage engine by calling the interface and based on the operation offset position and length.
[0066] In this embodiment, the working thread is executed, the interface is called, and the read and write operations of the storage engine are completed based on the operation offset position and length. After the read and write operations of the storage engine are completed, the performance information of the storage engine is statistically analyzed.
[0067] Step S24: Collect the performance information of the storage engine after each worker thread completes the read and write operations, and terminate the execution process of the worker thread when any worker thread meets the preset test termination condition, so as to complete the performance test.
[0068] In this embodiment, collecting the performance information of the storage engine after each worker thread completes the read and write operation specifically includes: collecting the performance information of the storage engine after each worker thread completes the read and write operation at a preset time interval, and ending the execution process of the worker thread when any worker thread meets the preset test termination condition, so as to complete the performance test.
[0069] In summary, the performance testing method described in this application allows for the independent testing of the storage engine's performance while shielding it from interference from upper-layer applications. It includes features such as multi-object multi-threading, random and sequential processing, and fixed-length and variable-length data processing. The method is powerful, offers a wide controllable range, and boasts high precision. It enables multi-faceted testing of the storage engine's performance and is also highly helpful in verifying the storage performance of flash arrays.
[0070] This application proposes a performance testing method for a distributed storage engine, comprising: initializing global public information and creating a storage engine, a number of worker threads, and a number of test objects on a local node based on preset configuration parameters; evenly distributing the number of test objects to the number of worker threads and starting the worker threads; determining the operation type by executing the worker threads and calculating the operation offset position and length; wherein the operation type includes random read, random write, sequential read, and sequential write; determining the interface of the storage engine based on the operation type, and then completing the read and write operations of the storage engine by calling the interface and based on the operation offset position and length; collecting the performance information of the storage engine after each worker thread completes the read and write operations, and ending the execution process of the worker thread when any worker thread meets the preset test termination condition, thereby completing the performance test. It can be seen that this application tests the performance of the storage engine independently in a non-cluster mode, abandoning the logical processing of the OSD layer, reducing the impact of upper-layer applications on the performance testing of the storage engine, and fully verifying the storage engine's support for hard disks, thus greatly improving the accuracy and controllability of the storage engine performance test.
[0071] Accordingly, this application also discloses a performance testing device for a distributed storage engine, see [link to relevant documentation]. Figure 7 As shown, the device includes:
[0072] The engine, object, and thread creation module 11 is used to initialize global public information and create a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters.
[0073] In this application, global public information is first initialized, the distributed file system environment is initialized, and then a storage engine, a number of worker threads, and a number of test objects are created on the local node based on the preset configuration parameters in the configuration file.
[0074] The thread execution module 12 is used to evenly distribute a number of test objects to a number of worker threads, start the worker threads, and then complete the read and write operations of the storage engine by executing the worker threads.
[0075] In this application, after determining a certain number of test objects and worker threads, the certain number of test objects are evenly distributed to a certain number of worker threads, and the worker threads are started. Then, the read and write operations of the storage engine are completed by executing the worker threads.
[0076] The performance information collection module 13 is used to collect the performance information of the storage engine after each worker thread completes the read and write operation. When any worker thread meets the preset test termination condition, the execution process of the worker thread is terminated to complete the performance test.
[0077] In this application, collecting the performance information of the storage engine after each worker thread completes the read and write operation specifically includes: collecting the performance information of the storage engine after each worker thread completes the read and write operation at a preset time interval, and ending the execution process of the worker thread when any worker thread meets the preset test termination condition, so as to complete the performance test.
[0078] As can be seen, this application proposes a performance testing method for a distributed storage engine, including: initializing global public information, and creating a storage engine, a number of worker threads, and a number of test objects on a local node based on preset configuration parameters; evenly distributing the number of test objects to the number of worker threads, starting the worker threads, and then executing the worker threads to complete read and write operations on the storage engine; collecting the performance information of the storage engine after each worker thread completes the read and write operations, and ending the execution process of the worker thread when any worker thread meets a preset test termination condition, thereby completing the performance test. It is evident that this application tests the performance of the storage engine independently in a non-cluster mode, abandoning the logical processing of the OSD layer, reducing the impact of upper-layer applications on the storage engine performance test, and fully verifying the storage engine's support for hard disks, thus greatly improving the accuracy and controllability of the storage engine performance test.
[0079] In some specific embodiments, the performance testing device for the distributed storage engine may further include:
[0080] The set distribution module is used to evenly distribute a number of sets to a number of worker threads; wherein each set includes a number of test objects.
[0081] In some specific embodiments, the engine, object, and thread creation module 11 may specifically include:
[0082] The engine, object, collection, and thread creation unit is used to create a storage engine, a number of worker threads, a number of test objects, and a number of collections on the local node based on preset configuration parameters.
[0083] In some specific embodiments, the performance information collection module 13 may specifically include:
[0084] The performance information collection unit is used to collect the performance information of the storage engine after each working thread completes a read / write operation, according to a preset time interval.
[0085] In some specific embodiments, the thread execution module 12 may specifically include:
[0086] An operation type determination unit is used to determine the operation type by executing the worker thread, and to calculate the operation offset position and length; wherein, the operation type includes random read, random write, sequential read, and sequential write;
[0087] The thread execution unit is used to determine the interface of the storage engine based on the operation type, and then complete the read and write operations of the storage engine by calling the interface and based on the operation offset position and length.
[0088] In some specific embodiments, before the thread execution module 12, a further step may include:
[0089] The test condition judgment unit is used to determine whether any of the worker threads meets the preset test termination condition; if it does, the execution process of the worker thread is terminated to complete the performance test; if it does not, the operation type is determined by executing the worker thread.
[0090] In some specific embodiments, the performance testing device for the distributed storage engine may further include:
[0091] The performance information statistics module is used to collect performance information of the storage engine after the read and write operations of the storage engine are completed.
[0092] Furthermore, embodiments of this application also provide an electronic device. Figure 8This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0093] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the following steps:
[0094] Initialize global public information, and create a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters;
[0095] The test objects are evenly distributed to the worker threads, and the worker threads are started. Then, the read and write operations of the storage engine are completed by executing the worker threads.
[0096] The performance information of the storage engine is collected after each worker thread completes a read / write operation. When any worker thread meets the preset test termination condition, the execution process of the worker thread is terminated to complete the performance test.
[0097] In some specific embodiments, the processor may further perform the following steps by executing a computer program stored in the memory:
[0098] A number of sets are evenly distributed to a number of worker threads; wherein each set includes a number of test objects.
[0099] In some specific embodiments, the processor executes a computer program stored in the memory, specifically implementing the following steps:
[0100] Based on preset configuration parameters, a storage engine, a number of worker threads, a number of test objects, and a number of collections are created on the local node.
[0101] In some specific embodiments, the processor executes a computer program stored in the memory, specifically implementing the following steps:
[0102] At preset time intervals, the performance information of the storage engine is collected by each worker thread after completing read and write operations.
[0103] In some specific embodiments, the processor executes a computer program stored in the memory, specifically implementing the following steps:
[0104] The operation type is determined by executing the worker thread, and the operation offset position and length are calculated; wherein, the operation type includes random read, random write, sequential read, and sequential write;
[0105] The interface of the storage engine is determined based on the operation type, and then the read and write operations of the storage engine are completed by calling the interface and based on the operation offset position and length.
[0106] In some specific embodiments, the processor may further perform the following steps by executing a computer program stored in the memory:
[0107] Determine whether any of the worker threads satisfies the preset test termination condition;
[0108] If it exists, the execution process of the worker thread is terminated to complete the performance test; if it does not exist, the operation type is determined by executing the worker thread.
[0109] In some specific embodiments, the processor may further perform the following steps by executing a computer program stored in the memory:
[0110] After completing the read and write operations of the storage engine, the performance information of the storage engine is statistically analyzed.
[0111] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 24 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0112] Furthermore, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon may include computer programs 221, and the storage method may be temporary storage or permanent storage. The computer programs 221 may include, in addition to computer programs capable of performing the performance testing method of the distributed local storage engine executed by the electronic device 20 as disclosed in any of the foregoing embodiments, computer programs capable of performing other specific tasks.
[0113] Furthermore, embodiments of this application also disclose a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned performance testing method for the distributed local storage engine.
[0114] For the specific steps of this method, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.
[0115] The various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts between the various embodiments, refer to each other. As for the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to in the method section.
[0116] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0117] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0118] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0119] The above provides a detailed description of the performance testing method, apparatus, device, and storage medium for a distributed local storage engine provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A performance testing method for a distributed storage engine, characterized in that, include: Initialize global public information, and create a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters; The test objects are evenly distributed to the worker threads, and the worker threads are started. Then, the read and write operations of the storage engine are completed by executing the worker threads. Collect the performance information of the storage engine after each worker thread completes the read and write operations, and terminate the execution process of the worker thread when any worker thread meets the preset test termination conditions, so as to complete the performance test. The process of executing the worker thread to complete the read and write operations of the storage engine includes: The operation type is determined by executing the worker thread, and the operation offset position and length are calculated; wherein, the operation type includes random read, random write, sequential read, and sequential write; The interface of the storage engine is determined based on the operation type, and then the read and write operations of the storage engine are completed by calling the interface and based on the operation offset position and length.
2. The performance testing method for the distributed storage engine according to claim 1, characterized in that, Also includes: A number of sets are evenly distributed to a number of worker threads; wherein each set includes a number of test objects.
3. The performance testing method for the distributed storage engine according to claim 2, characterized in that, The process of creating a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters includes: Based on preset configuration parameters, a storage engine, a number of worker threads, a number of test objects, and a number of collections are created on the local node.
4. The performance testing method for the distributed storage engine according to claim 1, characterized in that, The collection of performance information of the storage engine after each worker thread completes a read / write operation includes: At preset time intervals, the performance information of the storage engine is collected by each worker thread after completing read and write operations.
5. The performance testing method for the distributed storage engine according to claim 1, characterized in that, Before determining the operation type by executing the worker thread, the method further includes: Determine whether any of the worker threads satisfies the preset test termination condition; If it exists, the execution process of the worker thread is terminated to complete the performance test; if it does not exist, the operation type is determined by executing the worker thread.
6. The performance testing method for the distributed storage engine according to claim 1, characterized in that, Also includes: After completing the read and write operations of the storage engine, the performance information of the storage engine is statistically analyzed.
7. A performance testing device for a distributed storage engine, characterized in that, include: The engine, object, and thread creation module is used to initialize global public information and create a storage engine, a number of worker threads, and a number of test objects on the local node based on preset configuration parameters. The thread execution module is used to evenly distribute a number of test objects to a number of worker threads, start the worker threads, and then complete the read and write operations of the storage engine by executing the worker threads. The performance information collection module is used to collect the performance information of the storage engine after each worker thread completes the read and write operation. When any worker thread meets the preset test termination condition, the execution process of the worker thread is terminated to complete the performance test. The process of executing the worker thread to complete the read and write operations of the storage engine includes: The operation type is determined by executing the worker thread, and the operation offset position and length are calculated; wherein, the operation type includes random read, random write, sequential read, and sequential write; The interface of the storage engine is determined based on the operation type, and then the read and write operations of the storage engine are completed by calling the interface and based on the operation offset position and length.
8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the performance testing method for the distributed storage engine as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the performance testing method of the distributed storage engine as described in any one of claims 1 to 6.