A method, tool and apparatus for testing performance of a shared file system

Abstract

A shared file system performance test method, tool, device and storage medium, comprising: obtaining the startup parameters of the test node in the shared file system to start the test program to perform write operation and shift operation on the target file system; when the write file flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory composed of year, serial day and hour; when the shift flag is 1, the data file is shifted from the write directory to the shift target directory by directly calculating the file path and file name in the memory; the success and failure times and time consumption of writing and shifting in the write operation and the shift operation are recorded, the test result is generated and output, the technical problem that the related technology generally starts from IOPS and cannot truly reflect the business application performance is solved, the performance test data truly reflecting the business scenario is generated, and the IOPS utilization rate is improved.

Description

Technical Field

[0001] This application relates to the field of testing technology, specifically to a shared file system performance testing method, tool, device, and computer-readable storage medium. Background Technology

[0002] Network Attached Storage (NAS), as a core carrier for enterprise-level data sharing, has been widely adopted in key scenarios such as office collaboration, video surveillance, virtualization clusters, and scientific research data management. As business scale expands, NAS performance requirements have shifted from "high throughput for a single client" to "concurrent response from multiple terminals"—for example, a design firm with 20+ designers simultaneously reading 10GB of CAD drawings, a cloud service provider supporting simultaneous data backup for 500+ virtual machines, and a monitoring center storing real-time streaming media from 32 4K cameras. These scenarios all require the file system to simultaneously "efficiently process data transmission" while also ensuring "low-latency response to business commands" and "stable handling of concurrent loads." Existing vendors typically provide file system performance metrics for NAS and other devices based on IOPS. In actual business applications, it's common to see shared file system parameters such as network and IOPS appearing normal, but the data seen by the business application is already piling up. Summary of the Invention

[0003] This application provides a shared file system performance testing method, tool, device, and computer-readable storage medium, which can solve the technical problem that existing technologies generally measure IOPS and cannot truly reflect the performance of business applications.

[0004] In a first aspect, embodiments of this application provide a shared file system performance testing method, characterized in that the shared file system performance testing method includes:

[0005] Obtain the startup parameters of the test node in the shared file system to start the test program to perform write and shift operations on the target file system. The startup parameters include the write directory, the shift target directory, the total number of files, the size of a single file, the number of files in a single folder, the initial file name, the write file flag, and the shift flag.

[0006] When the write file flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory consisting of year, date, and hour, wherein the subdirectory is constructed in the write directory;

[0007] When the shift flag is 1, the data file is shifted from the write directory to the shift target directory by directly calculating and generating the file path and file name in memory.

[0008] Record the number of successful and failed write and shift operations and the time consumed during the write and shift operations, generate test results and output them, wherein the test results are used to evaluate the performance of the shared file system at the application layer.

[0009] In conjunction with the first aspect, in one implementation, obtaining the startup parameters of the test node in the shared file system to start the test program to perform write and shift operations on the target file system includes:

[0010] Obtain the configuration file in the shared file system, wherein the configuration file includes startup parameters and test commands for one or more nodes, the startup parameters include configuration files for write directory, shift target directory, total number of files, size of a single file, number of files in a single folder, initial file name, write file flag, and shift flag, and the test commands include single-process test name and multi-process test command;

[0011] Log in remotely to each of the test nodes via SSH passwordless access, and obtain the node name and test command of each test node from the configuration file;

[0012] The configuration file is parsed to obtain the test command corresponding to the node name in the configuration file and the startup parameters, so as to start the corresponding test program to perform write and shift operations on the target file system;

[0013] The step of remotely logging into each of the test nodes via SSH without a password, and before obtaining the node name of each of the test nodes, further includes:

[0014] Obtain the shared file system information mounted on each of the test nodes;

[0015] The validity of the test environment is determined based on the shared file system information mounted on each of the test nodes.

[0016] In conjunction with the first aspect, when the file write flag is 1, generating a data file containing pseudo-random number initialization and writing it to a subdirectory consisting of year, date, and hour includes:

[0017] When the write file flag is set to 1, a memory block of the same size as a single file is generated;

[0018] Based on the obtained loop count, modify specific bytes of the memory block to generate a target filename with the initial filename as the base and a fixed 8-digit suffix. The specific bytes include the 0th byte, the 2nd byte, the 20th byte, the current loop count modulo the file size byte, and the last byte.

[0019] The contents of the memory block are initialized with pseudo-random numbers to generate a data file containing non-all zeros.

[0020] Use the target file name as the file name of the data file;

[0021] The data file is written to a subdirectory consisting of year, date, and hour, wherein the subdirectory is constructed in the writing directory.

[0022] In conjunction with the first aspect, writing the data file into a subdirectory consisting of year, date, and hour includes:

[0023] Get the current year as the year prefix for subdirectories;

[0024] The sequence date is calculated according to the first preset calculation rule, wherein the first preset calculation rule is to divide the current number of cycles by the number of files in a single folder, divide the remainder by 24 and add 1, and the result is fixed to 3 digits;

[0025] The hour is calculated according to the second preset calculation rule, wherein the second preset calculation rule is to divide the current loop count by the number of files in a single folder minus 1, divide the remainder by 24 and add 1, and the result is fixed to 2 digits;

[0026] Construct a subdirectory structure for the year, the date, and the hour, and write the data file into the subdirectory;

[0027] After constructing the subdirectory structure for the year, the date, and the hour, the method further includes:

[0028] Obtain the number of subdirectory structures and the number of write operations;

[0029] Based on the number of subdirectory structures and the number of write operations, the correctness of the subdirectory structure is verified to determine the storage path of the data file and to construct the subdirectory.

[0030] In conjunction with the first aspect, the method of shifting the data file from the writing directory to the shift target directory by directly calculating and generating the file path and filename in memory when the shift flag is 1 includes:

[0031] Obtain the first and second preset calculation rules, and generate a target subdirectory in the shift target directory;

[0032] Based on the initial filename and the obtained loop count, generate the target filename and target file path;

[0033] The rename function is used to move the data file from the write directory to a target subdirectory in the target directory based on the target file name and the target file path. The target directory records the number of successful and failed moves of the move operation, as well as the move time calculated based on the start and end times of the move operation.

[0034] The shift operation does not check if the source file exists. Shift operations that fail to find the source file will result in a shift failure, which will be recorded in the number of shift failures.

[0035] In conjunction with the first aspect, the recording of the number of successful and failed write and shift operations and the time consumed during the write and shift operations, generating test results and outputting them, includes:

[0036] Record the number of successful and failed write operations;

[0037] Record the number of successful and failed shift operations;

[0038] Obtain the start and end times of the write operation and calculate the write time.

[0039] Obtain the start and end times of the shift operation and calculate the shift time.

[0040] Verify the completeness of the shift operation's time consumption, success count, and number of failures, generate test results containing performance test data, and output them.

[0041] In conjunction with the first aspect, the output test results include: test configuration information, test data, test time information, the status of the write file flag, and the status of the shift flag;

[0042] The test configuration information includes the running node, the write directory, the total number of files, the size of a single file, and the number of files in a single folder.

[0043] The test data includes the sum of write time, shift time, number of write errors, and number of shift errors;

[0044] The test timing information includes the start time of the write operation, the completion time of the write operation, the start time of the shift operation, and the completion time of the shift operation.

[0045] Secondly, embodiments of this application provide a shared file system performance testing tool, which includes:

[0046] The acquisition and startup module is used to acquire the startup parameters of the test node in the shared file system, so as to start the test program to perform write and shift operations on the target file system. The startup parameters include the write directory, the shift target directory, the total number of files, the size of a single file, the number of files in a single folder, the initial file name, the write file flag, and the shift flag.

[0047] The generation and writing module is used to generate a data file containing pseudo-random number initialization when the write file flag is 1, and write it to a subdirectory consisting of year, date, and hour.

[0048] The shift module is used to shift the data file from the write directory to the shift target directory in memory when the shift flag is 1;

[0049] The generation and output module is used to record the number of successful and failed writes and shifts and the time consumed during the write and shift operations, generate test results and output them, wherein the test results are used to evaluate the performance of the shared file system at the application layer.

[0050] Thirdly, this application provides a shared file system performance testing device, which includes a processor, a memory, and a shared file system performance testing program stored in the memory and executable by the processor. When the shared file system performance testing program is executed by the processor, it implements the steps of the shared file system performance testing method described above.

[0051] Fourthly, embodiments of this application provide a computer-readable storage medium storing a shared file system performance testing program, wherein when the shared file system performance test is executed by a processor, the steps of the shared file system performance testing method described above are implemented.

[0052] The beneficial effects of the technical solutions provided in this application include:

[0053] By obtaining the startup parameters of the test node in the shared file system, the test program is launched to perform write and shift operations on the target file system. The startup parameters include the write directory, the shift target directory, the total number of files, the size of a single file, the number of files in a single folder, the initial filename, the write file flag, and the shift flag. When the write file flag is 1, a data file containing pseudo-random numbers is generated and written to a subdirectory composed of year, day, and hour, where the subdirectory is constructed within the write directory. When the shift flag is 1, the data file is shifted from the write directory to the shift target directory in memory. The number of successful and failed writes and shifts, as well as the time consumed, are recorded during the write and shift operations. Test results are generated and output. These test results are used to evaluate the performance of the shared file system at the application layer. This solves the technical problem that related technologies generally focus on IOPS, which cannot truly reflect the performance of business applications. It enables write and shift operations of different file sizes from the application layer perspective, directly measuring the performance perceived by the application and generating test data that truly reflects the business scenario. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a first embodiment of a shared file system performance testing method according to this application;

[0055] Figure 2 This is a functional module diagram of an embodiment of a shared file system performance testing tool according to this application;

[0056] Figure 3 This is a schematic diagram of the hardware structure of a shared file system performance testing device involved in the embodiments of this application. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0058] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0060] Firstly, embodiments of this application provide a method for testing the performance of a shared file system.

[0061] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of a shared file system performance testing method according to this application. Figure 1 As shown, a shared file system performance testing method includes:

[0062] Step S10: Obtain the startup parameters of the test node in the shared file system to start the test program to perform write and shift operations on the target file system. The startup parameters include the write directory, the shift target directory, the total number of files, the size of a single file, the number of files in a single folder, the initial file name, the write file flag, and the shift flag.

[0063] Step S20: When the write file flag is 1, generate a data file containing pseudo-random number initialization and write it to a subdirectory consisting of year, date and hour, wherein the subdirectory is constructed in the write directory;

[0064] Step S30: When the shift flag is 1, the data file is shifted from the write directory to the shift target directory by directly calculating and generating the file path and file name in memory;

[0065] Step S40: Record the number of successful and failed write and shift operations and the time consumed during the write and shift operations, generate test results and output them, wherein the test results are used to evaluate the performance of the shared file system at the application layer.

[0066] As an example, the process involves obtaining configuration files from a shared file system. These configuration files include startup parameters and test commands for one or more nodes. Startup parameters include the write directory, target directory for migration, total number of files, size of a single file, number of files in a single folder, initial filename, write flag, and migration flag. Test commands include single-process test names and multi-process test commands. The process involves remotely logging into each test node via SSH without a password and obtaining the node name for each node. The configuration files are then parsed to obtain the test commands and startup parameters corresponding to the node names, enabling the corresponding test programs to perform write and migration operations on the target file system. Before obtaining the node names from the configuration files after remotely logging into each test node via SSH without a password, the process also includes: obtaining information about the shared file systems mounted on each test node; and determining the validity of the test environment based on this information.

[0067] When the write file flag is 1, a memory block of the same size as a single file is generated; based on the obtained loop count, specific bytes of the memory block are modified to generate a target filename with the initial filename as the base and a fixed 8-digit suffix, wherein the specific bytes include the 0th byte, the 2nd byte, the 20th byte, the remainder after dividing the current loop count by the file size in bytes, and the last byte; the contents of the memory block are initialized with pseudo-random numbers to generate a data file with non-all zero data; the target filename is used as the filename of the data file; the data file is written to a subdirectory consisting of the year, date, and hour, wherein the subdirectory is constructed in the write directory.

[0068] The process involves: obtaining the current year as the year prefix for the subdirectories; calculating the day according to a first preset calculation rule, where the remainder after dividing the current loop count by the number of files in a single folder is divided by 24 and then added to the result, resulting in a fixed 3-digit number; calculating the hour according to a second preset calculation rule, where the remainder after dividing the current loop count by the number of files in a single folder minus 1 is divided by 24 and then added to the result, resulting in a fixed 2-digit number; constructing a subdirectory structure for year, day, and hour, and writing the data files into the subdirectories; after constructing the subdirectory structure for year, day, and hour, the process further includes: obtaining the number of subdirectory structures and the number of write operations; verifying the correctness of the subdirectory structure based on the number of subdirectory structures and the number of write operations to determine the storage path of the data files and construct the subdirectories.

[0069] Obtain the first and second preset calculation rules, and generate a target subdirectory in the target directory. Based on the initial filename and the obtained loop count, generate the target filename and target file path. Use the rename function to move the data file from the directory written in memory to the target subdirectory in the target directory based on the target filename and target file path. The source filename and target filename of the shift are directly calculated and generated in memory, thereby avoiding scanning the source directory and improving the accuracy of the test. The target directory records the number of successful and failed shift operations, as well as the shift time calculated based on the start and end times of the shift operations.

[0070] Record the number of successful and failed write operations; record the number of successful and failed shift operations; obtain the start and end times of write operations and calculate the write time; obtain the start and end times of shift operations and calculate the shift time; verify the completeness of the shift operation time, success and failure counts, generate test results containing performance test data and output them.

[0071] For example, on a test node with a shared file system mounted, the test program is started. The program reads the startup parameters for subsequent operations. These parameters include the write directory, the target directory to be shifted, the total number of files, the size of a single file (in bytes), the number of files in a single folder, the initial filename, the write file flag, and the shift flag. When the write file flag is 1, the program allocates a memory block of size equal to the input parameter "single file size" and initializes the data with pseudo-random numbers to ensure that the memory block is not entirely empty (0), thus avoiding the special handling some file systems use for all-zero data. The tool uses the "total number of files" parameter as the total number and loops from 0 to the total number of files minus 1. Each loop generates a data file with the initial filename as its base, appended with the current loop count plus 1, and writes it to a subdirectory of the "write directory" specified by the parameter. The length of the appended filename suffix is ​​fixed at 8 digits; when the loop count is less than 10,000,000, leading zeros are added. For example, if the initial filename is test.txt, and the loop count is 99, the generated filename will be test.txt.00000100.

[0072] The subdirectory generation strategy for each loop's "Write Directory" is to simulate 24 hours over 365 days. Specifically, the day and hour are generated sequentially. The day generation method is: current loop count % number of files in a single folder / 24 + 1, fixed at 3 digits, padded with 0s if necessary; the hour generation method is: (current loop count % number of files in a single folder - 1) % 24, fixed at two digits, padded with 0s if necessary. Then, the current year is obtained, and the final generated subfolder is "Year / Day / Hour". Each time the data is written, based on the initial memory, a certain modification strategy is used to make it slightly different according to the loop count to avoid writing exactly the same data. The strategy is as follows, where data[n] represents the nth byte of the initial memory block:

[0073] data[0] = current loop count;

[0074] data[2] = current loop count - 1;

[0075] data

[20] = current loop count + 1;

[0076] data[current loop count % file size] = current loop count - 2;

[0077] data[file size - 1] = current loop count - 3;

[0078] When the write file flag is 1, the program does not poll the write directory. During the shift process, the program first uses the same date and hour generation algorithm as the file generation algorithm to generate the target directory subdirectory. Then, it establishes a "total number of files" parameter as the total number and creates a loop from 0 to the total number of files minus 1. Each time, it uses the same algorithm as the write operation, generating a filename identical to the write directory based on the loop count and the initial filename parameter. Then, it uses the `rename()` function to shift the filename from the write directory to the target directory. By not polling the write directory and directly using the same algorithm to generate the filename, the program eliminates the need for system polling and makes the shift performance calculation more realistic and accurate. The program records the number of successful and failed write operations, as well as the start and end times of the write and shift loops. After completion, the program outputs results used to calculate the performance of this test. The tool supports multi-node, multi-process testing. The program reads the configuration file, which has the following format:

[0079] Node Name 1 Single Node Single Process Test Command 1

[0080] Node Name 1 Single Node Single Process Test Command 2

[0081] Node Name 2 Single Node Single Process Test Command 2

[0082] Node Name 3 Single Node Single Process Test Command 3

[0083] Multi-process testing requires that all nodes in the configuration file mount the same file system, and that passwordless SSH access is enabled between the nodes. The program uses SSH to remotely log in to the specified node and execute the corresponding test commands, thereby achieving multi-node, multi-concurrency testing.

[0084] By dynamically generating subdirectory structures based on year, day, and hour, and combining pseudo-random number initialization and specific byte modification strategies, the special handling of all-zero data by the file system is avoided. Furthermore, the same algorithm used for writing is employed to directly generate filenames for shifting operations, eliminating the impact of polling time and improving the accuracy of shift performance testing.

[0085] In this embodiment, the test program is started to perform write and shift operations on the target file system by obtaining the startup parameters of the test node in the shared file system. The startup parameters include the write directory, the target shift directory, the total number of files, the size of a single file, the number of files in a single folder, the initial filename, the write file flag, and the shift flag. When the write file flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory composed of year, day, and hour, wherein the subdirectory is constructed in the write directory. When the shift flag is 1, the data file is moved from the target file system to the target file system by directly calculating and generating the file path and filename in memory. The process involves writing to a target directory and shifting to another directory; recording the number of successful and failed write and shift operations and their time consumption during the write and shift operations; generating and outputting test results; wherein the test results are used to evaluate the performance of the shared file system at the application layer, solving the technical problem that related technologies generally start from IOPS and cannot truly reflect the performance of business applications. This method realizes the write and shift speed of different file sizes from the application layer perspective, generates performance test data that truly reflects business scenarios, improves IOPS utilization, and is conducive to evaluating the overall performance of the shared file system under different business scenarios, providing a basis for optimizing the file system from a business perspective.

[0086] Secondly, embodiments of this application also provide a shared file system performance testing tool.

[0087] In one embodiment, reference is made to Figure 2 , Figure 2 This is a functional module diagram of an embodiment of a shared file system performance testing tool according to this application. Figure 2 As shown, a shared file system performance testing tool includes:

[0088] The acquisition and startup module 10 is used to acquire the startup parameters of the test node in the shared file system, so as to start the test program to perform write and shift operations on the target file system. The startup parameters include the write directory, the shift target directory, the total number of files, the size of a single file, the number of files in a single folder, the initial file name, the write file flag, and the shift flag.

[0089] The generation and writing module 20 is used to generate a data file containing pseudo-random number initialization when the file writing flag is 1, and write it to a subdirectory consisting of year, date and hour;

[0090] The shift module 30 is used to shift the data file from the write directory to the shift target directory by directly calculating and generating the file path and file name in memory when the shift flag is 1.

[0091] The generation and output module 40 is used to record the number of successful and failed writes and shifts and the time consumed during the write and shift operations, generate test results and output them, wherein the test results are used to evaluate the performance of the shared file system at the application layer.

[0092] Furthermore, in one embodiment, the acquisition and activation module 10 is used for:

[0093] Obtain the configuration file in the shared file system, wherein the configuration file includes startup parameters and test commands. The startup parameters include configuration files for the write directory, the target directory to be moved, the total number of files, the size of a single file, the number of files in a single folder, the initial file name, the write file flag, and the move flag. The test commands include a single-process test name and a multi-process test command.

[0094] Log in remotely to each of the test nodes via SSH passwordless access, and obtain the node name and test command of each test node from the configuration file;

[0095] The configuration file is parsed to obtain the test command corresponding to the node name in the configuration file and the startup parameters, so as to start the corresponding test program to perform write and shift operations on the target file system;

[0096] Before remotely logging into each of the test nodes via passwordless SSH access and obtaining the node name and test command of each test node from the configuration file, the process further includes:

[0097] Obtain the shared file system information mounted on each of the test nodes;

[0098] The validity of the test environment is determined based on the shared file system information mounted on each of the test nodes.

[0099] Furthermore, in one embodiment, the generation and writing module 20 is used for:

[0100] When the write file flag is set to 1, a memory block of the same size as a single file is generated;

[0101] Based on the obtained loop count, modify specific bytes of the memory block to generate a target filename with the initial filename as the base and a fixed 8-digit suffix. The specific bytes include the 0th byte, the 2nd byte, the 20th byte, the remainder after dividing the current loop count by the file size in bytes, and the last byte.

[0102] The contents of the memory block are initialized with pseudo-random numbers to generate a data file containing non-all zeros.

[0103] Use the target file name as the file name of the data file;

[0104] The data file is written to a subdirectory consisting of year, date, and hour, wherein the subdirectory is constructed in the writing directory.

[0105] Furthermore, in one embodiment, a shared file system performance testing tool also includes a new module for:

[0106] Get the current year as the year prefix for subdirectories;

[0107] The sequence date is calculated according to the first preset calculation rule, wherein the first preset calculation rule is to divide the current number of cycles by the number of files in a single folder, divide the remainder by 24 and add 1, and the result is fixed to 3 digits;

[0108] The hour is calculated according to the second preset calculation rule, wherein the second preset calculation rule is to divide the current loop count by the number of files in a single folder minus 1, divide the remainder by 24 and add 1, and the result is fixed to 2 digits;

[0109] Construct a subdirectory structure for the year, the date, and the hour, and write the data file into the subdirectory;

[0110] After constructing the subdirectory structure for the year, the date, and the hour, the method further includes:

[0111] Obtain the number of subdirectory structures and the number of write operations;

[0112] Based on the number of subdirectory structures and the number of write operations, the correctness of the subdirectory structure is verified to determine the storage path of the data file and to construct the subdirectory.

[0113] Furthermore, in one embodiment, the shift module 30 is used for:

[0114] Obtain the first and second preset calculation rules, and generate a target subdirectory in the shift target directory;

[0115] Based on the initial filename and the obtained loop count, generate the target filename and target file path;

[0116] The rename function is used to move the data file from the write directory to a target subdirectory in the target directory based on the target file name and the target file path. The target directory records the number of successful and failed moves of the move operation, as well as the move time calculated based on the start and end times of the move operation.

[0117] The shift operation does not check if the source file exists. Shift operations that fail to find the source file will result in a shift failure, which will be recorded in the number of shift failures.

[0118] Furthermore, in one embodiment, the generation and output module 40 is used for:

[0119] Record the number of successful and failed write operations;

[0120] Record the number of successful and failed shift operations;

[0121] Obtain the start and end times of the write operation and calculate the write time.

[0122] Obtain the start and end times of the shift operation and calculate the shift time.

[0123] Verify the completeness of the shift operation's time consumption, success count, and number of failures, generate test results containing performance test data, and output them.

[0124] The functions of each module in the above-mentioned shared file system performance testing tool correspond to the steps in the above-mentioned shared file system performance testing tool embodiment, and their functions and implementation processes will not be described in detail here.

[0125] Thirdly, embodiments of this application provide a shared file system performance testing device. A shared file system performance testing device can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0126] Reference Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of a shared file system performance testing device according to an embodiment of this application. In this embodiment, the shared file system performance testing device may include a processor, a memory, a communication interface, and a communication bus.

[0127] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0128] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within a shared file system performance testing device, as well as interfaces used for interconnecting the shared file system performance testing device with other devices (such as other computing devices or user devices). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user devices can be displays, keyboards, etc.

[0129] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0130] The processor can be a general-purpose processor, which can call a shared file system performance testing program stored in memory and execute a shared file system performance testing method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the shared file system performance testing program is called can refer to the various embodiments of the shared file system performance testing method of this application, and will not be repeated here.

[0131] Those skilled in the art will understand that Figure 3 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0132] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0133] This application provides a shared file system performance testing program stored on a computer-readable storage medium, wherein when the shared file system performance testing program is executed by a processor, it implements the steps of the shared file system performance testing method described above.

[0134] The method implemented when a shared file system performance testing program is executed can be referred to in various embodiments of a shared file system performance testing method of this application, and will not be repeated here.

[0135] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0136] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0137] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0138] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0139] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0141] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for testing the performance of a shared file system, characterized in that, The shared file system performance testing method includes: Obtain the startup parameters of the test node in the shared file system to start the test program to perform write and shift operations on the target file system. The startup parameters include the write directory, the shift target directory, the total number of files, the size of a single file, the number of files in a single folder, the initial file name, the write file flag, and the shift flag. When the write file flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory consisting of year, date, and hour, wherein the subdirectory is constructed in the write directory; When the file write flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory consisting of year, date, and hour, including: When the write file flag is set to 1, a memory block of the same size as a single file is generated; Based on the obtained loop count, modify specific bytes of the memory block to generate a target filename with the initial filename as the base and a fixed 8-digit suffix. The specific bytes include the 0th byte, the 2nd byte, the 20th byte, the remainder after dividing the current loop count by the file size in bytes, and the last byte. The contents of the memory block are initialized with pseudo-random numbers to generate a data file containing non-all zeros. Use the target file name as the file name of the data file; The data file is written to a subdirectory consisting of year, date, and hour, wherein the subdirectory is constructed within the writing directory; The step of writing the data file into a subdirectory consisting of year, date, and hour includes: Get the current year as the year prefix for subdirectories; The sequence date is calculated according to the first preset calculation rule, wherein the first preset calculation rule is to divide the current loop count by the number of files in a single folder, divide the remainder by 24 and add 1, and the result is fixed to 3 digits; The hour is calculated according to the second preset calculation rule, wherein the second preset calculation rule is to divide the current loop count by the number of files in a single folder minus 1, divide the remainder by 24 and add 1, and the result is fixed to 2 digits; Construct a subdirectory structure for the year, the date, and the hour, and write the data file into the subdirectory; After constructing the subdirectory structure for the year, the date, and the hour, the method further includes: Obtain the number of subdirectory structures and the number of write operations; Based on the number of subdirectory structures and the number of write operations, the correctness of the subdirectory structure is verified to determine the storage path of the data file and to form a subdirectory. When the shift flag is 1, the data file is shifted from the write directory to the shift target directory by directly calculating and generating the file path and file name in memory. The shift target directory records the number of successful and failed shift operations, as well as the shift time calculated based on the start and end times of the shift operations. The shift operation does not check whether the source file exists. Shift operations that fail for non-existent source files will result in shift failure, which will be recorded in the number of shift failures. Record the number of successful and failed write and shift operations and the time consumed during the write and shift operations, generate test results and output them, wherein the test results are used to evaluate the performance of the shared file system at the application layer.

2. The shared file system performance testing method as described in claim 1, characterized in that, The step of obtaining the startup parameters of the test node in the shared file system to start the test program to perform write and shift operations on the target file system includes: Obtain the configuration file in the shared file system, wherein the configuration file includes startup parameters and test commands for one or more nodes, the startup parameters include configuration files for write directory, shift target directory, total number of files, size of a single file, number of files in a single folder, initial file name, write file flag, and shift flag, and the test commands include single-process test name and multi-process test command; Log in remotely to each of the test nodes via SSH passwordless access, and obtain the node name and test command of each test node from the configuration file; The configuration file is parsed to obtain the test command corresponding to the node name in the configuration file and the startup parameters, so as to start the corresponding test program to perform write and shift operations on the target file system; The procedure, which involves remotely logging into each of the test nodes via SSH without a password and before retrieving the node name and test command of each test node from the configuration file, further includes: Obtain the shared file system information mounted on each of the test nodes; The validity of the test environment is determined based on the shared file system information mounted on each of the test nodes.

3. The shared file system performance testing method as described in claim 1, characterized in that, The step of shifting the data file from the writing directory to the shift target directory by directly calculating and generating the file path and file name in memory when the shift flag is 1 includes: Obtain the first and second preset calculation rules, and generate a target subdirectory in the shift target directory; Based on the initial filename and the obtained loop count, generate the target filename and target file path; The `rename` function is used to move a data file from the write directory to a target subdirectory within the target directory, based on the target file name and the target file path.

4. The shared file system performance testing method as described in claim 1, characterized in that, The system records the number of successful and failed write and shift operations and the time consumed during the write and shift operations, generates test results, and outputs them, including: Record the number of successful and failed write operations; Record the number of successful and failed shift operations; Obtain the start and end times of the write operation and calculate the write time. Obtain the start and end times of the shift operation and calculate the shift time. Verify the completeness of the shift operation's time consumption, success count, and number of failures, generate test results containing performance test data, and output them.

5. The shared file system performance testing method as described in claim 1, characterized in that, The test results include: test configuration information, test data, test time information, the status of the write file flag, and the status of the shift flag; The test configuration information includes the running node, the write directory, the total number of files, the size of a single file, and the number of files in a single folder. The test data includes the sum of write time, shift time, number of write errors, and number of shift errors; The test timing information includes the start time of the write operation, the completion time of the write operation, the start time of the shift operation, and the completion time of the shift operation.

6. A shared file system performance testing tool, characterized in that, The shared file system performance testing tool includes: The acquisition and startup module is used to acquire the startup parameters of the test node in the shared file system, so as to start the test program to perform write and shift operations on the target file system. The startup parameters include the write directory, the shift target directory, the total number of files, the size of a single file, the number of files in a single folder, the initial file name, the write file flag, and the shift flag. The generation and writing module is used to generate a data file containing pseudo-random number initialization when the write file flag is 1, and write it to a subdirectory consisting of year, date, and hour. When the file write flag is 1, a data file containing pseudo-random number initialization is generated and written to a subdirectory consisting of year, date, and hour, including: When the write file flag is set to 1, a memory block of the same size as a single file is generated; Based on the obtained loop count, modify specific bytes of the memory block to generate a target filename with the initial filename as the base and a fixed 8-digit suffix. The specific bytes include the 0th byte, the 2nd byte, the 20th byte, the remainder after dividing the current loop count by the file size in bytes, and the last byte. The contents of the memory block are initialized with pseudo-random numbers to generate a data file containing non-all zeros. Use the target file name as the file name of the data file; The data file is written to a subdirectory consisting of year, date, and hour, wherein the subdirectory is constructed within the writing directory; The step of writing the data file into a subdirectory consisting of year, date, and hour includes: Get the current year as the year prefix for subdirectories; The sequence date is calculated according to the first preset calculation rule, wherein the first preset calculation rule is to divide the current loop count by the number of files in a single folder, divide the remainder by 24 and add 1, and the result is fixed to 3 digits; The hour is calculated according to the second preset calculation rule, wherein the second preset calculation rule is to divide the current loop count by the number of files in a single folder minus 1, divide the remainder by 24 and add 1, and the result is fixed to 2 digits; Construct a subdirectory structure for the year, the date, and the hour, and write the data file into the subdirectory; After constructing the subdirectory structure for the year, the date, and the hour, the method further includes: Obtain the number of subdirectory structures and the number of write operations; Based on the number of subdirectory structures and the number of write operations, the correctness of the subdirectory structure is verified to determine the storage path of the data file and to form a subdirectory. The shift module is used to shift the data file from the write directory to the shift target directory in memory when the shift flag is 1. The shift target directory records the number of successful and failed shift operations, as well as the shift time calculated based on the start and end times of the shift operations. The shift operation does not check whether the source file exists. A shift operation that fails to detect a non-existent source file will result in a shift failure, which will be recorded in the number of shift failures. The generation and output module is used to record the number of successful and failed writes and shifts and the time consumed during the write and shift operations, generate test results and output them, wherein the test results are used to evaluate the performance of the shared file system at the application layer.

7. A shared file system performance testing device, characterized in that, The shared file system performance testing device includes a processor, a memory, and a shared file system performance testing program stored in the memory and executable by the processor. When the shared file system performance testing program is executed by the processor, it implements the steps of a shared file system performance testing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a shared file system performance testing program, wherein when the shared file system performance testing program is executed by a processor, it implements the steps of a shared file system performance testing method as described in any one of claims 1 to 5.

Images