Load testing method and system of storage device and electronic device

Abstract

The invention provides a load test method and system for a storage device and an electronic device, and relates to the technical field of storage, user behavior data is obtained by collecting an access log of a user on the storage device and preprocessing the access log, the real use behavior of the user can be obtained, and the defect that a general test case is separated from actual operation is avoided; features are extracted based on the user behavior data, and a user behavior portrait library is constructed, so that load tidal fluctuation features and abnormal behavior modes accessed by the user can be accurately captured, and the accuracy of a test case is ensured; the exclusive test case containing the tidal load case and the abnormal behavior injection case is generated according to the portrait library, the load fluctuation in the actual use of the user can be simulated, the abnormal behavior injection case is matched with the abnormal behavior characteristics in the portrait library, the emergency scene in the operation of the user can be reproduced, and the user experience is improved. The potential defects of the storage device under tidal load fluctuation and accidental abnormal behaviors are overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a load testing method and system of a storage device, and an electronic device. BACKGROUND

[0002] The existing load testing method of a storage device performs performance evaluation on the storage device by using a preset standardized test case. The preset standardized test case usually adopts average and standardized scene presetting. This way uses average and standardized scenes, which are different from actual user access behaviors, and thus may cause inaccurate test results, significant deviation from real business performance, or failure to effectively identify potential defects of the storage device under load fluctuation and sudden abnormality. In addition, the absence of occasional abnormal behavior simulation further weakens the comprehensiveness and practicality of the load testing. SUMMARY

[0003] The present application provides a load testing method and system of a storage device, and an electronic device, to at least solve the problem that the test results do not match the actual situation caused by using a preset standardized test case for load testing in the related art.

[0004] The present application provides a load testing method of a storage device, comprising: collecting access logs of a target user on the storage device, pre-processing the access logs, and obtaining user behavior data; extracting user behavior features based on the user behavior data to construct a user behavior portrait library; generating a user portrait test case of the target user according to the user behavior portrait library, the user portrait test case including a tidal load case with varying load intensity over time and an abnormal behavior injection case simulating abnormal behavior features; testing the storage device under injection of the load corresponding to the user portrait test case, and obtaining test results of the storage device.

[0005] The present application also provides a load testing system of a storage device, comprising a collection module, an extraction module, a generation module, and a testing module. The collection module is configured to collect access logs of a target user on the storage device, pre-process the access logs, and obtain user behavior data. The extraction module is configured to extract user behavior features based on the user behavior data to construct a user behavior portrait library. The generation module is configured to generate a user portrait test case of the target user according to the user behavior portrait library, the user portrait test case including a tidal load case and an abnormal behavior injection case. The test module is configured to execute a user portrait test case, test a load corresponding to the user portrait test case by inputting the load to the storage device under test, and obtain a test result of the storage device under test.

[0006] The application further provides an electronic device, including a memory for storing a computer program, and a processor for executing the computer program to implement the steps of the load test method of any of the storage devices.

[0007] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the load test method of any of the storage devices.

[0008] The application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of the load test method of any of the storage devices.

[0009] The application provides a load test method and system of a storage device, and an electronic device. The user behavior data is obtained by collecting and preprocessing the access log of a target user on the storage device, so that the real use behavior of the user can be obtained, and the defect that the general test case is separated from the actual user operation is avoided. The user behavior portrait library is constructed based on the user behavior data and the features are extracted, so that the load tidal fluctuation characteristics and abnormal behavior patterns of the user access can be accurately captured, and the accuracy of the test case is ensured. The exclusive test case including the tidal load case and the abnormal behavior injection case is generated according to the portrait library, wherein the tidal load case directly corresponds to the user load fluctuation rule extracted from the portrait library, the load fluctuation in the actual use of the user can be simulated, the abnormal behavior injection case matches the abnormal behavior characteristics in the portrait library, the burst scene in the user operation can be reproduced, and the potential defects of the storage device under the tidal load fluctuation and the occasional abnormal behavior are solved. The user portrait test case is executed, and the corresponding load is input to the storage device under test, so that the test process completely matches the load characteristics and the behavior patterns of the real use of the user, and the consistency between the test scene and the actual business environment is ensured.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings are used to better understand the present application, and do not constitute a limitation on the present disclosure. Among them: Figure 1 A flowchart of a load test method of a storage device provided by the embodiments of the present application is shown; Figure 2Another flowchart of a load testing method of a storage device provided by an embodiment of the present application is shown in FIG. 2. Figure 3 A structure diagram of a load testing system of a storage device provided by an embodiment of the present application is shown in FIG. 3. DETAILED DESCRIPTION

[0012] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help the understanding of the present disclosure. These should be considered in the context of the overall description and should not be considered to limit the scope of the present disclosure. Thus, those ordinarily skilled in the art will recognize various changes and modifications of the embodiments described herein, without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0013] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] In conjunction with the specific application environment architecture or the specific hardware architecture on which the load testing method of the storage device is dependent, the specific application environment architecture or the specific hardware architecture is described herein.

[0015] An embodiment of the present application provides a load testing method of a storage device, Figure 1 A flowchart of a load testing method of a storage device provided by an embodiment of the present application is shown in FIG. 1.

[0016] As shown in FIG. 1, the method comprises the following steps: Figure 1 Step 101, collecting access logs of a target user on a storage device, and pre-processing the access logs to obtain user behavior data.

[0017] ​In the embodiments of the present application, when collecting access logs of the storage device of the target user, in order to comprehensively collect the logs generated by the user using the storage device, a mode combining real-time monitoring and timing capture can be used, but is not limited to. For example, file operation events are captured in real time through the standard interface of the storage device (such as SCSI protocol, NAS interface), and a historical log supplement process (timing capture) is triggered once an hour to cover offline or delayed generated logs, so as to guarantee the integrity of data collection. Considering the diversity of storage device types, the collected logs need to be preprocessed to adapt to the log formats of different types of devices such as block storage, file storage and object storage. When collecting logs, a way of parsing device native logs (such as NTFS logs, EXT4 logs, S3 access logs) to extract unified fields can be used, but is not limited to, to solve the problem of log format difference in heterogeneous storage environment. The collected access logs cover three types of core contents: file operation records, including file size, read / write / delete operation types, operation time and operation frequency; concurrency information, including the number of access users and the number of IO requests at the same time point; and abnormal behavior records, such as burst IO of more than 1000 file writes in 1 minute, cross-privilege file access and other unexpected operations.

[0018] After collection, the preprocessing stage is entered. In the embodiments of the present application, the following exemplary processing methods can be used, but are not limited to: invalid data filtering is performed, repeated records are deleted according to the triple key values of "file path + operation time + user ID", format error items missing operation time, file size and other key fields are removed, and test records such as administrator script batch operations are filtered through user permission identification; standardization processing is performed, local time is uniformly converted to UTC time accurate to millisecond level, and file size is classified into multiple categories according to the preset size (such as 4K, 1M, 100M) of the file; finally, data aggregation is performed, file operation frequency, IO request total amount, concurrent user peak and other indicators are counted per hour, abnormal behaviors are marked according to the preset threshold and operation context is associated, and finally structured user behavior data containing "time period-file type-operation type-concurrency index-abnormality mark" is output.

[0019] Through full collection and accurate preprocessing, the integrity, accuracy and standardization of user behavior data are ensured, a reliable data foundation is provided for subsequent construction of accurate user behavior portrait, and the interference of invalid or distorted data on subsequent test process is avoided.

[0020] Step 102, based on the user behavior data, extracting user behavior features to construct a user behavior portrait library.

[0021] In the embodiments of the present application, based on the structured user behavior data output in step 101, feature extraction is carried out through statistical analysis or machine learning algorithm (such as cluster analysis), and then a user behavior portrait library is constructed. Feature extraction focuses on multiple core dimensions, for example: time feature dimension, based on time period aggregated data to define load peak (such as 8:00-18:00) and valley period (such as 0:00-6:00), calculate the duration and load proportion of each period (such as IO request in peak period accounts for 70% of the whole day); file feature dimension, analyze file size distribution in different periods (such as 4K file accounts for 60% in peak period, 1G file accounts for 80% in valley period) and read-write ratio (such as write operation accounts for 70% in peak period, read operation accounts for 90% in valley period); concurrency feature dimension, extract average concurrent user number, peak concurrent user number and IO request frequency (number of requests per second) in each period; abnormal feature dimension, sort out abnormal types (such as IO storm), occurrence frequency (such as 1-2 times per month), duration (such as 5 minutes on average) and load intensity (such as IO request is 5 times of normal period during abnormal period). It should be noted that the behavior data reflecting user operation on storage devices includes multiple dimensions, and the data of the aforementioned time feature dimension, file feature dimension and other dimensions are only exemplary and do not constitute a limitation on the scope of protection of the present application.

[0022] The extracted feature parameters are stored in a structured form (such as a relational database, a database type that stores data in a preset format and supports efficient query) to form a portrait library, and a dynamic updating mechanism is established, for example, the features are updated iteratively every 7 days according to new collected data to ensure that the portrait fits the actual habits of the user. Taking A customer as an example, the portrait can accurately present information such as "peak 9:00-22:00, 4K small file write operation accounts for 80%, concurrent peak value 200 people, IO request 500 times per second; 10-minute IO storm occurs 2 times per month on promotion day, request volume is 8 times of normal".

[0023] The accurate portrayal of user storage usage habits provides core data support for subsequent generation of test cases that match the actual scenario, solving the problem of disconnection between test cases and reality caused by lack of user behavior insight in traditional testing.

[0024] Step 103, generating a user portrait test case of a target user according to the user behavior portrait library, the user portrait test case including a tidal load case with load intensity changing with time and an abnormal behavior injection case simulating abnormal behavior characteristics.

[0025] In the embodiments of the present application, the user portrait test case of the target user is generated based on the time characteristics, file characteristics, concurrency characteristics and abnormal characteristics stored in the user behavior portrait library. The test case highly matched with the actual storage usage habits of the target user is constructed through parameterized generation logic. The core includes two key contents of tidal load test case and abnormal behavior injection test case. For the tidal load test case, first, the load peak and valley periods of the target user are accurately divided according to the time characteristics in the portrait library (for example, the A customer portrait clearly defines 9:00-22:00 as the peak period and 22:00-9:00 the next day as the valley period), then the corresponding file characteristics (such as 80% of small file write operations below 4K in the peak period and 1G large file read operations in the valley period) and concurrency characteristics (500 IO requests per second in the peak period, 200 concurrent users; 50 IO requests per second in the valley period, 20 concurrent users) are combined, and the matching load parameters are injected in the corresponding period. At the same time, in order to restore the natural fluctuation of the load in the actual business, a load switching mechanism is designed at the junction of the peak and valley periods, and the smooth transition of the load intensity is realized through a gradual algorithm (such as gradually increasing the IO request amount from 50 times per second to 500 times per second within 5 minutes), and finally the tidal load test case with dynamic changes of load intensity over time is formed. For the abnormal behavior injection test case, the key parameters such as abnormal type (such as IO storm), occurrence frequency (such as 2 times per month), duration (such as 10 minutes) and load intensity (such as 8 times of normal period) are automatically extracted based on the abnormal characteristics in the portrait library, and the abnormal scene conforming to the characteristics is randomly inserted in the running sequence of the tidal load test case (such as inserting an IO storm scene with 4000 IO requests per second for 10 minutes at 15:00 in the peak period of A customer); at the same time, the user can manually customize the abnormal parameters, including abnormal type, occurrence time point, load intensity, etc., to further improve the flexibility and accuracy of abnormal scene simulation.

[0026] The generated test case accurately replicates the actual load fluctuation rules and abnormal scene characteristics of the user, solving the scene distortion problem caused by the "averaging" of traditional general test cases, and providing scene support for subsequent tests that fit the real business.

[0027] Step 104, injecting the load corresponding to the user portrait test case into the storage device under test for testing, and obtaining the test results of the storage device under test.

[0028] Connect with the storage device under test through the storage device standard interface (such as SCSI protocol, NAS interface), and then configure the test environment parameters according to the actual deployment environment of the target user, including network bandwidth (such as matching the user's business 10Gbps bandwidth), device cache size (such as 8GB cache consistent with the user's actual use) and other key configurations, to ensure the consistency of the test environment with the user's real use scenario, and lay the foundation for the authenticity of the load test.

[0029] In the embodiments of the present application, after the environment is ready, the load injection process is started: strictly follow the timing logic of the user portrait test case, simulate the concurrent access behavior of the target user through multi-threading technology, send IO requests of corresponding types to the storage device under test - for the tidal load case, start 200 concurrent threads during the peak period (such as 9:00-22:00 of A customer), send 4K small file write requests at a frequency of 500 times per second; switch to 20 concurrent threads during the trough period (22:00-9:00 the next day), send 1G or more large file read requests at a frequency of 50 times per second, and realize smooth switching of load through thread number gradient adjustment at the junction of the period; for abnormal behavior injection cases, when running to the preset abnormal time point (such as 15:00), instantaneously increase the thread request frequency to simulate the IO storm scenario of 4000 times per second for 10 minutes.

[0030] During the whole load injection process, the test result data of the storage device under test is collected in real time at a preset sampling frequency, the collection indicators include performance indicators (response time, throughput), resource occupation indicators (CPU usage, cache hit rate) and stability indicators (error rate, fault frequency) and the like, and all data is stored in a structured form in real time. If a fault such as device timeout occurs during the test, the system automatically records the time point of the fault occurrence, the load intensity at that time (such as IO request amount, number of concurrent threads) and other context information, and triggers the fault tolerance mechanism to try to restart the test process or skip the fault period to continue the test, to ensure the integrity of the test results.

[0031] Through the injection method of accurately reproducing the user's actual load scenario and full-dimensional indicator collection, the test results obtained can truly reflect the real performance of the device under tidal load and abnormal scenarios, providing reliable data support for subsequent result analysis and optimization suggestion output.

[0032] The application provides a load test method of a storage device. The user behavior data is obtained by collecting the access log of a target user on the storage device and preprocessing, so that the real use behavior of the user can be obtained, and the defect that the general test case is separated from the actual user operation is avoided. The user behavior portrait library is constructed based on the user behavior data, so that the load tidal fluctuation characteristics and abnormal behavior patterns of the user access can be accurately captured, and the accuracy of the test case is ensured. The exclusive test case including the tidal load case and the abnormal behavior injection case is generated according to the portrait library, wherein the tidal load case directly corresponds to the user load fluctuation rule extracted from the portrait library, the load fluctuation in the actual use of the user can be simulated, the abnormal behavior injection case matches the abnormal behavior characteristics in the portrait library, the burst scene in the user operation can be reproduced, and the potential defects of the storage device under the tidal load fluctuation and the accidental abnormal behavior are solved. The user portrait test case is executed, and the corresponding load is injected into the measured device, so that the test process is completely consistent with the load characteristics and the behavior pattern of the real use of the user, and the consistency of the test scene and the actual business environment is ensured.

[0033] The above-mentioned embodiments do not constitute a limitation on the protection scope of the present application, that is, in addition to the above-mentioned embodiments, other embodiments that can be obtained by reasonable logical analysis, reasoning or limited experiments based on the technical content disclosed in the present application by those skilled in the art should also be covered within the protection scope of the present application. Another embodiment is provided in the present application, as shown in Figure 2 The embodiment includes the following steps: Step 201, capturing the access log by listening to the storage device interface and timing the access log.

[0034] In this step, the dual-mode of real-time listening and timing is used to carry out the access log collection work. Real-time listening is realized by connecting the standard interface of the storage device, such as the SCSI protocol interface for block storage data interaction, the NAS interface for file sharing scene, etc., to continuously capture various operation events such as file reading, writing, deleting, backing up, etc. that occur on the device, to ensure that each storage access behavior can be recorded in real time. At the same time, in order to avoid the omission of logs caused by offline generation or transmission delay, a timing capture mechanism is started, which automatically triggers a historical log supplement process every hour, and the log entries that are not captured in real time are obtained through the preset log storage path of the device. In addition, for different types of storage devices such as block storage, file storage and object storage, by analyzing the original log format (such as NTFS file system log, EXT4 file system log, object storage S3 access log, etc.), the unified core fields such as “file path, operation time, user ID, file size” are extracted, the log format difference problem in the heterogeneous storage environment is solved, and the standardized collection of multi-source logs is realized.

[0035] The dual-mode collection ensures the integrity of the log, and the multi-source adaptation realizes the unified acquisition of cross-device logs, providing a comprehensive and reliable raw data foundation for subsequent data cleaning and feature extraction.

[0036] In step 202, the access log is cleaned to filter out invalid data.

[0037] In this step, the collected access log performs data cleaning operation, and the core target is to filter out invalid data to ensure the accuracy of subsequent data processing. This is achieved through three types of rules. For duplicate records, a three-key combination of "file path + operation time + user ID" is used for deduplication judgment. Duplicate operation records performed by the same user on the same file at the same time are identified as redundant data and deleted, as such duplicate records cannot add effective information to the behavior description, but will cause data redundancy. For format error entries, field verification mechanism is used to remove records missing key information such as "operation time" and "file size". Such data loses its value as it cannot support subsequent feature extraction (such as time period division and file type statistics). In addition, test operation records need to be filtered out. Non-real business behaviors such as administrator manual script batch operations are distinguished by user permission identification to avoid interference of such artificial test data on the judgment of user actual usage habits. During the cleaning process, all filtered data will be marked with the reason and retained in the log to ensure that the cleaning operation is traceable.

[0038] By accurately filtering out duplicate, incomplete and test data, the quality of the access log is significantly improved, providing pure and effective data input for subsequent standardized processing and feature extraction.

[0039] In step 203, the access log after data cleaning is standardized to obtain user behavior data.

[0040] Specifically in this step, the standardized processing takes the access log after data cleaning as input, and the core is to carry out unified standardization operation around two key data of time format and file size, and finally form structured user behavior data. In terms of time format standardization, in view of the possible local time (such as CST China Standard Time) format difference in the log, the time conversion algorithm is used to convert it into UTC coordinated universal time, and the time accuracy is accurate to millisecond level. This unified format facilitates subsequent time period analysis and data comparison in cross-time zone scenarios, and avoids time feature extraction errors caused by time zone deviation. In terms of file size standardization, according to the preset threshold system (4K, 1M, 100M), the file size value recorded in the log is mapped into four types of standardized classification, namely "below 4K", "4K-1M", "1M-100M" and "above 100M". Through this hierarchical processing, the dimension of subsequent file feature extraction is simplified, and the data operation complexity is reduced. After the standardization processing is completed, the output data is structured and integrated based on the "time period-file type-operation type-concurrent index" as the basic dimension, so as to ensure that the cleaned logs of different sources and different formats form a unified and standardized user behavior data format, and provide a consistent data basis for subsequent anomaly marking and feature extraction.

[0041] Through the standardization of time and file size, the interference caused by data format difference is eliminated, the consistency and standardization of user behavior data are ensured, and key support is provided for the accuracy of subsequent multi-dimensional feature extraction.

[0042] Step 204, based on the preset abnormal threshold, marking the data in the user behavior data as abnormal.

[0043] Specifically in this step, the standardized user behavior data obtained in step 203 is taken as the processing object, and the abnormal marking work is carried out based on the preset multi-dimensional abnormal threshold. The preset abnormal threshold is set in combination with the typical abnormal scene in the actual use of the storage device, and covers two types of core dimensions: one is the load intensity threshold, for example, setting "IO request quantity exceeding 1000 times in 1 minute" as the burst IO determination standard, and this threshold is determined based on the statistical upper limit of the IO request frequency in the normal business period of most enterprises; the second is the operation permission threshold, which includes unexpected behaviors such as "cross- permission file access" and "unauthorized batch operation" in the abnormal judgment range, and realizes the identification by comparing the operation user permission identifier and the file access permission configuration.

[0044] In a specific operation, the structured user behavior data is scanned and compared in real time: when the IO request frequency of a period reaches or exceeds the load intensity threshold, the period data is immediately marked as "burst IO anomaly"; when it is detected that the access behavior of the operation user breaks the permission limit, the corresponding operation record is marked as "permission anomaly". At the same time, in order to retain the complete background information of the abnormal behavior, the corresponding operation context needs to be associated and marked, including operation user ID, file path, operation time point and the number of concurrent users at that time and other key data, ensuring that each abnormality mark is accompanied by traceable scene information, and finally outputting the marked user behavior data containing "abnormal type-occurrence period-associated context".

[0045] Through accurate threshold setting and context association marking, effective identification and retention of abnormal information in user behavior data are realized, which provides accurate data support for abnormal feature generation in subsequent multi-dimensional feature extraction, and guarantees the accuracy of user behavior portrait in abnormal scene description.

[0046] Step 205, multi-dimensional feature extraction of user behavior data is performed through a clustering algorithm to generate multi-dimensional feature parameters, including time features, file features, concurrency features and abnormal features.

[0047] Further, for the operation of "multi-dimensional feature extraction of user behavior data through a clustering algorithm to generate multi-dimensional feature parameters", there are multiple feasible specific implementation modes. In order to clearly and completely describe the technical solutions of the present disclosure, the following enumerated embodiments are only exemplary and do not constitute a limitation on the protection scope of the present disclosure, and the following specific introduction part describes some exemplary embodiments: the time data in the user behavior data is classified to divide the peak period and the valley period of the running load on the storage device, to generate time features; the file size distribution in the peak period and the valley period is obtained, and the read-write operation ratio of the peak period and the valley period is obtained, to generate file features; the average number of concurrent users, the peak number of concurrent users and the read-write request frequency in the peak period and the valley period are collected, to generate concurrency features; the user behavior data is analyzed to determine the data with abnormal behavior, to generate abnormal features.

[0048] Specifically, in this step, the user behavior data marked as abnormal is input, multi-dimensional feature extraction is carried out through a clustering algorithm (a machine learning method that aggregates similar data points into categories, which is used here to mine the common behavior patterns hidden in the data), and multi-dimensional feature parameters including time, file, concurrency, and abnormal features are generated. The specific implementation is as follows: for time feature generation, the "time period-load intensity" data in the user behavior data is used as a clustering sample, and the clustering algorithm is used to aggregate time periods with similar load intensity into two categories, which are defined as peak hours (such as 8:00-18:00) and off-peak hours, respectively. Then, the duration and load proportion of the two categories are calculated (such as peak IO requests accounting for 70% of the whole day), forming the time feature; for file features, based on the clustering division of peak and off-peak hours, the distribution proportion of normalized file size (4K or less, 1M or more, etc.) in each period is calculated, and the difference in read-write operation proportion (such as peak write operation accounting for 70%, and off-peak read operation accounting for 90%) is calculated, and the file features are integrated and generated; in the concurrency feature extraction, for the two categories of time periods, the average concurrent user number, peak concurrent user number, and read-write request frequency per second (IO request total amount / time period duration) are collected and calculated, forming the concurrency feature; and the abnormal feature is generated by clustering the marked abnormal data, extracting abnormal types (such as IO storm), occurrence frequency (1-2 times per month), duration (average 5 minutes), and load intensity (abnormal IO is 5 times normal) and other common parameters.

[0049] With the help of clustering algorithm, the behavior data pattern mining is realized, and the multi-dimensional feature parameters are accurately extracted, which provides core data support for subsequent construction of user behavior portrait library that fits the actual situation.

[0050] Step 206: Store the multi-dimensional feature parameters of the target user in the pre-set database according to the pre-set structure data to construct and generate the user behavior portrait library.

[0051] Specifically in this step, the user behavior portrait library is constructed with the target user multi-dimensional feature parameters (time feature, file feature, concurrency feature, and abnormal feature) generated in step 205 as core data, and is stored in a preset database according to a preset structure. The preset database is a relational database (a database type that organizes data through tables, rows, and columns, supports structured query and association operation), which is suitable for the structured storage requirement of the feature parameters; the preset structure is designed according to the core attributes of the four types of features, for example, the time feature corresponds to the fields of “peak period, valley period, and peak load proportion”, the file feature corresponds to the fields of “peak file size distribution, valley file size distribution, and peak read-write ratio”, the concurrency feature corresponds to the fields of “average concurrent user number, peak concurrent user number, and IO request frequency”, and the abnormal feature corresponds to the fields of “abnormal type, occurrence frequency, duration, and load intensity”, and the fields of “user ID and feature update timestamp” are added for user identification and update tracing. When storing, each type of feature parameter of the target user is mapped to the corresponding field one by one, for example, the parameters of “peak period 9:00-22:00, 4K file proportion 80%, concurrent peak 200, and IO storm 2 times per month” of A customer are filled into the corresponding fields to form a complete data record. In addition, in order to adapt to the change of user behavior, the data table is associated with a dynamic update mechanism, the old data is identified through the “feature update timestamp”, and the historical record is replaced by the newly generated multi-dimensional feature parameters every 7 days, so as to ensure that the portrait library is synchronized with the actual use habits of the user.

[0052] Through the structured storage and dynamic update mechanism, the standardized management and real-time iteration of the multi-dimensional feature parameters are realized, and the constructed user behavior portrait library can accurately and continuously adapt to the user habits, and provides stable and reliable feature support for subsequent test case generation.

[0053] In step 207, according to the file feature and concurrency feature in the peak period and the file feature and concurrency feature in the valley period, a tidal load test case is generated.

[0054] Specifically in this step, the tidal load use case is generated with the file features and concurrency features of the peak period and the trough period in the user behavior portrait library as the core basis, and is realized through the accurate mapping of the feature parameters and the load use case parameters. First, based on the time feature, the time period division framework of the test use case is determined, and the use case execution period is divided into the peak test period and the trough test period to ensure consistency with the actual load time law of the target user. For the peak test period, the key parameters of the file feature, such as the file size distribution ratio (such as 80% of small files below 4K) and the read-write operation ratio (such as 70% of write operation), are extracted, and the core indicators of the concurrency feature, such as the peak concurrent user number (such as 200 people) and the IO request frequency (such as 500 times per second), are called, and these parameters are converted into load injection rules, that is, “continuously inject small files below 4K in the peak period, with a write operation ratio of 70%, simulate 200 concurrent users, and send 500 IO requests per second”. For the trough test period, the corresponding feature parameters are also matched: according to the rules of “80% of large files above 1G, 90% of read operation” in the file feature, combined with the indicators of “20 people of average concurrent user number, 50 times of IO request per second” in the concurrency feature, the load rule of “inject large files above 1G in the trough period, with a read operation ratio of 90%, simulate 20 concurrent users, and send 50 IO requests per second” is formulated. Finally, the load rules of the peak and trough periods are concatenated in time sequence to form a tidal load use case with alternating load intensity with time period, which completely reproduces the peak-trough fluctuation law of the actual user storage access.

[0055] By directly mapping the file and concurrency features of the user real time period, the generated tidal load use case accurately restores the load fluctuation characteristics in the actual business, solves the scene distortion problem caused by the “averaging” of the traditional general use case, and provides support for the subsequent test of the load scene that fits the real business.

[0056] Step 208, at the junction of the peak period and the trough period, a load switching use case with gradually changing load intensity is generated.

[0057] Specifically in this step, the load switching case with load intensity gradient is generated based on the characteristic parameters of peak and valley periods in the user behavior profile library, and the core focuses on the load smooth transition design at the junction of the two periods. First, the key feature boundary values of peak and valley periods are extracted from the profile library, including the IO request frequency in the concurrency feature (such as 50 times per second at the end of the valley, 500 times per second at the beginning of the peak), the peak concurrent user number (such as 20 people at the end of the valley, 200 people at the beginning of the peak), and the file size distribution ratio in the file feature (such as 80% of files above 1G at the end of the valley, 80% of files below 4K at the beginning of the peak), and the read-write operation ratio (such as 90% of read operations at the end of the valley, 70% of write operations at the beginning of the peak), and at the same time, the preset transition period (such as 5 minutes) is combined with the time length rule of natural load change in actual business. Subsequently, based on the linear gradient algorithm (a gradient method that keeps the parameter change rate stable), the unit time increment of each characteristic parameter is calculated: the IO request frequency is calculated for 300 seconds in 5 minutes, and it needs to be increased by (500-50) / 300=1.5 times per second; the concurrent user number needs to be increased by (200-20) / 300=0.6 people per second; the file size distribution ratio needs to be adjusted by (80%-20%) / 300 of the proportion gradient per second, and the read-write operation ratio needs to be switched at a rate of (70%-10%) / 300 per second. When generating the case, these gradient parameters are decomposed into second-level execution instructions in time sequence to ensure that the IO request frequency, concurrent user number, file type proportion, read-write ratio and other parameters increase or decrease synchronously and smoothly in the junction period, avoiding load mutation. For example, when transitioning from the valley to the peak, the case instruction is "1st second: IO request 51.5 times per second, concurrent 20.6 people, 1G above file 79.8%, read operation 89.8%; 2nd second: IO request 53 times per second, concurrent 21.2 people, 1G above file 79.6%, read operation 89.6%……300th second: IO request 500 times per second, concurrent 200 people, 4K below file 80%, write operation 70%", and finally the load switching case with load intensity gradient is formed.

[0058] By simulating the natural gradient process of the load in real business, the distortion of the test scene caused by load mutation is avoided, and the stability and adaptability of the device at the load fluctuation critical point can be accurately verified, providing more actual test basis for device performance evaluation.

[0059] In step 209, according to the abnormal characteristics, the simulation abnormal test case is inserted at the preset time point, and the simulation abnormal test case is modified based on the received abnormal parameters to generate the abnormal behavior injection case.

[0060] Specifically in this step, the abnormal behavior injection use case is generated based on the abnormal features in the user behavior portrait library, combined with user-defined parameters to complete accurate construction. First, the abnormal feature core parameters of the target user are extracted from the user behavior portrait library, including abnormal type (such as IO storm, cross-privilege file access, sudden batch file upload, etc.), historical occurrence frequency (such as 2 times per month), duration (such as an average of 10 minutes), and load intensity (such as 8 times the IO request amount during normal period during abnormal period), which constitute the initial template basis for simulating abnormal test cases. Subsequently, in combination with the load characteristics of the storage device, a preset time point is determined - the peak period is preferentially selected (such as 15:00 within 9:00-22:00 marked in the portrait library, when the load is dense and it is easier to expose device defects), the initial simulation abnormal test case is inserted into the time point, and the basic use case is formed (such as "start IO storm simulation for 10 minutes at 15:00, IO request frequency 4000 times / sec").

[0061] On this basis, the system receives user-manual set abnormal parameters, which can adjust dimensions including abnormal type (such as modifying IO storm to file deletion storm), occurrence time point (such as adjusting to 16:30), load intensity (such as increasing to 10 times the normal period), duration (such as extending to 15 minutes), etc. According to the received parameters, the basic simulation abnormal test case is modified, for example, "IO storm" is replaced by "file deletion storm", the IO request frequency is adjusted to 5000 times / sec, the duration is updated to 15 minutes, and finally the abnormal behavior injection use case that fits the user's actual needs and custom scenarios is generated.

[0062] Both relying on historical abnormal features to ensure the authenticity of the use case and through user-defined parameters to improve the flexibility of the scene, the abnormal behavior injection use case generated by the system can comprehensively cover real and personalized abnormal scenarios, solving the problem of insufficient abnormal coverage in traditional testing.

[0063] Step 210, the tidal load use case, load switching use case, and abnormal behavior injection use case are subjected to parameterization processing to generate user portrait test cases.

[0064] Specifically in this step, the use case parameterization process is based on the tidal load use case, load switching use case, and abnormal behavior injection use case. By abstracting core variables and establishing parameter mapping rules, standardized integration is achieved, and finally user portrait test cases (UPTC) are generated. In specific operations, first, the key execution attributes of each use case are parameterized: for the tidal load use case, the peak / valley period range, the file size distribution ratio of each period (such as {high peak 4K file ratio}), the read / write operation ratio (such as {valley read operation ratio}), the number of concurrent users (such as {peak concurrent number}), the IO request frequency (such as {valley IO frequency}), etc. are abstracted as independent parameters; for the load switching use case, the transition period (such as {load gradual change length}) and the parameter gradual rate (such as {IO frequency increment per second}) are extracted; for the abnormal behavior injection use case, the abnormal type (such as {abnormal type}), the occurrence time point (such as {abnormal trigger time}), the duration (such as {abnormal duration}), and the load intensity multiple (such as {abnormal load multiple}) are set as adjustable parameters.

[0065] Subsequently, a parameter association mechanism is established to ensure the logical consistency of different use case parameters: for example, the {initial IO frequency} of the load switching use case is associated with the {valley end IO frequency} of the tidal load use case, and the {terminal IO frequency} is associated with the {peak start IO frequency} of the tidal load use case; the {abnormal trigger time} of the abnormal behavior injection use case must fall within the preset period of the tidal load use case (such as {peak period}). Finally, all parameterized use case segments are concatenated in time sequence, stored in a structured format (such as a script template containing parameter placeholders), and formed into a user portrait test case containing "period parameters-load parameters-switching parameters-exception parameters". The parameter values can be directly associated with the user behavior portrait library for dynamic calling.

[0066] Parameterization processing makes the test case free from fixed numerical limits, and can automatically adapt to parameter changes with the user behavior portrait library update, without the need to redesign the use case. At the same time, the flexibility and reusability of the use case are improved, ensuring the dynamic matching of the test scenario and user habits.

[0067] Step 211, execute the preset test case, inject load into the storage device under test, and collect the performance indicators of the storage device under test to obtain the test results.

[0068] Specifically in this step, first of all, the test environment initialization work is carried out, and a stable connection with the tested storage device is established through the standard interface of the storage device (such as SCSI protocol, NAS interface), and then according to the actual deployment scene of the target user, the core environment parameters such as network bandwidth and device cache size are accurately configured, to ensure that the test environment is highly consistent with the user's real use scene, and lay the foundation for the authenticity of load injection. After completing the initialization, start the execution process of the preset test case - according to the case timing plan, simulate concurrent user behavior through multi-threading technology, and inject load to the tested device: for the tidal load segment, send requests according to the preset file type, read-write ratio and IO frequency in the corresponding period; the load switching segment adjusts the load intensity smoothly according to the gradual parameters; the abnormal injection segment triggers burst load at the preset time point.

[0069] During the whole process of load injection, the performance indicators of the tested device are collected in real time at a sampling frequency of 1 per second, covering core performance indicators (response time, throughput), resource occupation indicators (CPU usage, cache hit rate) and stability indicators (error rate, fault frequency) and the like. All collected data is stored in a structured format (such as JSON, CSV) in real time to ensure that the indicators are traceable. If a fault such as device timeout occurs during the test, the system automatically records the fault time point, the load intensity at that time (such as IO request amount, number of concurrent threads) and other context information, and at the same time triggers the fault tolerance mechanism to try to restart the test process or skip the fault period to continue execution, to ensure the integrity of the test results.

[0070] Through environment adaptation and accurate load injection, combined with full-dimensional and high-frequency index collection, the test results obtained can truly reflect the performance and stability of the device under the preset scene, providing reliable data support for subsequent result comparison and analysis.

[0071] Step 212, compare and analyze the test results of the user portrait test case with the test results of the preset test case, and output an analysis report and optimization suggestions.

[0072] Specifically in this step, the test results of the user portrait test case (UPTC) and the preset test case (general test case GTC) are compared and analyzed as the core object, and accurate comparison is carried out around the three dimensions of performance, stability and abnormal response ability, and then an analysis report and optimization suggestions are output. First, sort out the core test data of the two types of cases: the UPTC data reflects the performance of the device under the actual tidal load of the user and the real abnormal scene, and the GTC data reflects the performance of the device under the standardized average scene.

[0073] The performance difference dimension compares key indicators such as response time and throughput, for example, the average response time of UPTC in the peak period is 30ms, and the average response time of GTC is 15ms, and the performance gap between the actual scene and the general scene is quantified by the data difference and the proportion; the stability difference dimension is used to count the fault frequency (such as timeout, buffer overflow, etc.) of the equipment under the two types of use cases, if UPTC has 2 buffer overflows and GTC has no fault, the stability short board is marked; the abnormal response ability dimension focuses on the performance of the equipment in the abnormal injection period of UPTC, and compares the index fluctuation of GTC without abnormal scene, for example, the throughput of UPTC decreases by 50% during the IO storm, while GTC has no fluctuation, and the equipment burst load response defect is defined.

[0074] The analysis report is presented in a visual form, including a response time comparison curve, a fault frequency table and an abnormal period index change graph, supplemented by a written explanation of the core causes of the difference. The optimization suggestions are targeted: when the performance is insufficient, it is suggested to "increase the read cache capacity to 16GB and enable small file aggregation write mechanism"; when the stability is lacking, it is suggested to "optimize the IO scheduling algorithm and increase the load prediction module"; when the abnormal response is weak, it is suggested to "configure an SSD cache pool to temporarily cache burst writes".

[0075] By quantifying and comparing, the short board of the equipment adapting to the actual scene is clear, the targeted suggestions provide a direct basis for enterprise optimization of storage configuration, and the problem of insufficient practicality of traditional test results is solved.

[0076] It should be noted that the embodiments of the present disclosure can include a plurality of steps, which are numbered for the sake of description, but these numbers do not limit the execution time slots and execution order between the steps; the steps can be implemented in any order, and the embodiments of the present disclosure do not limit this.

[0077] Corresponding to the above-mentioned load testing method of the storage device, the present application also proposes a load testing system of the storage device. Since the system embodiments of the present application correspond to the above-mentioned method embodiments, for the details not disclosed in the system embodiments, reference can be made to the above-mentioned method embodiments, which will not be described in detail in the present disclosure.

[0078] Figure 3 A structural schematic diagram of a load testing system of a storage device provided by an embodiment of the present application is shown as Figure 3 shown, comprising: a collection module 21, an extraction module 22, a generation module 23, and a testing module 24.

[0079] The collection module 21 is configured to collect access logs of target users on the storage device, and pre-process the access logs to obtain user behavior data; The extraction module 22 is configured to extract user behavior features based on the user behavior data to construct a user behavior portrait library. The generating module 23 is configured to generate a user portrait test case of the target user according to the user behavior portrait library, the user portrait test case including a tidal load case and an abnormal behavior injection case; The testing module 24 is configured to execute the user portrait test case, inject the load corresponding to the user portrait test case into the storage device under test to test the storage device under test, and obtain a test result of the storage device under test.

[0080] It should be noted that the above explanations and descriptions of the method embodiments are also applicable to the same system principles of the present embodiment, which are not limited in the present embodiment.

[0081] The features of the embodiments of the load testing system of the storage device can be referred to the related descriptions of the embodiments of the load testing method of the storage device, which will not be repeated here.

[0082] Embodiments of the present application also provide an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the load testing method of the storage device.

[0083] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above-mentioned embodiments of the load testing method of the storage device when running.

[0084] In an exemplary embodiment, the above-mentioned computer readable storage medium can include but is not limited to: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0085] Embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps in any of the above-mentioned embodiments of the load testing method of the storage device.

[0086] Embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above-mentioned embodiments of the load testing method of the storage device.

[0087] Those skilled in the art will further realize that the mere concepts, teachings, and embodiments described herein are merely meant to provide an enabling description of the claimed application. Accordingly, modifications and / or additions, other than those explicitly described herein, can be obvious to those skilled in the art in the light of this disclosure. The claimed application is intended to embrace all such modifications and / or additions.

[0088] The method and system for load testing of a storage device, and the electronic device are described in detail above. The principles and implementation manners of the present application are described by applying specific examples in the present document. The above description of the embodiments is only for the purpose of helping to understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A load testing method for a storage device, characterized in that, include: Collect access logs of the target user on the storage device, preprocess the access logs to obtain user behavior data; Based on the user behavior data, user behavior features are extracted to construct a user behavior profile library; Based on the user behavior profile database, user profile test cases for the target user are generated. The user profile test cases include tidal load test cases with load intensity varying over time and abnormal behavior injection test cases that simulate abnormal behavior characteristics. The test load corresponding to the user profile test case is injected into the storage device under test for testing, and the test results of the storage device under test are obtained.

2. The load testing method for a storage device according to claim 1, characterized in that, The step of extracting user behavior features based on the user behavior data to construct a user behavior profile library includes: Multidimensional features are extracted from the user behavior data using a clustering algorithm to generate multidimensional feature parameters, which include time features, file features, concurrency features, and anomaly features. The multidimensional feature parameters of the target user are stored in a preset database according to a preset structure to construct and generate the user behavior profile library.

3. The load testing method for a storage device according to claim 2, characterized in that, The step of extracting multidimensional features from the user behavior data using a clustering algorithm to generate multidimensional feature parameters includes: The time data in the user behavior data is classified to divide the peak and off-peak periods of the load on the storage device, so as to generate the time features; The file size distribution during peak and off-peak periods is obtained, and the read / write operation ratio during the peak and off-peak periods is obtained to generate the file characteristics. The average number of concurrent users, peak number of concurrent users, and read / write request frequency during peak and off-peak periods are collected to generate the concurrency characteristics. The user behavior data is analyzed to identify data with abnormal behavior, and the abnormal features are generated.

4. The load testing method for a storage device according to claim 3, characterized in that, The step of generating user profile test cases for the target user based on the user behavior profile database includes: Based on the file characteristics and concurrency characteristics during the peak period and the file characteristics and concurrency characteristics during the off-peak period, the tidal load use cases are generated. At the boundary between the peak period and the off-peak period, a load switching use case with gradually changing load intensity is generated; Based on the abnormal characteristics, simulated abnormal test cases are inserted at preset time points, and based on the abnormal parameters set in the received data, the simulated abnormal test cases are modified to generate abnormal behavior injection test cases. The tidal load test case, the load switching test case, and the abnormal behavior injection test case are parameterized to generate the user profile test cases.

5. The load testing method for a storage device according to claim 1, characterized in that, The method involves collecting access logs from the target user on the storage device, preprocessing the access logs to obtain user behavior data, including: Capture access logs by listening to the storage device interface and periodically retrieve access logs; The access logs are cleaned to filter out invalid data; The cleaned access logs are standardized to obtain the user behavior data.

6. The load testing method for a storage device according to claim 5, characterized in that, The method further includes: Based on a preset anomaly threshold, the data in the user behavior data is marked as abnormal.

7. The load testing method for a storage device according to claim 1, characterized in that, The method further includes: Execute preset test cases to inject load into the storage device under test and collect the performance indicators of the storage device under test to obtain test results.

8. The load testing method for a storage device according to claim 7, characterized in that, The method further includes: The test results of the user profile test cases are compared and analyzed with the test results of the preset test cases, and an analysis report and optimization suggestions are output.

9. A load testing system for a storage device, characterized in that, include: Acquisition module, extraction module, generation module, and testing module; The acquisition module is configured to collect access logs of the target user on the storage device, preprocess the access logs, and obtain user behavior data. The extraction module is configured to extract user behavior features based on the user behavior data to build a user behavior profile library; The generation module is configured to generate user profile test cases for the target user based on the user behavior profile library. The user profile test cases include tidal load test cases and abnormal behavior injection test cases. The testing module is configured to execute the user profile test cases, inject the load corresponding to the user profile test cases into the storage device under test for testing, and obtain the test results of the storage device under test.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the load testing method for the storage device according to any one of claims 1-8.

Images