Reliability test method, device, equipment and medium

Abstract

The invention discloses a reliability testing method, device and equipment and a medium, and belongs to the technical field of testing, and the method comprises the steps: carrying out the fault reliability testing of an all-in-one machine according to a target fault testing scene case library; dividing the full-amount fault reliability test result data set of the all-in-one machine into a plurality of fault reliability test result data subsets; performing average valuing or voting selection on the risk prediction results of the plurality of fault reliability test result data subsets to obtain a prediction result of target fault reliability test result data; and determining a risk level of each fault test scene case according to a prediction result of the target fault reliability test result data. According to the method, the corresponding scene test case can be executed according to needs, the test strength is enhanced, the reliability problem of a product is mined more deeply, the reliability of the product is improved, and then the accuracy, integrity and safety of data are improved.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to a reliability testing method, device, equipment and medium. Background Art

[0002] At present, most all-in-one products start from the idea of ​​software reliability testing, simulating the failures that users may trigger from the product function layer, testing through common software and hardware failures, and then confirming whether the product can make corresponding responses to protect the data. After the environment is restored, the data and platform will return to normal. In other words, the system's fault tolerance, data protection and automatic recovery capabilities in the event of a failure are verified.

[0003] This testing mechanism can verify the reliability of the most common usage scenarios from a multi-user perspective; however, the design of a real system covers many levels, from the operating system kernel to the virtualization layer, to the platform layer, and even to the application layer, rather than just triggering failures from the interface; because the IO system is always changing dynamically and is configured with different hardware, the path from task issuance to code processing is also very complex, resulting in random reliability issues, which will indirectly lead to probabilistic reliability issues in products released to customer sites, and even serious data loss. Summary of the invention

[0004] In order to overcome the above-mentioned technical defects, the purpose of the present application is to provide a reliability testing method, device, equipment and medium, the method comprising: obtaining the storage performance configuration parameters, storage usage data and fault type of the all-in-one machine; creating a list of fault reliability test influencing factors according to the storage performance configuration parameters, storage usage data and fault type of the all-in-one machine; performing permutation, combination and screening on the influencing factors in the list of fault reliability test influencing factors to obtain a target fault test scenario case library; performing a fault reliability test on the all-in-one machine according to the target fault test scenario case library to obtain a full fault reliability test result data set of the all-in-one machine; dividing the full fault reliability test result data set of the all-in-one machine into several fault reliability test result data subsets; obtaining a bootstrap aggregation model; performing risk prediction on each fault reliability test result data subset through the bootstrap aggregation model; averaging or voting on the risk prediction results of several fault reliability test result data subsets to obtain the prediction result of the target fault reliability test result data; determining the risk level of each fault test scenario case according to the prediction result of the target fault reliability test result data. This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0005] The specific technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, the present application provides a reliability testing method, which is applied to an all-in-one machine, and the method includes:

[0007] Obtaining storage performance configuration parameters, storage usage data, and fault types of the integrated device;

[0008] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0009] Performing permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library;

[0010] Performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device;

[0011] Dividing the full fault reliability test result data set of the integrated device into a plurality of fault reliability test result data subsets;

[0012] Get the ensemble learning model;

[0013] Perform risk prediction on each failure reliability test result data subset by using the integrated learning model;

[0014] The risk prediction results of several subsets of fault reliability test result data are averaged or voted to obtain the prediction results of the target fault reliability test result data;

[0015] The risk level of each fault test scenario use case is determined according to the prediction result of the target fault reliability test result data.

[0016] In one of the embodiments, creating a list of factors affecting a fault reliability test according to the storage performance configuration parameters, storage usage data, and fault types of the integrated machine, performing permutation and combination screening on the factors affecting the list of factors affecting the fault reliability test, and obtaining a target fault test scenario case library includes:

[0017] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data, and fault types of the integrated machine, wherein the storage performance configuration parameters include hybrid flash storage and all-flash storage, and the fault types include network failure, host failure, and disk failure;

[0018] Arrange and combine the influencing factors in the fault reliability test influencing factor list to generate a fault reliability test influencing factor combination;

[0019] The combination of factors affecting the fault reliability test is screened according to the product parameters of the all-in-one device, wherein the product parameters of the all-in-one device include product deployment environment parameters, technology stack parameters, and business logic parameters;

[0020] The screened failure reliability test influencing factors are used as the target failure test scenario case library.

[0021] In one embodiment, the performing a fault reliability test on the integrated device according to the target fault test scenario case library includes:

[0022] Get the configuration file list;

[0023] Expanding the target fault test scenario case library through the configuration file list;

[0024] According to the test framework and the expanded fault test scenario case library, the fault reliability test is performed on the all-in-one device one by one.

[0025] In one embodiment, the performing risk prediction on each failure reliability test result data subset by the integrated learning model includes:

[0026] Creating a test data set according to the failure reliability test result data subset;

[0027] Dividing the test data set into a test set and a training set by a data preprocessing function, wherein the test set and the training set both include feature data and label data;

[0028] Fitting and simulating the ensemble learning model through the training set data; wherein the ensemble learning model is a bootstrap aggregation model;

[0029] Predicting the feature data of the test set by fitting the simulated bootstrap aggregation model;

[0030] The fitting simulation of the bootstrap aggregation model using the training set data includes:

[0031] Setting the number of base learners of the bootstrap aggregation model to a first preset value;

[0032] Setting a seed parameter of a random number generator of the bootstrap aggregation model to a second preset value;

[0033] The bootstrap aggregation model is fitted and simulated using the feature data and label data of the training set data.

[0034] In one embodiment, after predicting the feature data of the test set by the bootstrap aggregation model completed by fitting simulation, the method includes:

[0035] Verifying the accuracy of the prediction results of the test set;

[0036] The checking of the accuracy of the prediction result of the test set includes:

[0037] Comparing the initial label data of the test set with the label data of the test set predicted by the bootstrap aggregation model by using a consistency evaluation function of the classification model;

[0038] In response to the initial label data of the test set being the same as the label data of the test set predicted by the bootstrap aggregation model, setting the predicted label data of the test set as the correctly predicted label sample data;

[0039] The accuracy of the test set prediction results is calculated using the formula: number of correctly predicted label sample data / number of initial label data in the test set = accuracy of the prediction results.

[0040] In one embodiment, determining the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data includes:

[0041] Evaluate the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data;

[0042] When the number of failures of the fault test execution according to the fault scenario is greater than or equal to a third preset value, the fault scenario use case is defined as a high-risk fault scenario use case;

[0043] When the number of failures of the fault test execution according to the fault scenario is less than a third preset value, the fault scenario use case is defined as a low-risk fault scenario use case;

[0044] Executing the fault scenario use case through the configuration file list;

[0045] When executing high-risk failure scenario use cases, the reliability test mode is set to sensitive mode;

[0046] When executing low-risk failure scenario use cases, the reliability test mode is set to normal mode.

[0047] In one embodiment, after determining the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data, the method includes:

[0048] Monitoring the fault reliability test process of the integrated device through a monitoring plug-in;

[0049] The monitoring of the integrated machine fault reliability test process by the monitoring plug-in includes:

[0050] When any fault scenario test case fails, an alarm will be issued for the error result of the fault scenario test case;

[0051] Record the failure scenario use case pass rate and test log of the failure reliability test.

[0052] In a second aspect, the present application further provides a reliability testing device, which is applied to an all-in-one machine, and comprises:

[0053] An acquisition module, used to acquire storage performance configuration parameters, storage usage data and fault types of the integrated machine; and to acquire an integrated learning model;

[0054] A creation module, used to create a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0055] A screening module, used for performing permutation and combination screening on the influencing factors in the list of influencing factors of the fault reliability test to obtain a target fault test scenario case library;

[0056] A testing module, used to perform a fault reliability test on the integrated device according to the target fault test scenario case library, and obtain a full fault reliability test result data set of the integrated device;

[0057] A partitioning module, used to partition the full fault reliability test result data set of the integrated machine into a plurality of fault reliability test result data subsets;

[0058] A prediction module, used to perform risk prediction on each failure reliability test result data subset through an integrated learning model;

[0059] A determination module is used to average or vote on the risk prediction results of several subsets of fault reliability test result data to obtain the prediction results of the target fault reliability test result data; and determine the risk level of each fault test scenario use case based on the prediction results of the target fault reliability test result data.

[0060] In a third aspect, a reliability testing device is also provided, comprising:

[0061] one or more processors;

[0062] A storage device for storing one or more programs;

[0063] When the one or more programs are executed by the one or more processors, the one or more processors implement the reliability testing method as described in any one of the first aspects.

[0064] In a fourth aspect, the present application further provides a computer device, the device comprising:

[0065] A memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the reliability testing method as described in any one of the first aspects.

[0066] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any reliability testing method described in the first aspect.

[0067] In a sixth aspect, the present application further provides a computer storage medium, the medium comprising:

[0068] A computer program is stored thereon, and when the computer program is executed by a processor, the steps of any reliability testing method described in the first aspect are implemented.

[0069] Compared with the prior art, the method described in the technical solution provided in the embodiment of the present application includes: obtaining the storage performance configuration parameters, storage usage data and fault type of the all-in-one machine; creating a list of fault reliability test influencing factors according to the storage performance configuration parameters, storage usage data and fault type of the all-in-one machine; performing permutation and combination screening on the influencing factors in the list of fault reliability test influencing factors to obtain a target fault test scenario case library; performing a fault reliability test on the all-in-one machine according to the target fault test scenario case library to obtain a full fault reliability test result data set of the all-in-one machine; dividing the full fault reliability test result data set of the all-in-one machine into several fault reliability test result data subsets; obtaining a bootstrap aggregation model; performing risk prediction on each fault reliability test result data subset through the bootstrap aggregation model; averaging or voting on the risk prediction results of several fault reliability test result data subsets to obtain the prediction result of the target fault reliability test result data; determining the risk level of each fault test scenario case according to the prediction result of the target fault reliability test result data. This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0070] The technical solution provided in the embodiment of the present application screens and accurately predicts the reliability failure scenario test results through an integrated learning model, identifies high-risk reliability use cases and medium- and low-risk reliability use cases with controllable risks, and based on this, can distinguish different failure use cases running at different stages according to the project situation, and can also set test cycles, cyclic tests, etc., which can more accurately expose deep-seated problems and effectively improve product reliability.

[0071] The technical solution provided in the embodiment of the present application solves the problem of insufficient reliability caused by probabilistic exposure problems in an all-in-one machine; through more intelligent algorithm optimization and combined with the input parameters of the product reliability test, higher-risk reliability failure test points can be identified, and then in the actual project process, the corresponding test cases can be executed more accurately as needed, the testing intensity can be strengthened, the reliability problems of the product can be explored more deeply, the reliability of the product can be improved, and the accuracy, integrity and security of existing data can be protected. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0073] Figure 1 A first flow chart of the reliability testing method provided in Embodiment 1 of the present application;

[0074] Figure 2 A second flow chart of the reliability testing method provided in Embodiment 2 of the present application;

[0075] Figure 3 A specific flow chart of the reliability testing method provided in Example 2 of the present application;

[0076] Figure 4 A structural diagram of a reliability testing device provided in Example 3 of the present application;

[0077] Figure 5 The exemplary system provided for Embodiment 7 of the present application can be used to implement the various embodiments described in the present application. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0079] It should be noted that, unless the context clearly requires otherwise, words such as “include”, “including” and similar words throughout the specification and claims should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is “including but not limited to”.

[0080] Furthermore, in the description of the present application, unless otherwise specified, “plurality” means two or more.

[0081] As an emerging technology that has developed rapidly in recent years, cloud computing provides users with convenient computing power, storage and application software services by integrating and rationally utilizing computer resources distributed all over the world. The "on-demand allocation, on-volume pricing" feature of cloud computing has greatly improved the flexibility, efficiency and responsiveness of enterprises. With the popularization of the Internet and the continuous advancement of computer technology, the management and maintenance costs of data centers for enterprises and individuals are gradually increasing, and the demand for cloud computing services is also growing. Cloud computing has become an indispensable part of the daily lives of enterprises and individuals, and all-in-one machines are now the main form and carrier of cloud computing.

[0082] As a data carrier, it requires strong reliability. Currently, all-in-one products have complex architectures and involve the interaction of computing, storage, network, platform and other resources. How to better ensure reliability has become a pain point in the industry.

[0083] Embodiment 1

[0084] The present application embodiment provides a reliability testing method, such as Figure 1 As shown, the method is applied to an all-in-one machine, and the method includes:

[0085] Obtaining storage performance configuration parameters, storage usage data, and fault types of the integrated device;

[0086] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0087] Performing permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library;

[0088] Performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device;

[0089] Dividing the full fault reliability test result data set of the integrated device into a plurality of fault reliability test result data subsets;

[0090] Get the bootstrapped aggregate model;

[0091] Perform risk prediction on each failure reliability test result data subset by using the bootstrap aggregation model;

[0092] The risk prediction results of several subsets of fault reliability test result data are averaged or voted to obtain the prediction results of the target fault reliability test result data;

[0093] The risk level of each fault test scenario use case is determined according to the prediction result of the target fault reliability test result data.

[0094] Specifically, this application uses a more complete reliability design solution, fully considers the combination of various fault points, and provides a more complete fault library for injection into the test system, and the fault configuration items can be easily expanded through the configuration file; after obtaining a valid data source, the execution results are collected after the fault is injected, and the fault use cases that fail to execute to a certain third preset value are defined as high-risk fault types through an integrated learning model, and the rest are classified as medium and low fault types; then different types of faults can be injected according to project requirements to verify system reliability. The entire solution only needs to divide the configuration files such as fault points, types, and verification points according to the product, and then fault testing can be performed. It can also trigger different levels of faults according to the demands of different stages of the project, thereby improving the diversity and richness of reliability testing and protecting the accuracy, integrity, and security of user data.

[0095] Among them, the third preset value is related to the type of fault scenario use case, and is determined according to the specific fault scenario, the importance of the system and historical data; the full fault reliability test result data set includes the reliability test results of the all-in-one machine under several fault test scenario use cases, and the test results under each fault test scenario use case include several sub-test results. Any fault reliability test result data subset includes several (one or more) sub-test results of the test results under all fault test scenario use cases.

[0096] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0097] This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0098] The technical solution provided in the embodiment of the present application screens and accurately predicts the reliability failure scenario test results through an integrated learning model, identifies high-risk reliability use cases and medium- and low-risk reliability use cases with controllable risks, and based on this, can distinguish different failure use cases running at different stages according to the project situation, and can also set test cycles, cyclic tests, etc., which can more accurately expose deep-seated problems and effectively improve product reliability.

[0099] Embodiment 2

[0100] The present application embodiment provides a reliability testing method, such as Figure 2 As shown, the method is applied to an all-in-one machine, and the method includes:

[0101] Step S01, obtaining storage performance configuration parameters, storage usage data and fault type of the integrated machine;

[0102] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0103] The influencing factors in the list of influencing factors for the fault reliability test are arranged, combined and screened to obtain a target fault test scenario case library.

[0104] Step S01 also includes:

[0105] Step S011, creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine, wherein the storage performance configuration parameters include hybrid flash storage and all-flash storage, and the fault types include network failure, host failure and disk failure;

[0106] Arrange and combine the influencing factors in the fault reliability test influencing factor list to generate a fault reliability test influencing factor combination;

[0107] The combination of factors affecting the fault reliability test is screened according to the product parameters of the all-in-one device, wherein the product parameters of the all-in-one device include product deployment environment parameters, technology stack parameters, and business logic parameters;

[0108] The screened failure reliability test influencing factors are used as the target failure test scenario case library.

[0109] Specifically, the all-in-one machine uses the server's local disk to achieve high-speed storage, and uses the distributed storage mechanism to give full play to the performance of the local disk, reaching or even exceeding the performance of independent centralized storage; the all-in-one machine is deployed in the form of hybrid flash storage and all-flash storage, and also involves reliability verification of Raid cards, network cards, hard drives, HBA cards of different models and under different back pressure models.

[0110] Product storage performance configurations include hybrid flash storage and all-flash storage (all-flash storage uses solid-state drives (SSDs) or flash memory as storage media, while hybrid flash storage combines solid-state drives and hard disks (HDDs)). Different product storage performance configurations affect product reliability test results.

[0111] Storage usage of the integrated machine: for example, low resource usage of 20% (i.e., only 20% of the resources are occupied by the current workload), medium resource usage of 60%, and high resource usage of 80%; that is, different storage usage of the integrated machine will affect the product reliability test results;

[0112] Fault type: Network fault: packet loss rate, delay, network disconnection; Host fault: power failure, restart, restart service; Disk fault: unplug the disk, bad disk, bad block;

[0113] The above influencing factors are arranged and combined through scripts, and invalid combinations are deleted according to the actual product situation (product parameters) to form an injectable fault scenario use case: N = [deployment * data preparation * fault type]

[0114] Creating a list of factors affecting a fault reliability test, performing permutation and combination screening on the factors in the list of factors affecting a fault reliability test, and obtaining a target fault test scenario case library comprises the following steps:

[0115] Create a list of influencing factors:

[0116] Deployment: [Hybrid flash storage, all-flash storage]

[0117] Data preparation: [Low resource utilization 20%, Medium resource utilization 60%, High resource utilization 80%]

[0118] Fault type: [network failure, host failure, disk failure]

[0119] Permutations and combinations:

[0120] Using the permutation and combination function of a programming language to perform permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to generate all possible combinations of influencing factors;

[0121] Filter invalid combinations:

[0122] Delete those impossible or meaningless combinations of influencing factors according to the actual situation of the product;

[0123] Form target failure scenario use cases:

[0124] Combine the screened influencing factors as the target fault scenario use case that can be injected;

[0125] At the same time, after the failure is restored in time, the checkpoints of the system are fixed. The checkpoints mainly include system alarms, whether the business IO is blocked, and whether the data is consistent;

[0126] The above completes the target fault test scenario case library.

[0127] Among them, the fault test scenario case library is a systematic collection that includes a series of test cases. Each test case is designed to simulate a specific fault or abnormal situation to verify the robustness and fault recovery capability of the system.

[0128] Step S02: performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device.

[0129] Step S02 also includes:

[0130] Step S021, obtaining a list of configuration files;

[0131] Expanding the target fault test scenario case library through the configuration file list;

[0132] According to the test framework and the expanded fault test scenario case library, the fault reliability test is performed on the all-in-one device one by one.

[0133] Specifically, reliability testing is performed according to fault scenario use cases; the all-in-one machine to be tested is prepared, and fault reliability testing is performed on the all-in-one machine one by one according to the test framework and the target fault test scenario use case library; the entire automated execution test framework can be implemented with the help of mainstream automation frameworks such as pytest and testng;

[0134] In order to facilitate the expansion of fault scenarios and ensure that the fault scenarios have good horizontal and vertical scalability, a special configuration list can be set. By using variable references in the code, the fault scenarios can be effectively expanded by flexibly setting configuration file items.

[0135] Step S03, dividing the full fault reliability test result data set of the integrated machine into a plurality of fault reliability test result data subsets;

[0136] Get the bootstrapped aggregate model;

[0137] The bootstrap aggregation model is used to perform risk prediction on each failure reliability test result data subset.

[0138] Specifically, compared with a single learning model, the advantage of the integrated learning model is that it can organically combine multiple single learning models to obtain a unified integrated learning model, thereby obtaining more accurate, stable and robust results; the fundamental idea of ​​​​integrated learning is to integrate the advantages of each base learner to reduce prediction errors.

[0139] This application uses the bootstrap aggregation model in the integrated learning model to train the test results of the reliability test. Multiple basic models can be trained on random subsets of the original test result data set, and the prediction results of multiple basic models are averaged or voted to reduce the variance. Each basic model is trained on an independent subset, and these subsets are obtained through bootstrap sampling. The prediction result of the final reliability test result is based on the average or voting of the prediction results of all basic models, which can more accurately distinguish the risk priority of fault test cases.

[0140] Step S03 also includes:

[0141] Step S031, creating a test data set according to the failure reliability test result data subset;

[0142] Dividing the test data set into a test set and a training set by a data preprocessing function, wherein the test set and the training set both include feature data and label data;

[0143] Performing fitting simulation on the bootstrap aggregation model through the training set data;

[0144] Predicting the feature data of the test set by fitting the simulated bootstrap aggregation model;

[0145] The fitting simulation of the bootstrap aggregation model using the training set data includes:

[0146] Setting the number of base learners of the bootstrap aggregation model to a first preset value;

[0147] Setting a seed parameter of a random number generator of the bootstrap aggregation model to a second preset value;

[0148] The bootstrap aggregation model is fitted and simulated using the feature data and label data of the training set data.

[0149] Specifically, 1. a sampling method with replacement is used to extract n subsets from the full fault reliability test result data set of the integrated machine, and a total of K rounds of extraction are performed to obtain K subsets, and the subsets are independent of each other;

[0150] 2. Code to load a subset of the failure reliability test result data named drtest; the data subset is used for machine learning or data analysis;

[0151] drtest=datasets.load_drtest_data();

[0152] 3. Divide the data subsets into test sets and training sets:

[0153] x_data=drtest.data[:,:2]y_data=drtest.target

[0154] As you can understand, the code to extract features (x_data) and labels (y_data) from the drtest subset;

[0155] drtest.data contains all the features in the data subset, and drtest.target contains the label of the data subset;

[0156] drtest.data[:,:2]: extracts the first two features (columns) of all samples (rows) from drtest.data; these two features will be used for subsequent data processing or model training;

[0157] y_data = drtest.target: assign drtest.target to y_data, so that y_data contains all the label data in the data subset;

[0158] x_train,x_test,y_train,y_test=train_test_split(x_data,y_data)

[0159] It can be understood that the data subset is divided into training set and test set using the train_test_split function;

[0160] x_train: will contain the feature data for the training set;

[0161] x_test: will contain the feature data for the test set;

[0162] y_train: will contain the labeled data for the training set;

[0163] y_test: will contain the labeled data for the test set.

[0164] 4. For high-risk failure scenarios, create a bagging classifier (bootstrap aggregation model) and perform a fitting simulation (use the BaggingClassifier class in the scikit-learn library to create a bagging classifier and fit it using the training set)

[0165] bagging_classifier=BaggingClassifier(base_estimator=base_classifier,n_estimators=10,random_state=42)

[0166] As you can understand, the n_estimators parameter specifies the number of base learners to be used (Bagging classifiers build a more powerful model by integrating multiple base learners). Here, 10 base learners are set.

[0167] The random_state parameter is used to control the seed of the random number generator to ensure the repeatability of the results. Here, set random_state = 42;

[0168] bagging_classifier.fit(X_train,y_train)

[0169] It can be understood that the fit method is used to fit the model on the training set, x_train is the feature data of the training set, and y_train is the label data of the training set;

[0170] Among them, the first preset value is 10; the second preset value is 42;

[0171] 5. Run the bootstrap aggregation model to predict data;

[0172] y_pred=bagging_classifier.predict(X_test)

[0173] Predict the feature data X_test of the test set;

[0174] bagging_classifier is the trained Bagging classifier;

[0175] Use multiple base learners in the ensemble to predict the label of each sample in the feature data X_test of the test set.

[0176] Step S04, verifying the accuracy of the prediction results of the test set;

[0177] The checking of the accuracy of the prediction result of the test set includes:

[0178] Comparing the initial label data of the test set with the label data of the test set predicted by the bootstrap aggregation model by using a consistency evaluation function of the classification model;

[0179] In response to the initial label data of the test set being the same as the label data of the test set predicted by the bootstrap aggregation model, setting the predicted label data of the test set as the correctly predicted label sample data;

[0180] The accuracy of the test set prediction results is calculated using the formula: number of correctly predicted label sample data / number of initial label data in the test set = accuracy of the prediction results.

[0181] Specifically, accuracy=accuracy_score(y_test,y_pred)

[0182] It is understandable that the accuracy of the prediction results of the test set is calculated;

[0183] accuracy_score is a function in the scikit-learn library used to evaluate the performance of classification models. It calculates the consistency between the label data of the test set predicted by the bootstrap aggregation model and the initial label data of the test set.

[0184] The accuracy_score function will compare each element in y_test (the true label of the test set) and y_pred (the label predicted by the model), calculate the number of samples that were predicted correctly, and then divide it by the total number of true labels in y_test to get the accuracy.

[0185] Step S05, averaging or voting the risk prediction results of several fault reliability test result data subsets to obtain the prediction results of the target fault reliability test result data;

[0186] The risk level of the fault reliability test result is determined according to the prediction result of the target fault reliability test result data.

[0187] Step S051, evaluating the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data;

[0188] When the number of failures of the fault test execution according to the fault scenario is greater than or equal to a third preset value, the fault scenario use case is defined as a high-risk fault scenario type;

[0189] When the number of failures of the fault test execution according to the fault scenario is less than a third preset value, the fault scenario use case is defined as a low-risk fault scenario type;

[0190] Executing the fault scenario use case through the configuration file list;

[0191] When executing high-risk failure scenario types, the reliability test mode is set to sensitive mode;

[0192] When executing low-risk failure scenario types, the reliability test mode is set to normal mode.

[0193] Specifically, fault scenario use cases in which the number of failures of fault testing execution according to the fault scenario is greater than or equal to a third preset value are collected as high-priority fault scenario use cases, and other fault scenario use cases are defined as medium and low priority fault scenarios; by distinguishing the configuration files, the configuration files can be specified in the automation platform for convenient execution; when executing high-priority fault use cases, the reliability test mode is set to the sensitive mode; when executing low-risk fault scenario types, the reliability test mode is set to the normal mode.

[0194] Step S06, monitoring the all-in-one machine fault reliability test process through a monitoring plug-in;

[0195] The monitoring of the integrated machine fault reliability test process by the monitoring plug-in includes:

[0196] When any fault scenario test case fails, an alarm will be issued for the error result of the fault scenario test case;

[0197] Record the failure scenario use case pass rate and test log of the failure reliability test.

[0198] Specifically, the integrated machine fault reliability test process is monitored through a monitoring plug-in, and the fault scenario use case pass rate of the fault reliability test, the error results of the fault scenario test case, the test log, etc. are fully displayed.

[0199] like Figure 3As shown, the present application obtains the storage performance configuration parameters, storage usage data and fault type of the all-in-one machine; creates a list of fault reliability test influencing factors according to the storage performance configuration parameters, storage usage data and fault type of the all-in-one machine; performs permutation, combination and screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library; performs a fault reliability test on the all-in-one machine according to the target fault test scenario case library to obtain a full fault reliability test result data set of the all-in-one machine; divides the full fault reliability test result data set of the all-in-one machine into several fault reliability test result data subsets; obtains a bootstrap aggregation model; performs risk prediction on each fault reliability test result data subset through the bootstrap aggregation model; averages or votes on the risk prediction results of several fault reliability test result data subsets to obtain a prediction result of the target fault reliability test result data; determines the risk level of the fault reliability test result according to the prediction result of the target fault reliability test result data.

[0200] Through the integrated learning model, the reliability failure scenario test results are screened and accurately predicted, and high-risk reliability use cases and medium- and low-risk reliability use cases with controllable risks are identified. Based on this, different failure use cases can be distinguished by running them at different stages according to the project situation, and test cycles, cyclic tests, etc. can also be set, which can more accurately expose deep-seated problems and effectively improve product reliability.

[0201] In addition, after determining the risk level of the fault reliability test result according to the prediction result of the target fault reliability test result data, the method further comprises:

[0202] Based on the reliability use case risk level, run different levels of fault use cases at different stages according to the project situation;

[0203] Set the test cycle and number of cyclic tests according to the risk level of reliability use cases;

[0204] Based on the reliability use case risk level, different levels of fault use cases are run at different stages according to the project situation, including:

[0205] When in the early stages of a project, run high-risk reliability use cases to identify potential issues early.

[0206] When in the mid-term stage of the project, run low- and medium-risk reliability use cases and perform system-level functional and regression testing;

[0207] At the end of the project, comprehensive system testing and performance testing are performed to verify all reliability use cases;

[0208] The test cycle and the number of cyclic tests are set according to the risk level of the reliability use case, including:

[0209] Based on the risk assessment results, prioritize the testing cycles for high-risk use cases;

[0210] Low- and medium-risk use cases are scheduled in the later stages of the test cycle for loop testing and regression testing to ensure the compatibility of new functions while ensuring the overall stability of the system.

[0211] Set more frequent testing cycles for high-risk test cases, and extend the testing intervals for medium- and low-risk cases.

[0212] More testing should be done on high-risk functions in the early stages of the test cycle to identify and fix problems in a timely manner. The frequency of cyclic testing can be adjusted based on the risk level of the use case. This can more accurately expose deep-seated problems, effectively improve the reliability of the product, and ensure that testing at different stages can address potential risks in a targeted manner.

[0213] The reliability testing method provided in the embodiment of the present application may be further improved and optimized without departing from the technical solution of the present application, and these improvements and optimizations should also be regarded as within the scope of protection of the present application.

[0214] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0215] This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0216] The technical solution provided in the embodiment of the present application screens and accurately predicts the reliability failure scenario test results through an integrated learning model, identifies high-risk reliability use cases and medium- and low-risk reliability use cases with controllable risks, and based on this, can distinguish different failure use cases running at different stages according to the project situation, and can also set test cycles, cyclic tests, etc., which can more accurately expose deep-seated problems and effectively improve product reliability.

[0217] The technical solution provided in the embodiment of the present application solves the problem of insufficient reliability caused by probabilistic exposure problems in an all-in-one machine; through more intelligent algorithm optimization and combined with the input parameters of the product reliability test, higher-risk reliability failure test points can be identified, and then in the actual project process, the corresponding test cases can be executed more accurately as needed, the testing intensity can be strengthened, the reliability problems of the product can be explored more deeply, the reliability of the product can be improved, and the accuracy, integrity and security of existing data can be protected.

[0218] Embodiment 3

[0219] The present application provides a reliability testing device, such as Figure 4 As shown, the device includes: an acquisition module, a creation module, a screening module, a testing module, a division module, a prediction module, a verification module, a determination module, and a monitoring module.

[0220] In this embodiment, the acquisition module is used to obtain storage performance configuration parameters, storage usage data and fault types of the integrated machine; obtain a bootstrap aggregation model;

[0221] A creation module, used to create a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0222] A screening module, used for performing permutation and combination screening on the influencing factors in the list of influencing factors of the fault reliability test to obtain a target fault test scenario case library;

[0223] A testing module, used to perform a fault reliability test on the integrated device according to the target fault test scenario case library, and obtain a full fault reliability test result data set of the integrated device;

[0224] A partitioning module, used to partition the full fault reliability test result data set of the integrated machine into a plurality of fault reliability test result data subsets;

[0225] A prediction module, used to perform risk prediction on each failure reliability test result data subset through a bootstrap aggregation model;

[0226] A determination module is used to average or vote on the risk prediction results of several subsets of fault reliability test result data to obtain the prediction results of the target fault reliability test result data; and determine the risk level of each fault test scenario use case based on the prediction results of the target fault reliability test result data.

[0227] In one embodiment, a screening module is used to create a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine, wherein the storage performance configuration parameters include hybrid flash storage and all-flash storage, and the fault types include network failure, host failure and disk failure;

[0228] Arrange and combine the influencing factors in the fault reliability test influencing factor list to generate a fault reliability test influencing factor combination;

[0229] The combination of factors affecting the fault reliability test is screened according to the product parameters of the all-in-one device, wherein the product parameters of the all-in-one device include product deployment environment parameters, technology stack parameters, and business logic parameters;

[0230] The screened failure reliability test influencing factors are used as the target failure test scenario case library.

[0231] In one of the embodiments, the testing module is used to obtain a list of configuration files;

[0232] Expanding the target fault test scenario case library through the configuration file list;

[0233] According to the test framework and the expanded fault test scenario case library, the fault reliability test is performed on the all-in-one device one by one.

[0234] In one of the embodiments, a prediction module is used to create a test data set based on the failure reliability test result data subset;

[0235] Dividing the test data set into a test set and a training set by a data preprocessing function, wherein the test set and the training set both include feature data and label data;

[0236] Performing fitting simulation on the bootstrap aggregation model through the training set data;

[0237] Predicting the feature data of the test set by fitting the simulated bootstrap aggregation model;

[0238] The fitting simulation of the bootstrap aggregation model using the training set data includes:

[0239] Setting the number of base learners of the bootstrap aggregation model to a first preset value;

[0240] Setting a seed parameter of a random number generator of the bootstrap aggregation model to a second preset value;

[0241] The bootstrap aggregation model is fitted and simulated using the feature data and label data of the training set data.

[0242] In one of the embodiments, a verification module is used to verify the accuracy of the prediction results of the test set;

[0243] The checking of the accuracy of the prediction result of the test set includes:

[0244] Comparing the initial label data of the test set with the label data of the test set predicted by the bootstrap aggregation model by using a consistency evaluation function of the classification model;

[0245] In response to the initial label data of the test set being the same as the label data of the test set predicted by the bootstrap aggregation model, setting the predicted label data of the test set as the correctly predicted label sample data;

[0246] The accuracy of the test set prediction results is calculated using the formula: the number of correctly predicted label sample data / the number of initial label data in the test set = the accuracy of the prediction results.

[0247] In one of the embodiments, a determination module is used to evaluate the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data;

[0248] When the number of failures of the fault test execution according to the fault scenario is greater than or equal to a third preset value, the fault scenario use case is defined as a high-risk fault scenario type;

[0249] When the number of failures of the fault test execution according to the fault scenario is less than a third preset value, the fault scenario use case is defined as a low-risk fault scenario type;

[0250] Executing the fault scenario use case through the configuration file list;

[0251] When executing high-risk failure scenario types, the reliability test mode is set to sensitive mode;

[0252] When executing low-risk failure scenario types, the reliability test mode is set to normal mode.

[0253] In one of the embodiments, a monitoring module is used to monitor the fault reliability test process of the integrated machine through a monitoring plug-in;

[0254] The monitoring of the integrated machine fault reliability test process by the monitoring plug-in includes:

[0255] When any fault scenario test case fails, an alarm will be issued for the error result of the fault scenario test case;

[0256] Record the failure scenario use case pass rate and test log of the failure reliability test.

[0257] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0258] This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0259] The technical solution provided in the embodiment of the present application screens and accurately predicts the reliability failure scenario test results through an integrated learning model, identifies high-risk reliability use cases and medium- and low-risk reliability use cases with controllable risks, and based on this, can distinguish different failure use cases running at different stages according to the project situation, and can also set test cycles, cyclic tests, etc., which can more accurately expose deep-seated problems and effectively improve product reliability.

[0260] The technical solution provided in the embodiment of the present application solves the problem of insufficient reliability caused by probabilistic exposure problems in an all-in-one machine; through more intelligent algorithm optimization and combined with the input parameters of the product reliability test, higher-risk reliability failure test points can be identified, and then in the actual project process, the corresponding test cases can be executed more accurately as needed, the testing intensity can be strengthened, the reliability problems of the product can be explored more deeply, the reliability of the product can be improved, and the accuracy, integrity and security of existing data can be protected.

[0261] Embodiment 4

[0262] The present invention also provides a reliability testing device, comprising:

[0263] one or more processors;

[0264] A storage device for storing one or more programs;

[0265] When the one or more programs are executed by the one or more processors, the one or more processors execute the reliability testing method described below:

[0266] Obtaining storage performance configuration parameters, storage usage data, and fault types of the integrated device;

[0267] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0268] Performing permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library;

[0269] Performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device;

[0270] Dividing the full fault reliability test result data set of the integrated device into a plurality of fault reliability test result data subsets;

[0271] Get the bootstrapped aggregate model;

[0272] Perform risk prediction on each failure reliability test result data subset by using the bootstrap aggregation model;

[0273] The risk prediction results of several subsets of fault reliability test result data are averaged or voted to obtain the prediction results of the target fault reliability test result data;

[0274] The risk level of each fault test scenario use case is determined according to the prediction result of the target fault reliability test result data.

[0275] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0276] This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0277] The technical solution provided in the embodiment of the present application screens and accurately predicts the reliability failure scenario test results through an integrated learning model, identifies high-risk reliability use cases and medium- and low-risk reliability use cases with controllable risks, and based on this, can distinguish different failure use cases running at different stages according to the project situation, and can also set test cycles, cyclic tests, etc., which can more accurately expose deep-seated problems and effectively improve product reliability.

[0278] Embodiment 5

[0279] The present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following reliability test method can be executed:

[0280] Obtaining storage performance configuration parameters, storage usage data, and fault types of the integrated device;

[0281] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0282] Performing permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library;

[0283] Performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device;

[0284] Dividing the full fault reliability test result data set of the integrated device into a plurality of fault reliability test result data subsets;

[0285] Get the bootstrapped aggregate model;

[0286] Perform risk prediction on each failure reliability test result data subset by using the bootstrap aggregation model;

[0287] The risk prediction results of several subsets of fault reliability test result data are averaged or voted to obtain the prediction results of the target fault reliability test result data;

[0288] The risk level of each fault test scenario use case is determined according to the prediction result of the target fault reliability test result data.

[0289] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0290] This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0291] The technical solution provided in the embodiment of the present application solves the problem of insufficient reliability caused by probabilistic exposure problems in an all-in-one machine; through more intelligent algorithm optimization and combined with the input parameters of the product reliability test, higher-risk reliability failure test points can be identified, and then in the actual project process, the corresponding test cases can be executed more accurately as needed, the testing intensity can be strengthened, the reliability problems of the product can be explored more deeply, the reliability of the product can be improved, and the accuracy, integrity and security of existing data can be protected.

[0292] Embodiment 6

[0293] The present application also provides a computer program product, including a computer program, which can implement the following reliability testing method when executed by a processor:

[0294] Obtaining storage performance configuration parameters, storage usage data, and fault types of the integrated device;

[0295] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0296] Performing permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library;

[0297] Performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device;

[0298] Dividing the full fault reliability test result data set of the integrated device into a plurality of fault reliability test result data subsets;

[0299] Get the bootstrapped aggregate model;

[0300] Perform risk prediction on each failure reliability test result data subset by using the bootstrap aggregation model;

[0301] The risk prediction results of several subsets of fault reliability test result data are averaged or voted to obtain the prediction results of the target fault reliability test result data;

[0302] The risk level of each fault test scenario use case is determined according to the prediction result of the target fault reliability test result data.

[0303] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0304] This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0305] The technical solution provided in the embodiment of the present application screens and accurately predicts the reliability failure scenario test results through an integrated learning model, identifies high-risk reliability use cases and medium- and low-risk reliability use cases with controllable risks, and based on this, can distinguish different failure use cases running at different stages according to the project situation, and can also set test cycles, cyclic tests, etc., which can more accurately expose deep-seated problems and effectively improve product reliability.

[0306] Embodiment 7

[0307] The present application provides a computer storage medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0308] Obtaining storage performance configuration parameters, storage usage data, and fault types of the integrated device;

[0309] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine;

[0310] Performing permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library;

[0311] Performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device;

[0312] Dividing the full fault reliability test result data set of the integrated device into a plurality of fault reliability test result data subsets;

[0313] Get the bootstrapped aggregate model;

[0314] Perform risk prediction on each failure reliability test result data subset by using the bootstrap aggregation model;

[0315] The risk prediction results of several subsets of fault reliability test result data are averaged or voted to obtain the prediction results of the target fault reliability test result data;

[0316] The risk level of each fault test scenario use case is determined according to the prediction result of the target fault reliability test result data.

[0317] In one of the embodiments, creating a list of factors affecting a fault reliability test according to the storage performance configuration parameters, storage usage data, and fault types of the integrated machine, performing permutation and combination screening on the factors affecting the list of factors affecting the fault reliability test, and obtaining a target fault test scenario case library includes:

[0318] Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data, and fault types of the integrated machine, wherein the storage performance configuration parameters include hybrid flash storage and all-flash storage, and the fault types include network failure, host failure, and disk failure;

[0319] Arrange and combine the influencing factors in the fault reliability test influencing factor list to generate a fault reliability test influencing factor combination;

[0320] The combination of factors affecting the fault reliability test is screened according to the product parameters of the all-in-one device, wherein the product parameters of the all-in-one device include product deployment environment parameters, technology stack parameters, and business logic parameters;

[0321] The screened failure reliability test influencing factors are used as the target failure test scenario case library.

[0322] In one embodiment, the performing a fault reliability test on the integrated device according to the target fault test scenario case library includes:

[0323] Get the configuration file list;

[0324] Expanding the target fault test scenario case library through the configuration file list;

[0325] According to the test framework and the expanded fault test scenario case library, the fault reliability test is performed on the all-in-one device one by one.

[0326] In one of the embodiments, the step of performing risk prediction on each failure reliability test result data subset by using the bootstrap aggregation model includes:

[0327] Creating a test data set according to the failure reliability test result data subset;

[0328] Dividing the test data set into a test set and a training set by a data preprocessing function, wherein the test set and the training set both include feature data and label data;

[0329] Performing fitting simulation on the bootstrap aggregation model through the training set data;

[0330] Predicting the feature data of the test set by fitting the simulated bootstrap aggregation model;

[0331] The fitting simulation of the bootstrap aggregation model using the training set data includes:

[0332] Setting the number of base learners of the bootstrap aggregation model to a first preset value;

[0333] Setting a seed parameter of a random number generator of the bootstrap aggregation model to a second preset value;

[0334] The bootstrap aggregation model is fitted and simulated using the feature data and label data of the training set data.

[0335] In one embodiment, after predicting the feature data of the test set by the bootstrap aggregation model completed by fitting simulation, the method includes:

[0336] Verifying the accuracy of the prediction results of the test set;

[0337] The checking of the accuracy of the prediction result of the test set includes:

[0338] Comparing the initial label data of the test set with the label data of the test set predicted by the bootstrap aggregation model by using a consistency evaluation function of the classification model;

[0339] In response to the initial label data of the test set being the same as the label data of the test set predicted by the bootstrap aggregation model, setting the predicted label data of the test set as the correctly predicted label sample data;

[0340] The accuracy of the test set prediction results is calculated using the formula: number of correctly predicted label sample data / number of initial label data in the test set = accuracy of the prediction results.

[0341] In one embodiment, determining the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data includes:

[0342] Evaluate the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data;

[0343] When the number of failures of the fault test execution according to the fault scenario is greater than or equal to a third preset value, the fault scenario use case is defined as a high-risk fault scenario type;

[0344] When the number of failures of the fault test execution according to the fault scenario is less than a third preset value, the fault scenario use case is defined as a low-risk fault scenario type;

[0345] Executing the fault scenario use case through the configuration file list;

[0346] When executing high-risk failure scenario types, the reliability test mode is set to sensitive mode;

[0347] When executing low-risk failure scenario types, the reliability test mode is set to normal mode.

[0348] In one embodiment, after determining the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data, the method includes:

[0349] Monitoring the fault reliability test process of the integrated device through a monitoring plug-in;

[0350] The monitoring of the integrated machine fault reliability test process by the monitoring plug-in includes:

[0351] When any fault scenario test case fails, an alarm will be issued for the error result of the fault scenario test case;

[0352] Record the failure scenario use case pass rate and test log of the failure reliability test.

[0353] This application can execute corresponding scenario test cases on demand, strengthen testing efforts, dig deeper into product reliability issues, improve product reliability, and thereby improve data accuracy, integrity, and security.

[0354] Figure 5 An exemplary system provided for Embodiment 7 of the present application that can be used to implement various embodiments described in the present application;

[0355] like Figure 5As shown, in some embodiments, the system can be used as any of the above-mentioned devices for reliability testing in each of the described embodiments. In some embodiments, the system may include one or more computer-readable media (e.g., system memory or NVM / storage device) having results and one or more processors (e.g., (one or more) processors) coupled to the one or more computer-readable media and configured to execute the results to implement the module to perform the actions described in the present application.

[0356] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0357] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0358] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A reliability testing method, characterized in that: The method is applied to an all-in-one machine, and the method comprises: Obtaining storage performance configuration parameters, storage usage data, and fault types of the integrated device; Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine; Performing permutation and combination screening on the influencing factors in the fault reliability test influencing factor list to obtain a target fault test scenario case library; Performing a fault reliability test on the integrated device according to the target fault test scenario case library to obtain a full fault reliability test result data set of the integrated device; Dividing the full fault reliability test result data set of the integrated machine into a plurality of fault reliability test result data subsets; Get the ensemble learning model; Perform risk prediction on each failure reliability test result data subset by using the integrated learning model; The risk prediction results of several subsets of fault reliability test result data are averaged or voted to obtain the prediction results of the target fault reliability test result data; The risk level of each fault test scenario use case is determined according to the prediction result of the target fault reliability test result data.

2. The reliability testing method according to claim 1, characterized in that: The method of creating a list of factors affecting the reliability test of a fault according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine, and performing permutation and combination screening on the factors affecting the reliability test of a fault to obtain a target fault test scenario case library includes: Creating a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data, and fault types of the integrated machine, wherein the storage performance configuration parameters include hybrid flash storage and all-flash storage, and the fault types include network failure, host failure, and disk failure; Arrange and combine the influencing factors in the fault reliability test influencing factor list to generate a fault reliability test influencing factor combination; The combination of factors affecting the fault reliability test is screened according to the product parameters of the all-in-one device, wherein the product parameters of the all-in-one device include product deployment environment parameters, technology stack parameters, and business logic parameters; The screened failure reliability test influencing factors are used as the target failure test scenario case library.

3. The reliability testing method according to claim 1, characterized in that: The performing a fault reliability test on the integrated device according to the target fault test scenario case library includes: Get the configuration file list; Expanding the target fault test scenario case library through the configuration file list; According to the test framework and the expanded fault test scenario case library, the fault reliability test is performed on the all-in-one device one by one.

4. The reliability testing method according to claim 1, characterized in that: The risk prediction of each failure reliability test result data subset by the integrated learning model includes: Creating a test data set according to the failure reliability test result data subset; Dividing the test data set into a test set and a training set by a data preprocessing function, wherein the test set and the training set both include feature data and label data; Fitting and simulating the ensemble learning model through the training set data; wherein the ensemble learning model is a bootstrap aggregation model; Predicting the feature data of the test set by fitting the simulated bootstrap aggregation model; The fitting simulation of the bootstrap aggregation model using the training set data includes: Setting the number of base learners of the bootstrap aggregation model to a first preset value; Setting a seed parameter of a random number generator of the bootstrap aggregation model to a second preset value; The bootstrap aggregation model is fitted and simulated using the feature data and label data of the training set data.

5. The data error location method according to claim 4, characterized in that: After the bootstrap aggregation model completed by fitting simulation predicts the feature data of the test set, it includes: Verifying the accuracy of the prediction results of the test set; The checking of the accuracy of the prediction result of the test set includes: Comparing the initial label data of the test set with the label data of the test set predicted by the bootstrap aggregation model by using a consistency evaluation function of the classification model; In response to the initial label data of the test set being the same as the label data of the test set predicted by the bootstrap aggregation model, setting the predicted label data of the test set as the correctly predicted label sample data; The accuracy of the test set prediction results is calculated using the formula: number of correctly predicted label sample data / number of initial label data in the test set = accuracy of the prediction results.

6. The reliability testing method according to claim 3, characterized in that: Determining the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data includes: Evaluate the risk level of each fault test scenario use case according to the prediction result of the target fault reliability test result data; When the number of failures of the fault test execution according to the fault scenario is greater than or equal to a third preset value, the fault scenario use case is defined as a high-risk fault scenario use case; When the number of failures of the fault test execution according to the fault scenario is less than a third preset value, the fault scenario use case is defined as a low-risk fault scenario use case; Executing the fault scenario use case through the configuration file list; When executing high-risk failure scenario use cases, the reliability test mode is set to sensitive mode; When executing low-risk failure scenario use cases, the reliability test mode is set to normal mode.

7. The reliability testing method according to claim 1, characterized in that: After determining the risk level of each fault test scenario use case according to the target fault reliability test result data prediction result, the method includes: Monitoring the fault reliability test process of the integrated device through a monitoring plug-in; The monitoring of the integrated machine fault reliability test process by the monitoring plug-in includes: When any fault scenario test case fails, an alarm will be issued for the error result of the fault scenario test case; Record the failure scenario use case pass rate and test log of the failure reliability test.

8. A reliability testing device, characterized in that: The device is applied to an all-in-one machine, and the device comprises: An acquisition module, used to acquire storage performance configuration parameters, storage usage data and fault types of the integrated machine; and to acquire an integrated learning model; A creation module, used to create a list of factors affecting the fault reliability test according to the storage performance configuration parameters, storage usage data and fault types of the integrated machine; A screening module, used for performing permutation and combination screening on the influencing factors in the list of influencing factors of the fault reliability test to obtain a target fault test scenario case library; A testing module, used to perform a fault reliability test on the integrated device according to the target fault test scenario case library, and obtain a full fault reliability test result data set of the integrated device; A partitioning module, used to partition the full fault reliability test result data set of the integrated machine into a plurality of fault reliability test result data subsets; A prediction module is used to perform risk prediction on each failure reliability test result data subset through an integrated learning model; A determination module is used to average or vote on the risk prediction results of several subsets of fault reliability test result data to obtain the prediction results of the target fault reliability test result data; and determine the risk level of each fault test scenario use case based on the prediction results of the target fault reliability test result data.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the reliability testing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the reliability testing method according to any one of claims 1 to 7 are implemented.

Images