An automated test and data management system and method

Abstract

The application discloses an automatic test and data management system and method, and the system comprises an automatic test module, a standardized basic test resource library built-in, which is used for customizing configuration test object information, executing a test script input by a user, and performing real-time progress monitoring and interactive control in a test process; test version information, test object interface information and test configuration requirement information are supported to be inputted in a customized manner, so that test organization personnel information, a measured device IP address, a measured device username and a login password, a detection expected requirement result, customized configuration requirement index key core configuration information can be updated and modified; a data management module is used for recording basic data and displaying the basic data through a page, dynamically displaying a test process and main information in real time, filtering and analyzing collected data, and recording and displaying all test logs in full amount. The application can improve the standardization and automation degree of test and evaluation work.

Description

Technical Field

[0001] This invention relates to the field of computer hardware testing, and more particularly to an automated testing and data management system and method. Background Technology

[0002] With the rapid development of information technology in China, various CPUs, components and complete machines based on innovative technology architectures have been rapidly developed and gradually put into daily use. To date, there are dozens of categories and hundreds of models of products, including servers, desktop computers, portable computers and all-in-one machines. The huge product system has put forward higher standards and requirements for the selection, testing and evaluation of related products, factory testing and inspection, and management and maintenance of test data.

[0003] Currently, most IT products still rely primarily on manual testing methods for component selection, prototype verification, and trial production testing. This often results in low efficiency, significant subjective influence, difficulty in process control, and low usability of results. Furthermore, most testing tools on the market are developed for the Win-Tel architecture and focus on single-component or single-function testing (e.g., fio only tests the hard drive's core performance, spec CPU only tests the CPU's performance). When testing devices using innovative technologies such as C86 and Arm, they often become unusable or malfunction, and cannot effectively support the system-wide testing needs of servers, desktops, and other products. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide an automated testing and data management system and method to solve the problems of inconsistent standardization, low efficiency, poor accuracy, limited functionality, and insufficient compatibility in traditional manual testing.

[0005] Technical solution: The automated testing and data management system of the present invention includes:

[0006] The automated testing module has a built-in standardized basic test resource library for customizing test object information, executing user-defined test scripts, and performing real-time progress monitoring and interactive control during the testing process. It supports custom input of test version information, test object interface information, and test configuration requirements to update and modify key core configuration information such as test organization personnel information, IP address of the device under test, username and login password of the device under test, expected test results, and customized configuration requirements.

[0007] The data management module is used to record basic data and display it on the page, dynamically showing the test progress and key information in real time. It filters and analyzes the collected data and records and displays all test logs in full.

[0008] Furthermore, the basic test resource library includes a hardware information acquisition test library, a system stress test library, and a performance test library.

[0009] Furthermore, the hardware information acquisition and testing library includes information acquisition and comparison tests for motherboards, BIOS, CPUs, memory, hard drives, network cards, graphics cards, sound cards, and power supplies.

[0010] Furthermore, the system stress testing library includes CPU stress testing, memory testing, and restart stability testing.

[0011] Furthermore, the performance testing library includes multiple tests for CPU, memory, hard drive, and operating system benchmarks.

[0012] Furthermore, the basic data includes: test process status, parameter and command input and execution records for each test, result compliance comparison, test category, and test result determination.

[0013] Furthermore, the filtering analysis includes: retrieving and analyzing the core indicators in the test results through keyword retrieval, automatically removing invalid data, retaining key data, comparing it with the corresponding test item's required indicator parameters, providing a result judgment, and highlighting the processed key test results on the page using different colors.

[0014] Furthermore, the test log includes timestamps, input command information, system return information, test execution results, and error messages.

[0015] Furthermore, the information displayed on the page includes a summary of overall test information, test data statistics, test item details, and log details.

[0016] Test data statistics include full statistics, statistics categorized by tag, and statistics categorized by selected test suite;

[0017] The test item details view supports full details viewing, viewing by test tag, viewing by test suite, and custom search; custom search includes suite name search, test item name search, search including tag keywords, and search excluding tag keywords.

[0018] An automated testing and data management method, comprising:

[0019] Customize test object information, execute user-defined test scripts, generate data during the test process, and perform real-time progress monitoring and interactive control during the test;

[0020] Collect data during the testing process, filter and analyze the collected data, compare compliance, and record and display all test logs.

[0021] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. This design achieves standardization and automation of the testing process, greatly reducing manual intervention, minimizing the impact of human error on test results, improving testing efficiency, objectivity, and accuracy, and enhancing the efficiency of evaluation work; 2. Through interactive and visual operation design, users can monitor, pause, continue, and terminate the testing process at any time during the testing process, and it provides step-by-step functionality, allowing users to flexibly adjust the test plan according to actual conditions, improving the flexibility of tool use; 3. By automatically collecting multiple key data, including basic data and test data, it provides rich data for test result comparison and analysis. Complete original information; at the same time, by recording all original data, the entire lifecycle of the test is traceable, effectively reducing dirty data pollution; 4. Through the built-in data filtering and analysis function, invalid data can be automatically removed, key data can be retained, and test results can be displayed in an intuitive way, improving the referenceability and usability of data; 5. Through the automated execution and implementation of this tool, the problems of low efficiency and high error rate caused by manual input of commands are effectively solved. According to actual testing, taking the "processor-information consistency" single test as an example, the manual test time of 32 seconds can be reduced to 3 seconds, the test time of manual test-related tests is reduced by more than 90%, and the overall test time is reduced by 42%. Attached Figure Description

[0022] Figure 1 This is a diagram illustrating the system functions and framework design architecture described in this invention.

[0023] Figure 2 This is a schematic diagram of the automated testing process of the present invention;

[0024] Figure 3 This is a visual interface diagram of the automated testing module of the present invention;

[0025] Figure 4 This is a visualization of the test result report interface of the data management module of the present invention. Detailed Implementation

[0026] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0027] Example 1

[0028] See Figure 1 The automated testing and data management system of the present invention includes an automated testing module and a data management module;

[0029] The automated testing module has a built-in standardized basic test resource library for customizing test object information, executing user-defined test scripts, and performing real-time progress monitoring and interactive control during the testing process. It supports custom input of test version information, test object interface information, and test configuration requirements to update and modify key core configuration information such as test organization personnel information, IP address of the device under test, username and login password of the device under test, expected test results, and customized configuration requirements.

[0030] The basic test resource library includes a hardware information acquisition test library, a system stress test library, and a performance test library.

[0031] The hardware information acquisition and testing library includes information acquisition and comparison tests for motherboards, BIOS, CPUs, memory, hard drives, network cards, graphics cards, sound cards, and power supplies.

[0032] The system stress testing library includes CPU stress testing, memory testing, and reboot stability testing.

[0033] The performance testing library includes multiple tests for CPU, memory, hard drive, and operating system benchmarks.

[0034] The automated testing module supports test information configuration, including configuration of basic platform information and information related to the device under test. Regarding basic platform information, it supports editing and modifying platform version information, test tag information, etc., and allows selection of test dependency libraries. For information related to the device under test, it supports inputting and editing connection information such as IP address, connection port, device password, and passphrase. Regarding configuration requirements for the device under test, it supports editing and modifying key core configuration information such as expected test results and customized configuration requirements. Through the test information configuration, editing, and modification functions, the module meets the customized configuration modification needs of different tests, greatly improving the versatility and flexibility of the testing platform.

[0035] The automated testing module features customizable test item selection and customizable test script editing. It supports the full or customizable selection and execution of built-in standardized test scripts as needed, or the execution of user-defined test scripts through dynamic user input.

[0036] The automated testing module supports real-time display and viewing of console logs and message logs during the testing process, enabling full monitoring of the testing process and timely detection of potential problems. The module interface provides interactive buttons for stopping, pausing, continuing, moving to the next step, and skipping steps. This module has good environmental compatibility and can run stably on various CPU architectures such as X86, C86, and ARM, as well as various Linux-like operating systems such as CentOS, Red Hat, UnionTech, and Kylin.

[0037] The data management module is used to record basic data and display it dynamically on the page in real time, showing the test progress and key information. It filters and analyzes the collected data and, when automatically collecting test data, records all script inputs, system returns, and status prompts during the test process.

[0038] The basic data includes: test process status (test start time, test status, test end time, test results, etc.), parameter and command input and execution records for each test (including timestamps and execution log records), result compliance comparison (expected results and actual test data), test category and test result judgment.

[0039] The filtering analysis includes: retrieving and analyzing the core indicators in the test results through keyword search, automatically removing invalid data, retaining key data, comparing it with the corresponding test item requirement parameters (default or user-defined), providing a result judgment, and highlighting the processed key test results on the page using different colors.

[0040] The test log includes timestamps, input command information, system return information, test execution results, and error information.

[0041] The page displays information including a summary of overall test information, test data statistics, test item details, and log details.

[0042] Test data statistics include full statistics, statistics categorized by tag, and statistics categorized by selected test suite;

[0043] Test item details viewing supports full details viewing, viewing by test tag, viewing by test suite, and custom search; custom search includes suite name search, test item name search, search including tag keywords, and search excluding tag keywords, displaying the corresponding test item name, overview, tag, results, information messages, time taken, and start / end time information; log details viewing allows you to jump to the log display page by clicking the button, displaying overall test information, a complete record of process script input and output, start time, end time, time taken, test result evaluation criteria, result comparison, and result status information.

[0044] The data management module's page display function can clearly monitor the entire test lifecycle and each test execution, minimizing the risk of fatal errors.

[0045] The automated testing and data management method of the present invention includes:

[0046] Customize test object information, execute user-defined test scripts, generate data during the test process, and perform real-time progress monitoring and interactive control during the test;

[0047] Collect data during the testing process, filter and analyze the collected data, compare compliance, and record and display all test logs.

[0048] The filtering analysis includes: retrieving and analyzing the core indicators in the test results through keyword search, automatically removing invalid data, retaining key data, comparing it with the corresponding test item requirement parameters (default or user-defined), providing a result judgment, and highlighting the processed key test results on the page using different colors.

[0049] Example 2

[0050] See Figure 2 The following are the specific implementation steps of the automated testing and data management system of the present invention:

[0051] Step 1: Start the software;

[0052] Automated testing module: Users open the test client's Linux operating system, locate the software installation location, launch the software, and enter the automated testing module. (See below for details.) Figure 3 ;

[0053] Step 2: Configure key parameters;

[0054] In the automated testing module interface, open or create a test suite as needed and enter it. In the "Editor" interface, customize or use the default test version information, and import or select the required test resource library.

[0055] According to the actual situation of the device under test, fill in the connection information and configuration requirements of the device under test, such as the IP address of the device under test, connection port, password of the device under test, login password of the device under test, expected test results, customized configuration requirements and other key core configuration information;

[0056] Customization is achieved by configuring, editing, and modifying test information to meet different testing needs.

[0057] Step 3: Test item selection and test execution;

[0058] Based on the selection of the test resource library in step two, select test items as needed, either in full or in a custom format.

[0059] After selecting the test items, in the "Run" interface of the automated testing module, complete the selection of the test configuration file, filter configuration, log output configuration, etc., and click the "Start" button to execute the test.

[0060] Step 4: Passing key parameters;

[0061] Based on the parameters passed in steps one through three, the tool automatically acquires and passes the parameters, executes the corresponding test scripts in the test library one by one according to the selected test items, records the entire script execution process, and completes the acquisition, annotation, and saving of test tags, judgment conditions, and judgment information.

[0062] Step 5: Data Recording and Intelligent Analysis;

[0063] During testing, basic data is recorded in real time, including test progress status (test start time, test status, test end time, test results, etc.), parameter and command input and execution records for each test (including timestamps and execution logs), result compliance comparison (expected results, actual test data), test category, and test result determination. The test progress and key information are dynamically displayed on a webpage in real time. This feature allows for clear monitoring of the entire test lifecycle and each test execution, minimizing the risk of fatal errors.

[0064] Meanwhile, the collected data is filtered and analyzed during the process. The core indicators in the test results are analyzed by keyword retrieval, invalid data is automatically removed, key data is retained, and the data is compared with the corresponding test item requirements parameters (default or user-defined). The results are judged, and the processed key test results are highlighted on the page using different colors.

[0065] Step Six: View the results data;

[0066] After the test is completed, the data management module can be accessed and navigated to via the "Report" button on the "Run" interface of the automated test module.

[0067] The summary information displays key details such as test results, platform version, test start time, test end time, test duration, and log files. Different colors are used on the data display page to differentiate and highlight the test results. (See also...) Figure 4 .

[0068] Step 7: Click on the specific test item in "Test Data Statistics" on the data management module page as needed to jump to the test summary information display for that item. Click on the item again to jump to the detailed information of that test and view the entire process information record of that test. Click the upper right corner to return to the homepage of the data management module.

[0069] In terms of test data statistics, it includes full statistics, statistics by tag category, and statistics by selected test suite. Full statistics include the total number of tests, the number of passed tests, the number of failed tests, the number of skipped tests, the test duration, and the pass / fail / skip percentage. Statistics by tag category include the total number of tests corresponding to each tag, the number of passed tests corresponding to each tag category, the number of failed tests corresponding to each tag category, the number of skipped tests corresponding to each tag category, the test duration corresponding to each tag category, and the pass / fail / skip percentage for each tag category. Statistics by selected test suite include the total number of tests, the number of passed tests, the number of failed tests, the number of skipped tests, the test duration, and the pass / fail / skip percentage.

[0070] Step 8: View test details;

[0071] The "Please view test items" section on the data management module page supports viewing full details, selecting and viewing by test tag, selecting and viewing by test suite, and custom search.

[0072] The full details display shows the corresponding test item name, overview, tag, result, information message, duration, and start / end time information;

[0073] Select a test tag from the drop-down menu to view the overall overview of the test for that tag, the name of each test item for that tag, the overview of each test for that tag, the test tag to which each test belongs, the test results for each test for that tag, the test messages for each test for that tag, the test duration for each test for that tag, and the start / end time for each test for that tag.

[0074] Select the test suite by clicking the drop-down box. The core display includes the overall test overview, the name of each test item, the overview of each test, the test tag to which each test belongs, the test results, the corresponding test messages, the test duration, and the start / end time of each test.

[0075] Custom search allows users to search and display test data by selecting the suite name, test item name, search including tags, or search excluding tags. The core display includes the overall test overview, test item name, test summary, test tags, test results, corresponding test messages, test duration, and start / end time.

[0076] To verify the effectiveness of the present invention, comparative experiments were conducted, and the experimental results are shown in Table 1.

[0077] Table 1 Comparative Experiments

[0078] As shown in Table 1, the automated execution and implementation method of the present invention effectively solves the problems of low efficiency and high error rate caused by manual input of commands. Actual testing has verified that, taking the "processor-information consistency" single test as an example, the manual test time of 32 seconds can be reduced to 3 seconds, the test time of manual test-related tests is reduced by more than 90%, and the overall test time is reduced by 42%.

Claims

1. An automated testing and data management system, characterized in that, include: The automated testing module has a built-in standardized basic test resource library for customizing test object information, executing user-defined test scripts, and performing real-time progress monitoring and interactive control during the testing process. It supports custom input of test version information, test object interface information, and test configuration requirements to update and modify key core configuration information such as test organization personnel information, IP address of the device under test, username and login password of the device under test, expected test results, and customized configuration requirements. The data management module is used to record basic data and display it on the page, dynamically showing the test progress and key information in real time. It filters and analyzes the collected data and records and displays all test logs in full.

2. The automated testing and data management system according to claim 1, characterized in that, The basic test resource library includes a hardware information acquisition test library, a system stress test library, and a performance test library.

3. The automated testing and data management system according to claim 3, characterized in that, The hardware information acquisition and testing library includes information acquisition and comparison tests for motherboards, BIOS, CPUs, memory, hard drives, network cards, graphics cards, sound cards, and power supplies.

4. The automated testing and data management system according to claim 3, characterized in that, The system stress testing library includes CPU stress testing, memory testing, and reboot stability testing.

5. The automated testing and data management system according to claim 3, characterized in that, The performance testing library includes multiple tests for CPU, memory, hard drive, and operating system benchmarks.

6. The automated testing and data management system according to claim 1, characterized in that, The basic data includes: test process status, parameters and command input and execution records for each test, result compliance comparison, test category, and test result judgment.

7. The automated testing and data management system according to claim 1, characterized in that, The filtering analysis includes: retrieving and analyzing the core indicators in the test results through keyword search, automatically removing invalid data, retaining key data, comparing it with the corresponding test item requirements parameters, providing a result judgment, and highlighting the processed key test results on the page using different colors.

8. The automated testing and data management system according to claim 1, characterized in that, The test log includes timestamps, input command information, system return information, test execution results, and error information.

9. The automated testing and data management system according to claim 1, characterized in that, The page displays information including a summary of overall test information, test data statistics, test item details, and log details. Test data statistics include full statistics, statistics categorized by tag, and statistics categorized by selected test suite; The test item details view supports full details viewing, viewing by test tag, viewing by test suite, and custom search; custom search includes suite name search, test item name search, search including tag keywords, and search excluding tag keywords.

10. An automated testing and data management method, characterized in that, include: Customize test object information, execute user-defined test scripts, generate data during the test process, and perform real-time progress monitoring and interactive control during the test; Collect data during the testing process, filter and analyze the collected data, compare compliance, and record and display all test logs.

Images