System, method, device and equipment for automatically testing operating system and storage medium

Abstract

The invention provides an automatic test system, method and device for an operating system, electronic equipment and a computer readable storage medium, and is applied to the technical field of computers, a front-end application module is used for determining equipment to be tested and obtaining a test case according to a front-end human-computer interaction interface, and constructing a test task based on the equipment to be tested and the test case; the task scheduling module is used for responding to a test starting operation of the front-end human-computer interaction interface, analyzing a test task through the task scheduling module, obtaining test parameters including priorities of test cases, and transmitting the test task and the test parameters to the test framework according to the sequence of the priorities; the test framework is used for executing the test task according to the test parameters to obtain a test result. According to the method and the device, the to-be-tested equipment and the test cases can be managed in a unified manner, the test tasks can be managed and maintained in a unified manner, and automatic test task execution is realized, so that the test efficiency is greatly improved, and the test accuracy is also greatly improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an automatic operating system testing system, method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Operating system testing can reflect the stability of the operating system. Specifically, the testing can be achieved by executing test cases on the device under test that is running the operating system.

[0003] Currently, the testing process is executed manually or semi-automatically. The manual method requires manually importing test cases into the device under test for execution and manually collecting and summarizing the test results. The semi-automatic testing method can achieve semi-automatic testing with the help of testing software, but manual configuration is required during the testing process.

[0004] However, in the current solutions, both manual and semi-automatic testing methods rely heavily on human resources, resulting in reduced testing efficiency and lower testing accuracy. Summary of the Invention

[0005] This application provides an automatic operating system testing system, method, and apparatus to address the problems in the prior art.

[0006] In a first aspect, embodiments of this application provide an automatic operating system testing system, the method comprising:

[0007] Front-end application module, task scheduling module, and testing framework;

[0008] The front-end application module is used to: provide a front-end human-computer interaction interface, and based on the front-end human-computer interaction interface, determine the device to be tested and obtain test cases, construct test tasks based on the device to be tested and the test cases, and summarize the test results of the test tasks into a test report for display.

[0009] The task scheduling module is used to: respond to the start test operation of the front-end human-computer interaction interface, parse the test task through the task scheduling module, obtain test parameters including the priority of the test cases, and pass the test task and the test parameters to the test framework in the order of the priority.

[0010] The testing framework is used to: execute the testing task according to the testing parameters and obtain the test results.

[0011] Secondly, embodiments of this application provide an automatic operating system testing method, the method comprising:

[0012] Based on the front-end human-computer interaction interface, determine the device to be tested and obtain test cases;

[0013] Based on the device under test and the test cases, construct a test task;

[0014] In response to the start test operation of the front-end human-computer interaction interface, the test task is parsed, the test parameters are obtained, and the test task is executed according to the test parameters to obtain the test results;

[0015] The test results of the aforementioned test tasks are summarized into a test report for presentation.

[0016] Thirdly, embodiments of this application provide an automatic operating system testing device, the device comprising:

[0017] The acquisition module is used to determine the device to be tested and acquire test cases based on the front-end human-computer interaction interface;

[0018] A construction module is used to construct test tasks based on the device under test and the test cases;

[0019] The testing module is used to respond to the start test operation of the front-end human-computer interaction interface, parse the test task, obtain test parameters, execute the test task according to the test parameters, and obtain test results;

[0020] The output module is used to summarize the test results of the test task into a test report for display.

[0021] Fourthly, embodiments of this application also provide an electronic device, including a processor;

[0022] Memory used to store the processor's executable instructions;

[0023] The processor is configured to execute the instructions to implement the method.

[0024] Fifthly, embodiments of this application also provide a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method described thereon.

[0025] The automated operating system testing system of this application embodiment can uniformly manage the devices under test and test cases, uniformly manage and maintain test tasks, and realize automated test task execution and test result collection, summary and display. For the process of frequently executed test cases on the devices under test in the test scenario, it can significantly improve testing efficiency, save testing time, and the entire testing process is simple to operate and easy to maintain. By adopting an automated and uniformly deployed testing method, the testing accuracy will also be greatly improved, avoiding errors caused by human factors.

[0026] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0027] Figure 1 This is a block diagram of an automatic operating system testing system provided in an embodiment of this application;

[0028] Figure 2 This is a flowchart illustrating the steps of an automatic operating system testing system provided in an embodiment of this application;

[0029] Figure 3 This is a test flowchart provided in an embodiment of this application;

[0030] Figure 4 This is a flowchart illustrating the operation of a test system based on a three-tier architecture, as provided in an embodiment of this application.

[0031] Figure 5 This is a block diagram of an automatic operating system testing device provided in an embodiment of this application;

[0032] Figure 6 This is a block diagram of an apparatus according to this application;

[0033] Figure 7 This is a schematic diagram of the server structure in some embodiments of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or more. Furthermore, the term "and / or" in the specification and claims is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0036] Figure 1 This is a block diagram of an automatic operating system testing system provided in an embodiment of this application. The system includes:

[0037] Front-end application module, task scheduling module, and testing framework;

[0038] The front-end application module is used to: provide a front-end human-computer interaction interface, determine the device to be tested and obtain test cases based on the front-end human-computer interaction interface, construct test tasks based on the device to be tested and test cases, and summarize the test results of the test tasks into a test report for display.

[0039] The task scheduling module is used to: respond to the start test operation of the front-end human-computer interaction interface, parse the test task through the task scheduling module, obtain test parameters including the priority of test cases, and pass the test task and test parameters to the test framework in order of priority.

[0040] The testing framework is used to execute test tasks based on test parameters and obtain test results.

[0041] This application embodiment can construct the testing system as a three-tier architecture, including a front-end application module, a task scheduling module, and a testing framework. The front-end application module provides a user interface for managing and maintaining test cases, devices under test, and test tasks through controls within the interface. The front-end application module can automatically import test cases, select devices under test, and construct test tasks. Different test tasks select different devices under test, and the test scope is customized according to requirements. When automated testing is triggered, the front-end application module passes the test tasks and test parameters to the task scheduling module.

[0042] The device under test can be selected from multiple devices, and it can run the operating system to be tested (such as Linux). The operating system is then tested by executing test cases on the device under test. A test case is the smallest unit of execution in a test scenario. It can be understood as a standardized verification step designed for a specific function or scenario. It includes input data, execution conditions, and expected results. It is a test file used to determine whether the system behavior meets the requirements. The test case defines the test objectives of the function to be tested, the environmental conditions that must be met to execute the test, the parameters or operations used during test execution, the specific operation sequence of the test steps, the correct response that the operating system should have, and the test priority.

[0043] In some embodiments, the front-end application module can be an automated test management web application developed based on the Lin CMS open-source framework. The Lin CMS framework is a front-end and back-end separated open-source content management system (CMS) framework, which includes the most common CMS requirements: user management, permission management, and a logging system. The front-end web application is technically implemented using Node.js and Vue.js development languages, employs a MySQL database, and uses Sequelize and Model components to connect and operate between the application and the database. Functionally, it enables visual access and management of relevant elements of the automated testing process through a browser, including: management of devices under test, test case management, test task management, and test report display.

[0044] In addition, the front-end application module is also connected to the identity authentication server and test management tool of the device under test, so as to realize unified identity authentication login and hierarchical operation, and can synchronize the test orders of the test management tool to the test system, so as to quickly lock the test scope of different test tasks. After the automated test is completed, the test system can also automatically fill back the test results to the test order.

[0045] The task scheduling module, in some embodiments, can be a Jenkins continuous integration tool. This tool can parse test tasks, trigger the test framework to connect to the device under test based on the parsing results, and then control the device under test to automatically execute tests according to the steps of the test cases and the issued test instructions. It then determines whether the tests pass or fail, and after the backend tests are completed, the results are sent back to the task scheduling module. For test scenarios, the task scheduling module can also trigger the parsing of test logs and save performance test data. The task scheduling module can then send the test results back to the frontend application module, finally achieving visualized test report management. This test report management mainly includes: test log display, real-time monitoring of test logs, statistical display of test task execution results, test log download, sending test logs via email, and performance test data display and download.

[0046] A testing framework is a test framework that can be deployed on the operating system of the test system's server. In one embodiment, the testing framework is built based on the Python development language and uses the Avocado open-source testing framework to trigger the execution of automated tests.

[0047] Based on the three-tier architecture of the testing system described above, during test execution, in response to the test initiation operation, the task scheduling module first parses the test task to obtain test parameters, including the priority configured for the test cases. Then, the task scheduling module passes the test tasks and test parameters to the testing framework according to priority. The testing framework then uses the test tasks and test parameters to trigger a connection to the device under test, controlling the device to automatically execute the tests according to the steps of the test cases and the issued test instructions, and determines whether the tests pass or fail. After the backend tests are completed, the results are sent back to the task scheduling module.

[0048] Furthermore, the front-end application module can also provide a test report management interface to summarize test results into test reports for display. The test report management interface allows users to view information such as the creation date, architecture, creator, test status, and release status of all test tasks, and retains all records. For each test task, users can view the test case number, test case name, test client, test priority, script path, execution time, test results, and test log information. This application embodiment also provides test statistics, test report download, and email forwarding functions, solving problems such as test report collection and unified management by providing visualized test reports.

[0049] The automated operating system testing system of this application embodiment can uniformly manage the devices under test and test cases, uniformly manage and maintain test tasks, and realize automated test task execution and test result collection, summary and display. For the process of frequently executed test cases on the devices under test in the test scenario, it can significantly improve testing efficiency, save testing time, and the entire testing process is simple to operate and easy to maintain. By adopting an automated and uniformly deployed testing method, the testing accuracy will also be greatly improved, avoiding errors caused by human factors.

[0050] Optionally, the task scheduling module is further configured to:

[0051] The device under test included in the test task is locked, and the locked device under test is configured not to be occupied by other test tasks.

[0052] In this embodiment, the task scheduling module can lock the device under test included in the test task. The locked device under test is configured not to be occupied by other test tasks. That is, by marking the device under test included in the test task as "in use", the device under test can be prevented from being occupied by other test tasks, which would lead to inaccurate test results.

[0053] Optionally, the testing framework is specifically used for:

[0054] Obtain the configuration file, which includes the communication information of the device under test included in the test task.

[0055] Based on the communication information included in the configuration file, the test framework sends test commands to the device under test included in the test task via the SSH protocol to control the device under test to perform the test.

[0056] Obtain the test results returned by the device under test.

[0057] The testing framework of this application embodiment can automate testing based on the operating system terminal command line of the device under test. In some embodiments, the testing framework can be the avocado framework. The avocado framework can connect to the public functions of the device under test based on the Secure Shell (SSH) protocol and remotely issue test commands, thereby enabling the avocado framework to control the device under test to implement the test process.

[0058] In some embodiments, this application can support simultaneous connection to multiple devices under test via the SSH protocol to meet the testing requirements of interactive scenarios, such as network communication testing between two devices under test. Specifically, the implementation method involves filling in the communication information of multiple devices under test using an INI format configuration file. When the `avocado run` command is executed to run automated test cases, the communication information of the devices under test is obtained from the INI configuration file. Based on the test case requirements, one or more devices under test can be connected via SSH to execute the test, thereby satisfying test cases requiring interaction between multiple devices under test.

[0059] Communication information is information that reflects the communication address of a device, such as the device's Internet Protocol (IP) address, port number, etc.

[0060] In some embodiments, the avocado framework can also support operating system terminal command lines, enabling automated testing via the operating system terminal command lines to achieve a seamless testing effect.

[0061] In some implementations, the avocado framework supports performing various types of tests on the operating system, including: preboot execution environment (PXE) installation and deployment testing, functional testing, stability testing, performance testing, etc.

[0062] In some implementations, the avocado framework supports major operating system testing tools, including: the LTP (LinuxTest Project, an open-source project) tool for performing kernel tests; and performance testing tools such as Unixbench, Lmbench, Iozone, Stream, Specjvm, and Netperf for performing memory, network, and port performance tests.

[0063] In some implementations, the avocado framework can acquire and save the terminal's input commands and output results through Python file streams, generating terminal log files to facilitate complete backtracking of test steps and outputs.

[0064] Optionally, the testing framework is specifically used for:

[0065] The device under test is controlled to identify the interface controls of the device under test based on an image recognition service.

[0066] Based on the test parameters, the device under test is controlled to operate the interface controls based on the peripheral control service, so that the device under test executes the corresponding test cases.

[0067] The device under test is controlled to take a screenshot of the execution result interface of the test case based on the screenshot service, and the device under test is controlled to recognize the text content in the screenshot result based on the image recognition service to obtain the execution result.

[0068] In this embodiment, the process of the test framework controlling the execution of test cases by the device under test can specifically include: utilizing the coordinate values ​​of each control on the interface, peripheral control services (kernel-level keyboard and mouse control programs), screenshot tools, and an Optical Character Recognition (OCR) server with a text recognition library as its core technology, firstly, based on the coordinate values ​​of each control in the interface of the device under test, the peripheral control services perform operations such as opening the test program and inputting test content on the device under test according to the test steps in the test cases, thereby simulating the real test process. After the test is completed, a screenshot service can be used to capture a specified area in the test result interface of the device under test and send it to the OCR server for text recognition. The OCR server can output the recognition results, and at this time, the test framework can determine whether the test meets the expected results, i.e., whether the test passes, based on the recognition results.

[0069] Optionally, the testing framework is specifically used for:

[0070] The test task is executed according to the test parameters, and the test results of the list data structure are output through a result output function based on regular expressions.

[0071] In this embodiment, the testing framework also supports Python regular expressions, providing a unified output function. This function can return a Python list data structure (reflecting the test results) based on regular expressions. This output format improves the accuracy of test result judgment. In subsequent test case development, the list index can be used to further determine subsequent test steps or provide clear prompts, facilitating the judgment of test results.

[0072] Optionally, the front-end application module is specifically used for:

[0073] In response to operations on the front-end human-computer interaction interface, the device to be tested participating in the test task is determined;

[0074] In response to the operation of the front-end human-computer interaction interface, the test cases participating in the test task are determined, and the test task is obtained. The test task is used to: control the device under test participating in the test task and execute the test cases participating in the test task.

[0075] In this embodiment of the application, regarding the process of constructing a test task, the devices to be tested participating in the test task can first be determined by operating the determination control in the front-end human-computer interaction interface provided by the front-end application module. For example, several devices can be selected from the device list as devices to be tested.

[0076] Subsequently, by manipulating the associated controls in the front-end human-computer interaction interface provided by the front-end application module, the test cases participating in the test task can be determined, which are the test cases in the test case library associated with the device under test participating in the test task. Alternatively, the test cases participating in the test task can also be obtained synchronously from the test requirement form. Because there is a one-to-one correspondence between test cases and functions to be tested, the required test cases for this test task can be determined based on the functions to be tested recorded in the test requirement form.

[0077] The execution of test cases on which device under test is performed is dynamically allocated by the task scheduling system. The task scheduling system monitors the execution status of test cases on the device under test, and once a test case is completed, it will allocate the next test case to continue execution until completion.

[0078] Next, the device under test can be configured. Specifically, the runtime environment of the device under test is configured to meet the test requirements and enable the execution of test cases for those requirements. For example, for server performance testing, the device under test needs to be configured as an x86 architecture device with 64GB of memory. The test system can then filter available devices under test based on architecture / memory requirements.

[0079] After configuring the device under test, the test parameters can be configured. Test parameters control the implementation of the test process and typically include environment parameters, behavioral parameters, performance parameters, and data parameters. For example, environment parameters can indicate the operating system version being tested; behavioral parameters can define the number of retries for test failures; performance parameters can define the number of connections; and data parameters can define the size of the test dataset. Additionally, test parameters define the priority and dependencies of test cases within the test task. Once the test parameters are configured, the test task is created.

[0080] The automated operating system testing system of this application embodiment can uniformly manage the devices under test and test cases, uniformly manage and maintain test tasks, and realize automated test task execution and test result collection, summary and display. For the process of frequently executed test cases on the devices under test in the test scenario, it can significantly improve testing efficiency, save testing time, and the entire testing process is simple to operate and easy to maintain. By adopting an automated and uniformly deployed testing method, the testing accuracy will also be greatly improved, avoiding errors caused by human factors.

[0081] Figure 2 This is a flowchart illustrating the steps of an automatic operating system testing method provided in an embodiment of this application. Figure 2 As shown, the method may include:

[0082] Step 101: Based on the front-end human-computer interaction interface, determine the device to be tested and obtain the test cases.

[0083] Step 102: Based on the device under test and the test cases, construct a test task.

[0084] Step 103: In response to the start test operation of the front-end human-computer interaction interface, parse the test task, obtain the test parameters, execute the test task according to the test parameters, and obtain the test results.

[0085] Step 104: Summarize the test results of the test tasks into a test report and display it.

[0086] In this application embodiment, regarding steps 101-104, in some embodiments, the process of determining the device under test is also referred to as the maintenance process of the device under test. This application embodiment provides a front-end human-computer interaction interface, so that through the relevant controls in the operation interface, it is possible to create, edit, and delete devices under test, and to display information such as the classification, hardware architecture, machine model, processor model, communication address, and idle status of the device under test in real time. During testing, in response to the selection operation, an idle device under test with the corresponding architecture can be selected, thus achieving unified management of devices under test.

[0087] In some embodiments, test cases can also be managed in a unified manner. That is, through the front-end human-computer interaction interface provided in the embodiments of this application, functions such as creating test cases, importing test cases, modifying test cases, and deleting test cases can be realized by operating the relevant controls in the interface.

[0088] For example, you can import test cases by clicking the "Import Test Cases" button in the interface, downloading an Excel file template, and filling in the basic information of the newly developed test cases. This allows you to import test cases in batches into the test system. For instance, for test cases testing network performance, you can import information such as: module name "Network Performance", test case name "Network Card Bandwidth Test", test case path " / tests / network / netperf.py", priority "P0", and expected result description "Bandwidth ≥ 950Mbps" to an Excel file.

[0089] Test cases can be displayed by using the tree control on the left side of the interface, which filters and displays different test cases according to different operating system product libraries, functional modules, and priorities.

[0090] You can edit test cases by clicking the "Edit" button in the interface. This allows you to edit the basic information of the test cases, such as the module and name.

[0091] Test cases can be deleted by clicking the "Delete" button in the interface. Test cases that have not been added to any test plan can be removed.

[0092] The following shows the structure of a test case for testing network performance:

[0093] [Use Case ID]: NET-STRESS-001

[0094] [Module]: Network Performance

[0095] [Name]: Network Card Long Connection Stress Test

[0096] [Prerequisites]:

[0097] 1. The two servers are directly connected via a 10 Gigabit optical port.

[0098] 2. Install network performance benchmarking tools

[0099] [Input parameters]:

[0100] - Concurrent connections: 1000

[0101] - Test duration: 300 seconds

[0102] [Execution Steps]:

[0103] 1. Start netserver on the host: ˋnetserver -p12865ˋ

[0104] 2. Execute on the standby machine: ˋnetperf -H <host IP> -l300 -tTCP_STREAM -P0 -cˋ

[0105] [Expected Result]:

[0106] 1. No connection interrupted

[0107] 2. Average throughput ≥ 9.8Gbps

[0108] [Priority]: P0

[0109] [Related script:] / tests / network / tcp_stress.py

[0110] Furthermore, the device to be tested participating in the test task and the test cases participating in the test task can be determined. Then, the test parameters can be configured to build a test task. By repeating this process, multiple test tasks can be built in batches.

[0111] In some embodiments, test tasks can also be managed in a unified manner. Through the front-end human-computer interaction interface provided in this application embodiment, the functions of creating test tasks, modifying test tasks, deleting test tasks, associating test tasks with test cases, starting and executing test tasks, stopping running test tasks, viewing real-time test progress, viewing test result logs, displaying test task execution information, re-executing the entire test task, re-executing individual test cases, publishing test tasks, and withdrawing published test tasks can be realized through the relevant controls in the operation interface.

[0112] In this application embodiment, the front-end human-computer interaction interface can be a web page.

[0113] In some embodiments, the test can be initiated by operating the start test control in the front-end human-computer interaction interface. The test process can support the parallel execution of multiple test tasks, thereby improving test efficiency. During test execution, for each test task, test cases can be executed through the device under test included in the test task. After execution, test results can be collected and test case execution logs can be generated. Failed test cases or the entire test task can also be re-executed. This application embodiment supports graphics-based functional testing and terminal-based functional testing, performance testing, and reliability testing; it also supports performance comparison of different versions; smoke testing based on customized requirements, etc., mainly solving the problem of low test efficiency.

[0114] Optionally, step 102 may specifically include sub-steps 1021-1022:

[0115] Sub-step 1021: In response to the operation of the front-end human-computer interaction interface, determine the device to be tested that will participate in the test task.

[0116] Sub-step 1022: In response to the operation of the front-end human-computer interaction interface, determine the test cases participating in the test task, and obtain the test task. The test task is used to: control the device under test participating in the test task and execute the test cases participating in the test task.

[0117] In this embodiment of the application, for sub-steps 1021-1022, the relevant descriptions of the above-described embodiment of the automatic operating system testing system can be referred to, and will not be repeated here.

[0118] Optionally, step 103 may specifically include sub-steps 1031-1033:

[0119] Sub-step 1031: Obtain the configuration file through the test framework. The configuration file includes the communication information of the device under test included in the test task.

[0120] Sub-step 1032: Based on the communication information included in the configuration file, the test framework sends test commands to the device under test included in the test task via the SSH protocol to control the device under test to perform the test.

[0121] Sub-step 1033: Obtain the test results returned by the device under test through the test framework.

[0122] In this embodiment of the application, for sub-steps 1031-1033, the relevant descriptions of the above-described embodiment of the automatic operating system testing system can be referred to, and will not be repeated here.

[0123] Optionally, the method may further include:

[0124] Step A1: Lock the device to be tested included in the test task through the task scheduling module. The locked device to be tested is configured not to be occupied by other test tasks.

[0125] In this embodiment of the application, for step A1, the relevant description of the above embodiment of the automatic testing system for the operating system can be referred to, and will not be repeated here.

[0126] Optionally, sub-step 1032 may specifically include sub-steps 10321-10323:

[0127] Sub-step 10321: Identify the interface controls of the device under test based on the image recognition service.

[0128] Sub-step 10322: Based on the test parameters, operate the interface controls through the peripheral control service to enable the device under test to execute the corresponding test cases.

[0129] Sub-step 10323: Take a screenshot of the execution result interface of the device under test for the test case using the screenshot service, and identify the text content in the screenshot result using the image recognition service to obtain the execution result.

[0130] In this embodiment of the application, for sub-steps 10321-10323, the relevant descriptions of the above-described embodiment of the automatic operating system testing system can be referred to, and will not be repeated here.

[0131] Optionally, step 103 may specifically include sub-step 1034:

[0132] Sub-step 1034: Execute the test task according to the test parameters through the test framework, and output the test results of the list data structure through the result output function based on regular expressions.

[0133] In this embodiment, for sub-step 1034, the relevant description of the above-described embodiment of the automatic operating system testing system can be referred to, and will not be repeated here.

[0134] In some embodiments, refer to Figure 3 It shows a test flowchart, including:

[0135] Step 301: Maintenance of the device under test.

[0136] Step 302: Test case maintenance.

[0137] Step 303: Test task construction and management.

[0138] Step 304: Automated test execution.

[0139] Step 305: Test Report Management.

[0140] In some embodiments, refer to Figure 4 It shows a flowchart of a test system based on a three-tier architecture, which includes: a front-end application module, a task scheduling module, and a test framework.

[0141] S1. The front-end application module starts and creates automated test tasks.

[0142] S2. Select the device to be tested for the front-end application module.

[0143] S3. Configure test parameters for the front-end application module, determine the test scope, and obtain test tasks.

[0144] S4. Start automated testing for the front-end application module.

[0145] S5. The task scheduling module sends test tasks to the test framework according to priority.

[0146] S6: The testing framework automatically executes tests and returns the test results to the task scheduling module.

[0147] S7. The task scheduling module obtains test results from the test framework.

[0148] S8, Front-end application module: View test reports built from test results.

[0149] S9. The front-end application module determines whether to retest or complete the test.

[0150] If a retest is required, execute S10, which triggers a retest of the front-end application module.

[0151] If the test is completed, execute S11 and the front-end application module will backfill the test results to the test form.

[0152] In summary, the operating system automated testing system of this application embodiment can uniformly manage the devices under test and test cases, uniformly manage and maintain test tasks, and realize automated test task execution and test result collection, summary and display. For the process of frequently executed test cases on the devices under test in the test scenario, it can significantly improve testing efficiency, save testing time, and the entire testing process is simple to operate and easy to maintain. By adopting an automated and uniformly deployed testing method, the testing accuracy will also be greatly improved, avoiding errors caused by human factors.

[0153] Figure 5 This is a block diagram of an automatic operating system testing device provided in an embodiment of this application. The device includes:

[0154] The acquisition module 401 is used to determine the device to be tested and acquire test cases based on the front-end human-computer interaction interface;

[0155] Module 402 is used to construct test tasks based on the device under test and the test cases;

[0156] The test module 403 is used to respond to the start test operation of the front-end human-computer interaction interface, parse the test task, obtain test parameters, execute the test task according to the test parameters, and obtain test results;

[0157] The output module 404 is used to summarize the test results of the test task into a test report for display.

[0158] Optionally, the device further includes:

[0159] The configuration module is used to lock the devices to be tested included in the test task through the task scheduling module, and the locked devices to be tested are configured not to be occupied by other test tasks.

[0160] Optionally, the test module includes:

[0161] The communication information submodule is used to obtain a configuration file through the test framework. The configuration file includes the communication information of the device under test included in the test task.

[0162] The test command submodule is used to send test commands to the device under test included in the test task through the test framework based on the SSH protocol, according to the communication information included in the configuration file, so as to control the device under test to execute the test.

[0163] The test results submodule is used to obtain the test results returned by the device under test through the test framework.

[0164] Optionally, the test command submodule includes:

[0165] The first recognition unit is used to identify the interface controls of the device under test based on image recognition services.

[0166] The control unit is used to operate the interface controls through the peripheral control service according to the test parameters, so that the device under test executes the corresponding test cases;

[0167] The second recognition unit is used to take a screenshot of the execution result interface of the device under test for the test case through the screenshot service, and to recognize the text content in the screenshot result through the image recognition service to obtain the execution result.

[0168] Optionally, the test module includes:

[0169] The output submodule is used to execute the test task according to the test parameters through the test framework, and output the test results of the list data structure through a result output function based on regular expressions.

[0170] Optionally, the building module 402 includes:

[0171] The allocation submodule is used to determine the device to be tested that will participate in the test task in response to the operation of the front-end human-computer interaction interface;

[0172] The association submodule is used to respond to the operation of the front-end human-computer interaction interface, determine the test cases participating in the test task, and obtain the test task. The test task is used to: control the device under test participating in the test task and execute the test cases participating in the test task.

[0173] In summary, the operating system automated testing system of this application embodiment can uniformly manage the devices under test and test cases, uniformly manage and maintain test tasks, and realize automated test task execution and test result collection, summary and display. For the process of frequently executed test cases on the devices under test in the test scenario, it can significantly improve testing efficiency, save testing time, and the entire testing process is simple to operate and easy to maintain. By adopting an automated and uniformly deployed testing method, the testing accuracy will also be greatly improved, avoiding errors caused by human factors.

[0174] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0175] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0176] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0177] This application provides an automatic operating system testing device, including a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include methods for performing the methods described in one or more of the above embodiments.

[0178] Figure 6 This is a block diagram illustrating an operating system automatic testing device 800 according to an exemplary embodiment. For example, device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0179] Reference Figure 6 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0180] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0181] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0182] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.

[0183] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of the touch or swipe action but also examine the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0184] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice information processing mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0185] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0186] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 can detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, and can also detect changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0187] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency information processing (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0188] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0189] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0190] Figure 7This is a schematic diagram of the server structure in some embodiments of this application. The server 1900 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1922 (e.g., one or more processors) and memory 1932, and one or more storage media 1930 (e.g., one or more mass storage devices) for storing application programs 1942 or data 1944. The memory 1932 and storage media 1930 can be temporary or persistent storage. The program stored in the storage media 1930 may include one or more modules, each module including a series of instruction operations on the server. Furthermore, the CPU 1922 may be configured to communicate with the storage media 1930 and execute the series of instruction operations in the storage media 1930 on the server 1900.

[0191] Server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input / output interfaces 1958, one or more keyboards 1956, and / or one or more operating systems 1941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0192] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a device (server or terminal), enable the device to perform the methods of the embodiments described above.

[0193] A non-transitory computer-readable storage medium is provided, wherein when the instructions in the storage medium are executed by the processor of a device (server or terminal), the device is able to perform the methods described in the above embodiments; therefore, further details will not be repeated here. Additionally, the beneficial effects of employing the same method will also not be repeated. For technical details not disclosed in the computer program products or computer program embodiments related to this application, please refer to the description of the method embodiments of this application.

[0194] Furthermore, it should be noted that this application also provides a computer program product or computer program, which may include computer instructions, which may be stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor can execute the computer instructions to cause the computer device to perform the methods described in the above embodiments; therefore, these will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer program product or computer program involved in this application, please refer to the description of the method embodiments of this application.

[0195] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0196] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0197] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0198] The above provides a detailed description of an automatic operating system testing system, apparatus, electronic device, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An automatic operating system testing system, characterized in that, The system includes: Front-end application module, task scheduling module, and testing framework; The front-end application module is used to: provide a front-end human-computer interaction interface, and determine the device to be tested and obtain test cases based on the front-end human-computer interaction interface; construct test tasks based on the device to be tested and the test cases; and summarize the test results of the test tasks into a test report for display. The task scheduling module is used to: respond to the start test operation of the front-end human-computer interaction interface, parse the test task through the task scheduling module, obtain test parameters including the priority of the test cases, and pass the test task and the test parameters to the test framework in the order of the priority. The testing framework is used to: execute the testing task according to the testing parameters and obtain the test results.

2. The automatic operating system testing system according to claim 1, characterized in that, The task scheduling module is also used for: The device under test included in the test task is locked, and the locked device under test is configured not to be occupied by other test tasks.

3. The automatic operating system testing system according to claim 1, characterized in that, The testing framework is used for: Obtain the configuration file, which includes the communication information of the device under test included in the test task; Based on the communication information included in the configuration file, the test framework sends test commands to the device under test included in the test task via the SSH protocol to control the device under test to perform the test. Obtain the test results returned by the device under test.

4. The automatic operating system testing system according to claim 3, characterized in that, The testing framework is used for: The device under test is controlled to identify the interface controls of the device under test based on an image recognition service. Based on the test parameters, the device under test is controlled to operate the interface controls based on the peripheral control service, so that the device under test executes the corresponding test cases; The device under test is controlled to take a screenshot of the execution result interface of the test case based on the screenshot service, and the device under test is controlled to recognize the text content in the screenshot result based on the image recognition service to obtain the execution result.

5. The automatic operating system testing system according to claim 1, characterized in that, The testing framework is used for: The test task is executed according to the test parameters, and the test results of the list data structure are output through a result output function based on regular expressions.

6. The automatic operating system testing system according to claim 1, characterized in that, The front-end application module is used for: In response to operations on the front-end human-computer interaction interface, the device to be tested participating in the test task is determined; In response to the operation of the front-end human-computer interaction interface, the test cases participating in the test task are determined, and the test task is obtained. The test task is used to: control the device under test participating in the test task and execute the test cases participating in the test task.

7. An automatic testing method for an operating system, characterized in that, The method includes: Based on the front-end human-computer interaction interface, determine the device to be tested and obtain test cases; Based on the device under test and the test cases, construct a test task; In response to the start test operation of the front-end human-computer interaction interface, the test task is parsed, the test parameters are obtained, and the test task is executed according to the test parameters to obtain the test results; The test results of the aforementioned test tasks are summarized into a test report for presentation.

8. An automatic operating system testing device, characterized in that, The device includes: The acquisition module is used to determine the device to be tested and acquire test cases based on the front-end human-computer interaction interface; A construction module is used to construct test tasks based on the device under test and the test cases; The testing module is used to respond to the start test operation of the front-end human-computer interaction interface, parse the test task, obtain test parameters, execute the test task according to the test parameters, and obtain test results; The output module is used to summarize the test results of the test task into a test report for display.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in claim 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in claim 7.

Images