Distributed automated test platform

Abstract

This invention discloses a distributed automated testing platform, belonging to the field of software testing technology. Specifically, the invention involves: obtaining test task configuration information through a graphical interface; the task engine, based on this information, deploying the environment and distributing data files to the target load group in a distributed parallel manner; subsequently, the task engine generates test tasks and adds them to the task queue for scheduling, executing load tests on the target load group in a distributed mode and generating result files; the computing engine performs distributed parsing of the result files, generating performance indicators and triggering report generation. This invention, through the collaboration of the task engine and the computing engine, achieves full-process automation and parallel processing of test environment management, task scheduling, process monitoring, and result analysis, effectively solving the problems of high manual operation ratio, low efficiency, and difficulty in integration in traditional testing processes, and significantly improving the throughput of test execution and the efficiency of result analysis.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and more specifically to a distributed automated testing platform. Background Technology

[0002] In the software development lifecycle, performance testing is a crucial step in ensuring system reliability and stability. Traditional performance testing processes typically rely on testers manually operating a series of toolchains, which presents significant efficiency bottlenecks and operational risks.

[0003] Specifically, firstly, setting up the test environment heavily relies on manual deployment by logging into each load server individually, including installing the Java runtime environment and configuring testing tools, a tedious and time-consuming process. Secondly, configuring and adjusting test parameters requires a command-line interface, which has low fault tolerance and is prone to human error leading to test deviations. Thirdly, the data files required for testing (such as test datasets and dependency libraries) are usually manually copied between load servers using physical storage media (such as USB drives) or simple network transfer tools, resulting in lengthy data transfer times for large files or large-scale load servers. Fourthly, the recording, monitoring, and collection and analysis of test results often rely on manual management using spreadsheets, making it difficult to monitor the test status in real time, and the subsequent data analysis is labor-intensive and time-consuming. Fifthly, this discrete manual operation mode is difficult to integrate effectively with continuous integration / continuous delivery pipelines, leading to a disconnect between the testing phase and the development and operations process, becoming a breakpoint in the automation process. Although some improvement solutions attempt to achieve visual configuration of some operations through scripting, they still fail to fundamentally solve the problems of weak visualization and analysis capabilities of test results and low degree of automation throughout the entire process.

[0004] Therefore, an innovative automated testing solution is needed to address the problems of traditional performance testing processes, such as high reliance on manual operations, low efficiency, and difficulty in automation integration. Summary of the Invention

[0005] The purpose of this invention is to provide a distributed automated testing platform to overcome the aforementioned problems in the prior art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A testing method based on a distributed automated testing platform includes the following steps: In response to user operations via a graphical interface, test task configuration information is obtained, which includes at least test scripts, target load unit information, and test parameters. Based on the test task configuration information, the task engine performs environment deployment and data file distribution on each load machine in the target load group in a distributed parallel manner. After the environment deployment and data file distribution are completed, based on the test scripts and test parameters in the test task configuration information, the test task is generated by the task engine and added to the task queue for scheduling. The load test is executed in a distributed mode on the target load group, and a test result file is generated. Based on the test result file, the computing engine performs distributed parsing of the test result file to generate test performance indicators and trigger the generation of test reports.

[0007] Furthermore, the environment deployment includes: concurrent interaction with each load machine based on multi-threaded pool technology to perform system parameter verification, runtime environment installation, and test tool cluster configuration.

[0008] Furthermore, the data file distribution includes: Obtain the data file to be distributed that is associated with the test task configuration information; The data file to be distributed is fragmented and uploaded concurrently to each load machine in the target load group through multi-threading, and the distribution process supports breakpoint resumption and consistency verification.

[0009] Furthermore, the test task is added to a task queue for scheduling, including: the task queue is an asynchronous task queue; test tasks are created and managed based on the asynchronous task queue; the target load group is constructed into a test cluster using a master-slave mode; and the execution status of the test task is synchronized to the front-end interface in real time.

[0010] Furthermore, the distributed parsing of the test result file using the computing engine includes: The test result files to be parsed are automatically detected through a timed scanning mechanism; The distributed computing framework is invoked to split the test result file into multiple data blocks and distribute them to multiple computing nodes for parallel parsing, in order to calculate at least one performance metric among transaction processing rate (TPS), response time, and error rate.

[0011] Furthermore, the timed scanning mechanism is based on APScheduler to implement scanning tasks with a minute-level cycle.

[0012] Furthermore, the method also includes: The front-end interface integrates a multi-dimensional data statistics dashboard, dynamically displaying the operating status, resource usage, test script inventory, and time-based test execution volume analysis view of the target load unit.

[0013] Another objective of this invention is to provide a distributed automated testing platform for implementing the aforementioned testing method based on a distributed automated testing platform, comprising: The computing engine is configured to perform distributed parsing of test result files to generate test performance metrics and trigger test report generation. The task engine is configured to deploy the environment and distribute data files to the target load group in a distributed parallel manner based on the test task configuration information, generate test tasks based on the test task configuration information, add the test tasks to the task queue for scheduling, and execute load tests on the target load group in a distributed mode to generate the test result files.

[0014] Furthermore, the computing engine includes: The timed scanning module is configured to automatically detect the test result files to be parsed; The distributed parsing module is configured to call a distributed computing framework to split the test result file into multiple data blocks for parallel parsing and to calculate performance metrics.

[0015] Furthermore, the task engine includes: The environment deployment module is configured to deploy the environment of the target load unit in a distributed and parallel manner; The data distribution module is configured to distribute data files to the target load group in a distributed and parallel manner; The task scheduling module is configured to schedule test scripts to be executed in a distributed mode on the target load group.

[0016] This invention effectively solves several efficiency bottlenecks in traditional testing processes by integrating the distributed parallel processing capabilities of the task engine and the computing engine. Compared with existing technologies, it has the following advantages: 1. The task engine of this invention adopts multi-threaded concurrency and file sharding technology to realize parallel environment deployment and data distribution for multiple load machines. Compared with the traditional manual serial operation method, it can significantly shorten the test environment preparation time and improve deployment and distribution efficiency.

[0017] 2. By providing a graphical parameter configuration interface and automated scheduling based on asynchronous task queues, it replaces error-prone manual command line operations and manual task management, reducing operational complexity and the risk of human error, and enabling real-time monitoring of test execution status.

[0018] 3. The computing engine, based on a distributed computing framework, performs parallel parsing of massive test result files. It breaks down the parsing task of a single large file into multiple sub-tasks for parallel processing, significantly improving the speed of result parsing and report generation compared to single-machine sequential parsing. These technological improvements collectively construct an automated closed loop covering the entire process of test preparation, execution, and analysis, significantly improving test execution throughput and overall efficiency, and providing efficient and stable technical support for software system quality assurance. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the method described in this invention; Figure 2 This is a flowchart illustrating the collaborative processing of test results by the task engine and the computing engine in the method described in this invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] This embodiment provides a testing method based on a distributed automated testing platform, including the following steps: In response to user operations via a graphical interface, test task configuration information is obtained, which includes at least test scripts, target load unit information, and test parameters. Based on the test task configuration information, the task engine performs environment deployment and data file distribution on each load machine in the target load group in a distributed parallel manner. After the environment deployment and data file distribution are completed, based on the test scripts and test parameters in the test task configuration information, the test task is generated by the task engine and added to the task queue for scheduling. The load test is executed in a distributed mode on the target load group, and a test result file is generated. Based on the test result file, the computing engine performs distributed parsing of the test result file to generate test performance indicators and trigger the generation of test reports.

[0022] The above method is based on a testing platform, a distributed automated testing platform with a layered architecture, mainly including: a front-end application layer, a back-end application layer, a data storage layer, and two core processing engines: a task engine and a computing engine. The front-end application layer is a web-based graphical interface, accessed and operated by users through a browser. The back-end application layer is built on Python and the Django framework, providing RESTful API interfaces for handling business logic. The data storage layer uses a MySQL database to store structured data such as user information, test configurations, and historical records, and utilizes a Redis cache database to store high-concurrency access data such as the real-time status of the load balancer and task queues. The task engine and computing engine are the core of the platform's automated and distributed processing capabilities. The task engine is mainly responsible for preparing the test environment, deploying data, and scheduling and executing test tasks. The computing engine is responsible for efficiently parsing and analyzing the test results. The platform also includes a central scheduling system, such as DSS, for unified task orchestration and distribution. This central scheduling system receives test task requests initiated by the front-end interface and coordinates their distribution to the task engine and computing engine for execution.

[0023] Figure 1 This is a schematic diagram of the specific process of the method described in this embodiment. The following is a detailed explanation in conjunction with... Figure 1 The implementation process of the method described in this embodiment will be explained in detail using the aforementioned testing platform.

[0024] Step 1: In response to the user's operation through the graphical interface, obtain the test task configuration information, which includes at least the test script, target load unit information, and test parameters.

[0025] Users operate through a front-end graphical interface. In the test script management module, they upload or select existing test scripts, such as JMeter scripts. In the load generator resource management module, they select multiple load generators from the platform-managed resource pool to form the target load generator group for this test. Subsequently, in the distributed group configuration module, they set core test parameters such as the number of test threads, loop count, and duration through a visual configuration panel. The platform integrates the user-selected test scripts, target load generator group information, and set test parameters to form structured test task configuration information.

[0026] Step 2: Based on the test task configuration information, the task engine performs environment deployment and data file distribution to each load machine in the target load group in a distributed parallel manner; After receiving the test task configuration information containing the target load group information, the task engine initiates an automated environment preparation process. This process includes two steps: environment deployment and data file distribution, both executed in a distributed parallel manner. Specifically: The environment deployment includes: concurrent interaction with each load machine based on multi-threaded pool technology, performing system parameter verification, runtime environment installation, and test tool cluster configuration. Specifically, the task engine's environment deployment module establishes concurrent connections with all load machines in the target load machine group based on multi-threaded pool technology. For each load machine, the following standardized operations are automatically executed sequentially, achieving one-click distributed initialization: System parameter verification checks the basic conditions of the load machine, such as the operating system and disk space. Runtime environment installation: Automatically installs the specified Java runtime environment (JDK). Configure the test tool cluster, deploy the specified performance testing tool such as Apache JMeter, and complete the basic configuration of the cluster mode.

[0027] Through the above concurrent operations, the traditional method of manual deployment one machine at a time is transformed into efficient batch parallel processing, which significantly shortens the preparation time of the test environment.

[0028] The data file distribution includes: acquiring the data file to be distributed associated with the test task configuration information; fragmenting the data file to be distributed; and concurrently uploading it to each load machine in the target load group using multi-threaded concurrency, supporting breakpoint resumption and consistency verification during the distribution process. Specifically, the task engine's data distribution module acquires the data file to be distributed associated with the test task configuration information using secure file transfer protocols such as SFTP. These files may include JAR packages that test scripts depend on, CSV datasets that the test needs to read, etc. For large files, the module fragments them. Then, using multi-threaded concurrent upload technology, the fragmented data blocks are simultaneously transmitted to each load machine in the target load group. The entire distribution process supports breakpoint resumption and consistency verification, meaning that after network interruption is recovered, transmission can resume from the point of interruption, ensuring the integrity of files on all load machines. This fragmented concurrent transmission mechanism improves the deployment speed of large test datasets in a distributed cluster.

[0029] Step 3: After the environment deployment and data file distribution are completed, based on the test script and test parameters in the test task configuration information, the test task is generated by the task engine and added to the task queue for scheduling. The load test is executed in a distributed mode on the target load group, and a test result file is generated. Once the environment and data are ready, the test execution phase begins. The task engine generates an executable test task based on the test script and test parameters in the test task configuration information. Task scheduling then proceeds. The test task is added to a task queue for scheduling, including: the task queue is an asynchronous task queue; test tasks are created and managed based on the asynchronous task queue; the target load group is constructed into a test cluster using a master-slave mode; and the execution status of the test task is synchronized to the front-end interface in real time. Specifically, the task queue is an asynchronous task queue, implemented using Celery on the platform. The task engine creates and manages the test task based on the asynchronous task queue. During execution, the platform uses a master-slave mode to construct the target load group into a test cluster: designating one load machine as the control node (Master) and the rest as execution nodes (Slaves). The test task is scheduled to execute load tests on this cluster in a distributed mode. Simultaneously, the task engine synchronizes the execution status of the test task, such as running, paused, or aborted, to the monitoring panel on the front-end interface in real time for users to view. This scheduling architecture, based on asynchronous queues and master-slave mode, achieves decoupling of test tasks, reliable scheduling and high-concurrency execution capabilities, and provides real-time process visibility.

[0030] After the test is completed, each load machine generates the original test result file (such as a JMeter JTL file). Specifically, the central scheduling system DSS triggers the result processing flow. The computing engine first collects, extracts, and initially organizes these result files from each load machine node using file transfer protocols such as SFTP.

[0031] Step 4: Based on the test result file, perform distributed parsing of the test result file through the computing engine to generate test performance indicators and trigger the generation of a test report.

[0032] After the test load is executed, the task engine does not passively wait for it to finish, but actively parses the execution logs, such as... Figure 2 As shown, the system determines whether the task has been completed based on the list of parsed output results. The logic is as follows: if one or more results in the output list are true, the task is considered complete.

[0033] If the determination is negative (i.e., incomplete), then further investigation is conducted to determine if an execution exception occurred. If an exception occurred, a rollback and deletion operation is performed to clean up the environment; if no exception occurred, monitoring and log parsing continue until the task is completed.

[0034] If the task is deemed complete, the result file upload process is triggered. The task engine uploads the generated raw test result file, such as a JTL file, to a dedicated parsing service via the secure file transfer protocol SFTP.

[0035] The parsing service continuously checks whether all task engines have completed uploading. Once all result files are ready, the central scheduling system triggers the computing engine to perform parsing.

[0036] The distributed parsing of the test result file by the computing engine includes: automatically detecting the test result file to be parsed through a timed scanning mechanism; calling the distributed computing framework to split the test result file into multiple data blocks and distribute them to multiple computing nodes for parallel parsing, so as to calculate at least one performance indicator among transaction processing rate (TPS), response time, and error rate.

[0037] Specifically, the scheduled scanning module within the computing engine implements minute-level scanning tasks based on APScheduler, automatically detecting test result files to be parsed in the result file storage directory. This replaces manual triggering of parsing. The engine then calls a distributed computing framework, such as Dask used on the platform. This framework splits the massive raw result file JTL into multiple data blocks and distributes them to multiple computing nodes for parallel parsing. Each computing node independently calculates performance metrics within its assigned data block, such as transaction processing speed (TPS), average response time (AVG), minimum response time (MIN), maximum response time (MAX), and error rate. The calculation results from all nodes are aggregated to generate complete test performance metrics. These metrics are indexed by the task engine number and stored in a Redis cache database for real-time querying and front-end display. The parsed complete result data is stored in a persistent database (DB) such as MySQL for historical queries, in-depth analysis, and report generation. This distributed parallel parsing architecture transforms the serial computation task of a single large file into the parallel processing of multiple subtasks, thereby significantly improving the efficiency of result parsing. Once the performance metrics are calculated, the platform automatically triggers a test report generation event, generating a test report containing charts and analytical conclusions based on the metric data, in formats such as HTML and PDF. The report is stored in the platform's database and can be managed centrally through the front-end test report aggregation and query module. Simultaneously, the system sends the report to relevant testers via email notification.

[0038] In one specific embodiment, the method further includes: integrating a multi-dimensional data statistics dashboard into the front-end interface to dynamically display the operating status, resource usage, test script inventory, and time-dimensional test execution volume analysis view of the target load units. The platform's front-end interface integrates a multi-dimensional data statistics dashboard. This dashboard can dynamically display the operating status of all target load units, such as online / offline status; resource usage, such as CPU and memory usage curves; the inventory of test scripts within the platform; and time-dimensional test execution volume analysis views, such as the test task trends for the past half month and year. This provides users with a global perspective on test assets and performance, achieving deep visualization of the testing process and results, and improving management efficiency.

[0039] This embodiment also provides a distributed automated testing platform for implementing the aforementioned testing method based on a distributed automated testing platform, including: The computing engine is configured to perform distributed parsing of test result files to generate test performance metrics and trigger test report generation. The task engine is configured to deploy the environment and distribute data files to the target load group in a distributed parallel manner based on the test task configuration information, generate test tasks based on the test task configuration information, add the test tasks to the task queue for scheduling, and execute load tests on the target load group in a distributed mode to generate the test result files.

[0040] The computing engine includes: The timed scanning module is configured to automatically detect the test result files to be parsed; The distributed parsing module is configured to call a distributed computing framework to split the test result file into multiple data blocks for parallel parsing and to calculate performance metrics.

[0041] The task engine includes: The environment deployment module is configured to deploy the environment of the target load unit in a distributed and parallel manner; The data distribution module is configured to distribute data files to the target load group in a distributed and parallel manner; The task scheduling module is configured to schedule test scripts to be executed in a distributed mode on the target load group.

[0042] Through the collaborative work of the task engine and computing engine, as well as the cooperation with the front-end and back-end layers, this invention achieves full automation from test resource management, parameter configuration, task scheduling, process monitoring to result analysis, and builds an efficient, stable and scalable distributed automated testing solution.

[0043] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0045] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0047] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A test method based on a distributed automated test platform, characterized in that, Includes the following steps: In response to user operations via a graphical interface, test task configuration information is obtained, which includes at least test scripts, target load unit information, and test parameters. Based on the test task configuration information, the task engine performs environment deployment and data file distribution on each load machine in the target load group in a distributed parallel manner. After the environment deployment and data file distribution are completed, based on the test scripts and test parameters in the test task configuration information, the test task is generated by the task engine and added to the task queue for scheduling. The load test is executed in a distributed mode on the target load group, and a test result file is generated. Based on the test result file, the computing engine performs distributed parsing of the test result file to generate test performance indicators and trigger the generation of test reports.

2. The test method based on the distributed automation test platform according to claim 1, wherein, The environment deployment includes: concurrent interaction with each load machine based on multi-threaded pool technology, performing system parameter verification, runtime environment installation, and test tool cluster configuration.

3. The test method based on the distributed automation test platform according to claim 2, characterized in that, The data file distribution includes: Obtain the data file to be distributed that is associated with the test task configuration information; The data file to be distributed is fragmented and uploaded concurrently to each load machine in the target load group through multi-threading, and the distribution process supports breakpoint resumption and consistency verification.

4. The test method based on the distributed automation test platform according to claim 1, wherein, The test task is added to the task queue for scheduling, including: the task queue is an asynchronous task queue; test tasks are created and managed based on the asynchronous task queue; the target load group is built into a test cluster using a master-slave mode; and the execution status of the test task is synchronized to the front-end interface in real time.

5. The method of claim 1, wherein, The distributed parsing of the test result file using a computing engine includes: The test result files to be parsed are automatically detected through a timed scanning mechanism; The distributed computing framework is invoked to split the test result file into multiple data blocks and distribute them to multiple computing nodes for parallel parsing, in order to calculate at least one performance metric among transaction processing rate (TPS), response time, and error rate.

6. The test method based on the distributed automation test platform according to claim 5, characterized in that, The timed scanning mechanism is based on APScheduler to implement scanning tasks with a minute-level cycle.

7. The test method based on the distributed automation test platform according to claim 1, wherein, The method further includes: The front-end interface integrates a multi-dimensional data statistics dashboard, dynamically displaying the operating status, resource usage, test script inventory, and time-based test execution volume analysis view of the target load unit.

8. A distributed automated testing platform, characterized by, A testing method based on a distributed automated testing platform, as described in any one of claims 1-7, comprises: The computing engine is configured to perform distributed parsing of test result files to generate test performance metrics and trigger test report generation. The task engine is configured to deploy the environment and distribute data files to the target load group in a distributed parallel manner based on the test task configuration information, generate test tasks based on the test task configuration information, add the test tasks to the task queue for scheduling, and execute load tests on the target load group in a distributed mode to generate the test result files.

9. The distributed automated test platform of claim 8, wherein, The computing engine includes: The timed scanning module is configured to automatically detect the test result files to be parsed; The distributed parsing module is configured to call a distributed computing framework to split the test result file into multiple data blocks for parallel parsing and to calculate performance metrics.

10. The distributed automated test platform of claim 8, wherein, The task engine includes: The environment deployment module is configured to deploy the environment of the target load unit in a distributed and parallel manner; The data distribution module is configured to distribute data files to the target load group in a distributed and parallel manner; The task scheduling module is configured to schedule test scripts to be executed in a distributed mode on the target load group.

Images