Statistical method and system based on dynamic distributed unit test coverage rate
By dividing unit tests into independent modules and building containerized images, and using distributed clusters for intelligent scheduling and result aggregation, the problems of long unit test times and environment conflicts are solved, achieving efficient test execution and admission decisions.
Patent Information
- Application Number
- CN202511660962.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-13
- Publication Date
- 2026-02-24
AI Technical Summary
In existing technologies, unit testing is concentrated on execution on a single machine, resulting in long testing times, environment configuration conflicts, and resource contention, which makes it difficult to meet the needs of rapid iteration in large-scale projects.
Unit tests in the project code are divided into independent modules, containerized images are built, and intelligent scheduling and result aggregation are performed through a distributed test execution cluster to generate test reports to determine code admission.
It enables isolation and customized configuration of the test environment, shortens test time, avoids resource contention and environment conflicts, and improves test efficiency and the level of automation of admission judgment.
Smart Images
Figure CN121560740A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of software testing technology, and in particular to a method and system for statistical analysis of dynamic distributed unit test coverage. Background Technology
[0002] In modern software development, unit testing is a crucial step in ensuring code quality. Current technologies typically concentrate unit tests for all modules on a single machine, leading to problems such as long test execution times, conflicts in environment configurations between different modules, and test failures caused by resource contention, making it difficult to meet the needs of rapid iteration in large-scale projects.
[0003] In current software development models, unit test coverage is a crucial and effective metric for evaluating the quality of unit tests. In project management, unit test coverage is used to measure whether branch code is allowed to be merged into the main branch, thus ensuring code quality. However, in daily use, high demands are placed on the execution time of unit tests and the time required to calculate coverage. Current development models typically use a large repository model, where all project code is stored in one large repository for easy access. This approach results in a very large number of unit tests, leading to the following problems:
[0004] 1. Unit test execution time is particularly long, which in turn makes unit test coverage statistics take a particularly long time.
[0005] 2. Unit tests for different modules are executed on the same machine, which requires that the execution machine supports the execution of unit tests for each module. However, the execution environments of different unit test modules may conflict.
[0006] 3. When unit tests are executed on the same machine, there may be resource contention and conflicts during the coverage collection process, which may lead to the failure of coverage execution. Summary of the Invention
[0007] This application addresses the technical problems existing in the prior art by providing a method for calculating coverage based on dynamic distributed unit tests, including the following steps:
[0008] S1: Based on the execution environment and module affiliation of the unit tests, divide the unit tests in the project code into several independent modules, and build a containerized image for each independent module;
[0009] S2: Build a distributed test execution cluster, respond to code commit events, trigger the test process, and identify the module to which the code changes belong;
[0010] S3: Based on the identified modules, intelligently schedule the corresponding cluster nodes to execute unit tests;
[0011] S4: The master node summarizes the test results and coverage data of each node, generates a test report, and decides whether the code is allowed to enter the test based on the coverage and pass rate in the report.
[0012] Furthermore, in S1, the unit tests in the project code are divided into several independent modules, specifically by dividing the unit tests into several modules and obtaining a file list for each module.
[0013] Furthermore, in S1, the method for building containerized images for each independent module is as follows: different containerized images are built for unit tests, and each containerized image contains a unit test environment and a unit test coverage collection module.
[0014] Furthermore, in S2, the distributed test execution cluster consists of several test nodes, each of which is automatically created based on the image.
[0015] Furthermore, after the node is created, the unit test environment, initial test files, and code coverage monitoring and collection tools are initialized;
[0016] The test node maintains communication with the master node via WebSocket, supporting heartbeat detection and state synchronization.
[0017] Furthermore, step S2 also includes:
[0018] Configure a Webhook in the code management platform to trigger the unit test startup script when code is committed; parse the code change set and match the module to which the changed code file belongs.
[0019] Furthermore, step S3, based on the identified modules, intelligently schedules the corresponding cluster nodes to execute unit tests, including: enabling intelligent scheduling based on the matched modules, enabling the execution of the corresponding unit tests, and collecting coverage data from the nodes.
[0020] At the same time, the code is deployed to start executing the unit tests of the corresponding module. After the unit tests are completed, the corresponding coverage is collected.
[0021] The method for calculating coverage based on dynamic distributed unit test as described in claim 1 is characterized in that, in step S4, generating a test report and determining whether the code is admitted based on the coverage and pass rate of the report includes:
[0022] The generated test report includes:
[0023] Summarize the individual coverage reports generated by each node and merge them to generate an overall coverage report;
[0024] Summarize the unit test pass rate reports generated at each node and merge them to generate an overall unit test pass rate report;
[0025] Calculate incremental unit test coverage based on the number of lines of code covered and the number of lines of code to be scanned.
[0026] The method call chain of the current version is compared and analyzed with other versions to obtain information on the differences in the code, which is used to assess the impact of the code changes.
[0027] At the same time, the system analyzes whether the unit test coverage and pass rate meet the code admission criteria: if the admission criteria are met, the code submission is successful; if the admission criteria are not met, the system prompts the user to add unit tests and resubmit the code.
[0028] This application also provides a dynamic distributed unit test coverage statistics system, applied to the dynamic distributed unit test coverage statistics method, comprising a module partitioning unit, a cluster construction and triggering unit, an intelligent scheduling and execution unit, and a summarization and admission unit, wherein:
[0029] Module division: Based on the execution environment and module affiliation of unit tests, the unit tests in the project code are divided into several independent modules, and a containerized image is built for each independent module;
[0030] Cluster construction and triggering unit: Builds a distributed test execution cluster, responds to code commit events, triggers the test process, and identifies the module to which the code changes belong;
[0031] Intelligent scheduling and execution unit: Based on the identified modules, the corresponding cluster nodes are intelligently scheduled to execute unit tests;
[0032] Summary and Admission Unit: The master node summarizes the test results and coverage data of each node, generates a test report, and decides whether the code is admitted based on the coverage and pass rate in the report.
[0033] This application also provides a storage medium storing a computer program, which is executed by a processor as described in the dynamic distributed unit test coverage statistics method.
[0034] This application also provides an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the dynamic distributed unit test coverage statistics method as described above.
[0035] In practical applications, the system and modules disclosed in this application can be deployed on a single target server, or each module can be deployed independently on different target servers. In particular, as needed, to provide more powerful computing capabilities, modules can also be deployed on cluster target servers.
[0036] Therefore, the technical effects achieved by the technical approach adopted in this application are as follows:
[0037] 1. By dividing into modules and building independent environments, the test environment can be isolated and customized. Each module has its own unit test execution environment, and the module unit test execution environment can be configured independently.
[0038] 2. By utilizing distributed clusters and intelligent scheduling, test tasks can be executed in parallel, significantly shortening test time. Modular unit test execution no longer experiences resource contention or conflicts.
[0039] 3. By using dynamic triggering and result aggregation mechanisms, the automatic collection of coverage data and access judgment are realized, thereby improving the level of process automation.
[0040] To provide a clearer and more comprehensive understanding of this application, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0042] Figure 1 This is a flowchart illustrating an embodiment of the dynamic distributed unit test coverage statistics method of this application. Detailed Implementation
[0043] This method dynamically builds different unit test execution environments for different modules in a distributed manner. After the unit tests are executed in a specific execution environment, the unit test coverage is collected. The results of the executed tests and the received coverage are submitted to the summary master node for aggregation, and then submitted to the continuous integration platform. The continuous integration platform analyzes whether the code submission meets the code admission criteria in terms of test case success rate and code coverage.
[0044] The specific technical solution is as follows: a method for calculating the coverage of dynamic distributed unit tests, including the following steps:
[0045] S1: Based on the execution environment and module affiliation of the unit tests, divide the unit tests in the project code into several independent modules, and build a containerized image for each independent module;
[0046] S2: Build a distributed test execution cluster, respond to code commit events, trigger the test process, and identify the module to which the code changes belong;
[0047] S3: Based on the identified modules, intelligently schedule the corresponding cluster nodes to execute unit tests;
[0048] S4: The master node summarizes the test results and coverage data of each node, generates a test report, and decides whether the code is allowed to enter the test based on the coverage and pass rate in the report.
[0049] Please see Figure 1 The technical solutions of this application will be described in detail with reference to various preferred embodiments.
[0050] S1: Environment Preparation and Module Division: Based on the execution environment and module affiliation of the unit tests, the unit tests in the project code are divided into several independent modules, and a containerized image is built for each independent module.
[0051] Dividing unit tests in the project code into several independent modules specifically involves dividing the unit tests into several modules, such as M1, M2, M3, M4, etc., and obtaining a file list for each module.
[0052] The method for building containerized images for each independent module is as follows: Different containerized images (Docker images) are built for unit tests, namely D1, D2, D3, D4... Each containerized image contains a unit test environment and a unit test coverage collection module. The images are built based on containerization technologies such as Docker, ensuring environment isolation and customized configuration.
[0053] During the module partitioning phase, multi-dimensional partitioning can be carried out based on the code repository structure, dependencies, and team division of labor. For example, in addition to partitioning by business function, modularization can also be carried out based on technical level (such as data access layer, service layer, control layer) or test type (such as rapid test suite, deep integration test suite).
[0054] During the partitioning process, static code analysis tools (such as SonarQube) can be used to automatically identify module boundaries and dependencies, generating module mapping files. The containerized image building for each module can adopt a layered optimization strategy: the base layer contains the operating system and runtime environment; the middle layer installs language-specific support tools (such as JVM, Node.js) and dependency libraries; and the top layer deploys module-specific test scripts, configuration files, and coverage collection agents (such as JaCoCo for Java and Coverage.py for Python).
[0055] Images are versioned using Dockerfiles and stored in a private registry, ensuring environment consistency and reproducibility. Furthermore, support for multi-stage builds further optimizes image size and improves pull and startup speeds.
[0056] Through meticulous module partitioning and containerized encapsulation, complete isolation of the testing environment is achieved, fundamentally eliminating test failures caused by environment dependency conflicts between different modules. Each module has an independent execution sandbox, allowing for customized environments for specific technology stacks (such as different JDK versions or Python environments), improving the accuracy and flexibility of testing. Versioned management and layered optimization of containerized images not only simplify the environment deployment process but also significantly accelerate image building and distribution, laying a solid foundation for subsequent distributed parallel testing.
[0057] S2: Cluster Construction and Dynamic Triggering: Build a distributed test execution cluster, respond to code commit events, trigger the test process, and identify the module to which the code changes belong.
[0058] The distributed test execution cluster consists of several test nodes, each automatically created based on a containerized Docker image. After node creation, the following implementation details are also included:
[0059] Initialize the unit test environment (such as database, dependency libraries), the initial test files, and start the code coverage monitoring and collection tools (such as JaCoCo, Coverage.py, etc.);
[0060] The distributed cluster is built using container orchestration platforms such as Kubernetes for automated management, enabling elastic scaling of nodes. The master node acts as the brain of the cluster, responsible for resource scheduling and task coordination.
[0061] Test nodes maintain communication with the master node via WebSocket, supporting heartbeat detection and status synchronization. The WebSocket long connection is not only used for heartbeat keeping, but also enables real-time log streaming, test progress push and remote debugging support, greatly improving the observability of the test process.
[0062] Configure a webhook in a code management platform (such as GitHub / GitLab) to trigger the unit test startup script when code is committed. After the code commit event is triggered via the webhook, the master node receives the event payload and calls the built-in changeset resolver. This changeset resolver uses the Git diff algorithm to accurately identify added, deleted, or modified files, and, combined with the module-file mapping table generated in step S1, uses regular expressions or path matching rules to quickly locate the affected modules.
[0063] Parse the code changeset (obtain the changed code through Git changes), match the changed code with the module file list in step S1, and identify the module to which the changed code file belongs.
[0064] To improve matching efficiency, data structures such as Bloom filters can be introduced for fast queries, or file indexes can be created for large repositories. The node initialization process can be further refined: in addition to starting the coverage tool, test data can be preloaded, Mock services can be configured, and database connection pools can be established to ensure that the test execution environment is "ready to use out of the box".
[0065] This demonstrates that the event-driven dynamic triggering mechanism achieves seamless integration between code submission and test initiation, significantly shortening the feedback cycle from development to verification. Intelligent change identification and module matching ensure accurate deployment of test resources, avoiding resource waste associated with full-scale testing. The elastic cluster, combined with container orchestration technology, dynamically adjusts computing resources based on test load, guaranteeing testing efficiency under high concurrency while optimizing resource utilization. The introduction of WebSocket makes the entire testing process highly transparent, facilitating real-time monitoring of progress and rapid problem identification by developers and testers, thus improving team collaboration efficiency.
[0066] S3: Intelligent Scheduling and Distributed Execution: Based on the identified modules, intelligently schedule the corresponding cluster nodes to execute unit tests.
[0067] The intelligent scheduling corresponding cluster node execution unit test includes: according to the matched module, intelligent scheduling will be enabled, the corresponding unit test will be executed, and the coverage collection node will be carried out.
[0068] At the same time, the code is deployed to start executing the unit tests of the corresponding module. After the unit tests are completed, the corresponding coverage is collected.
[0069] The intelligent scheduler is the core of this step, and its scheduling decisions can be based on various strategies. The most basic strategy is "module affinity" scheduling, which distributes test tasks to nodes that have already loaded the corresponding module image, reducing environment preparation time. More advanced strategies can comprehensively consider factors such as real-time node load, task priority, and historical execution time, using weighted round-robin or least-load-first algorithms for dynamic allocation.
[0070] For modules with complex dependencies, the scheduler can perform topology sorting based on a predefined dependency graph to ensure that dependent modules are tested first. Task delivery can be decoupled through message queues (such as RabbitMQ and Kafka) to ensure task reentrancy and fault tolerance.
[0071] During test execution, nodes employ a "collect-as-you-go" approach, writing coverage data to shared storage (such as NFS or S3) in real-time via a proxy. Upon completion, in addition to reporting results, nodes automatically clean up test remnants (such as temporary files and database rollbacks) to maintain a clean environment. For further acceleration, test cases can be parallelized, and caching mechanisms (such as Test Result Caching) can be used to skip unchanged tests.
[0072] This application's intelligent scheduling strategy optimally matches computing resources with test tasks, maximizing the cluster's parallel processing capabilities and reducing the execution time of large-scale test tasks from hours to minutes. The message queue-based task distribution mechanism improves system throughput and resilience, ensuring that tasks are not lost and are rescheduled even when some nodes fail.
[0073] The simultaneous execution of tests and coverage collection avoids potential issues such as environmental changes or data loss that may occur during post-collection, ensuring the accuracy and completeness of coverage data. The entire execution process is highly automated, requiring no manual intervention, significantly improving the efficiency and reliability of the continuous integration process. Furthermore, the parallel execution of unit tests by test nodes avoids resource contention and environmental conflicts, greatly enhancing testing efficiency.
[0074] S4: Results Summary and Admission Decision: The master node summarizes the test results and coverage data of each node, generates a test report, and decides whether the code is admitted based on the coverage and pass rate in the report.
[0075] The master node monitors the execution status of all test nodes. Once all deployed unit tests have been executed and coverage data has been collected, the test results and coverage statistics are merged and analyzed.
[0076] The generated test report includes:
[0077] Summarize the individual coverage reports generated by each node and merge them to generate an overall coverage report;
[0078] Summarize the unit test pass rate reports generated at each node and merge them to generate an overall unit test pass rate report;
[0079] Calculate incremental unit test coverage based on the number of lines of code covered and the number of lines of code to be scanned.
[0080] The method call chain of the current version is compared and analyzed with other versions to obtain information on the differences in the code, which is used to assess the impact of the code changes.
[0081] At the same time, the system analyzes whether the unit test coverage and pass rate meet the code admission criteria: if the admission criteria are met, the code submission is successful; if the admission criteria are not met, the system prompts the user to add unit tests and resubmit the code.
[0082] During the results aggregation phase, the master node acts as both a data aggregator and analyst. It pulls standardized result files from various nodes (such as JUnit XML format test results and JaCoCo's .exec coverage data files).
[0083] Coverage merging is not a simple file concatenation process. Instead, it is performed by a dedicated "coverage merging engine" that parses each .exec file, deduplicates and aggregates coverage metrics such as lines of code, branches, and methods, and finally generates a unified overall report in HTML or XML format.
[0084] The incremental coverage statistics are one of the innovations of this application. Its core is to accurately identify the "number of lines of code to be scanned", which is usually achieved by comparing the difference between the current commit and the target branch (such as the main branch).
[0085] This can be achieved using bytecode analysis tools (such as ASM), which construct a call graph between the current version and the baseline version. By using graph difference algorithms, newly added, modified, or deleted call relationships can be accurately located, thereby identifying potential risk areas that may be affected by code changes but have not been covered by tests.
[0086] Admission decisions are configurable, allowing projects to set different thresholds (e.g., overall coverage >= 80%, new code coverage >= 90%, test pass rate 100%). Decision results can be automatically fed back to the code merge request via a comment bot (e.g., GitHub Bot), forming a quality gate.
[0087] This application's multi-dimensional report aggregation and in-depth analysis provide comprehensive and accurate data support for code quality assessment.
[0088] Incremental coverage statistics shift the focus from the massive full codebase to the current change, making coverage metrics more instructive and effectively avoiding the predicament of developers blindly adding test cases in an attempt to improve global coverage.
[0089] By comparing and analyzing call graphs, we can gain insights into the deeper impact of code changes, identify potential risk points, and achieve an advancement from "code coverage" to "logic coverage." Automated access decisions and real-time feedback seamlessly embed quality checks into the development process, forming an efficient quality control closed loop. This significantly reduces the risk of substandard code flowing into the main branch, ensuring the intrinsic quality of the software product from a process perspective.
[0090] Based on the methods described in the above embodiments, this application also provides a dynamic distributed unit test coverage statistics system, including a module partitioning unit, a cluster construction and triggering unit, an intelligent scheduling and execution unit, and a summarization and admission unit, wherein:
[0091] Module division: Based on the execution environment and module affiliation of unit tests, the unit tests in the project code are divided into several independent modules, and a containerized image is built for each independent module;
[0092] Cluster construction and triggering unit: Builds a distributed test execution cluster, responds to code commit events, triggers the test process, and identifies the module to which the code changes belong;
[0093] Intelligent scheduling and execution unit: Based on the identified modules, the corresponding cluster nodes are intelligently scheduled to execute unit tests;
[0094] Summary and Admission Unit: The master node summarizes the test results and coverage data of each node, generates a test report, and decides whether the code is admitted based on the coverage and pass rate in the report.
[0095] The application embodiment also provides a storage medium storing a computer program, which is executed by a processor to perform the dynamic distributed unit test coverage statistics method as described above.
[0096] This application also provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the dynamic distributed unit test coverage statistics method as described above.
[0097] It should be noted that those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, which may include, but is not limited to, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0098] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for calculating unit test coverage based on dynamic distributed testing, characterized in that, Including the following steps: S1: Based on the execution environment and module affiliation of the unit tests, divide the unit tests in the project code into several independent modules, and build a containerized image for each independent module; S2: Build a distributed test execution cluster, respond to code commit events, trigger the test process, and identify the module to which the code changes belong; S3: Based on the identified modules, intelligently schedule the corresponding cluster nodes to execute unit tests; S4: The master node summarizes the test results and coverage data of each node, generates a test report, and decides whether the code is allowed to enter the test based on the coverage and pass rate in the report.
2. The method for calculating coverage based on dynamic distributed unit test as described in claim 1, characterized in that, In S1, the unit tests in the project code are divided into several independent modules. Specifically, the unit tests are divided into several modules, resulting in a file list for each module.
3. The method for calculating coverage based on dynamic distributed unit test as described in claim 1, characterized in that the steps are as follows: In S1, the method for building containerized images for each independent module is as follows: different containerized images are built for unit tests, and each containerized image contains a unit test environment and a unit test coverage collection module.
4. The method for calculating coverage based on dynamic distributed unit test as described in claim 3, characterized in that, In step S2, the distributed test execution cluster consists of several test nodes, each of which is automatically created based on the image.
5. The method for calculating coverage based on dynamic distributed unit test as described in claim 4, characterized in that: After the node is created, initialize the unit test environment, the initial test files, and start the code coverage monitoring and collection tool; The test node maintains communication with the master node via WebSocket, supporting heartbeat detection and state synchronization.
6. The method for calculating coverage based on dynamic distributed unit test as described in claim 1, characterized in that, Step S2 also includes: Configure a Webhook in the code management platform to trigger the unit test startup script when code is committed; parse the code change set and match the module to which the changed code file belongs.
7. The method for calculating coverage based on dynamic distributed unit test as described in claim 1, characterized in that, Step S3, based on the identified modules, intelligently schedules the corresponding cluster nodes to execute unit tests, including: Based on the matched modules, intelligent scheduling will be enabled, and corresponding unit tests will be executed, and coverage collection nodes will be established. At the same time, the code is deployed to start executing the unit tests of the corresponding module. After the unit tests are completed, the corresponding coverage is collected.
8. The method for calculating coverage based on dynamic distributed unit test as described in claim 1, characterized in that, In step S4, a test report is generated, and the code is admitted based on the report's coverage and pass rate, including: The generated test report includes: Summarize the individual coverage reports generated by each node and merge them to generate an overall coverage report; Summarize the unit test pass rate reports generated at each node and merge them to generate an overall unit test pass rate report; Calculate incremental unit test coverage based on the number of lines of code covered and the number of lines of code to be scanned. The method call chain of the current version is compared and analyzed with other versions to obtain information on the differences in the code, which is used to assess the impact of the code changes. Analyze whether the unit test coverage and pass rate meet the code admission criteria: if the admission criteria are met, the code submission is successful; if the admission criteria are not met, the system prompts the user to add unit tests and resubmit the code.
9. A dynamic distributed unit test coverage statistics system, applied to the dynamic distributed unit test coverage statistics method described in any one of claims 1-8, characterized in that, The module division unit, cluster construction and triggering unit, intelligent scheduling and execution unit, and aggregation and admission unit are as follows: Module division: Based on the execution environment and module affiliation of unit tests, the unit tests in the project code are divided into several independent modules, and a containerized image is built for each independent module; Cluster construction and triggering unit: Builds a distributed test execution cluster, responds to code commit events, triggers the test process, and identifies the module to which the code changes belong; Intelligent scheduling and execution unit: Based on the identified modules, the corresponding cluster nodes are intelligently scheduled to execute unit tests; Summary and Admission Unit: The master node summarizes the test results and coverage data of each node, generates a test report, and decides whether the code is admitted based on the coverage and pass rate in the report.
10. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform a dynamic distributed unit test coverage statistics method as described in any of claims 1-8.