A method and system for generating an automated test report based on multi-assertion instance aggregation

Abstract

The application discloses a kind of based on the automatic test report generation method and system of multiple assertor instance aggregation, it is related to software automation testing technical field, the method of the present application includes class variable global registration and assertion data fingerprint generation, two-way tracing and pytest hook cooperative data acquisition and intelligent analysis and comprehensive report generation;The system of the present application includes data acquisition layer, data storage layer, intelligent analysis layer and application service layer, complete data processing closed loop is formed by data flow connection, the leap from simple report generation to assertion level depth analysis is realized;The present application is especially suitable for pytest framework under the scene of unified collection and report generation of assertion result across module, cross test function, especially for the technical problem that multiple assertions are difficult to aggregate statistics in power terminal automation testing project, with significant practical application value.

Description

Technical Field

[0001] This invention relates to the field of software automated testing technology, specifically to an automated test report generation method and system based on the aggregation of multiple assertion instances. Background Technology

[0002] In the field of automated testing, test reports are the core basis for measuring test quality and analyzing test results. As the software industry continues to mature, people's requirements for software quality are also constantly increasing. Therefore, it is becoming increasingly important to find ways to both cover software testing needs and save on human resources and financial costs.

[0003] Existing test report generation solutions primarily rely on pytest's native mechanism or third-party plugins (such as Allure and JUnit XML). However, these solutions have significant limitations when dealing with complex testing scenarios. Firstly, the statistical granularity is too coarse: existing solutions use test cases as the smallest statistical unit, failing to refine down to the assertion level. Multiple assertions within a single test function cannot be distinguished, resulting in insufficiently detailed test results and making it difficult for testers to quickly pinpoint specific issues. Secondly, cross-function state isolation is difficult: pytest's framework limitations make it difficult to persist shared state across test functions. Assertion instance instances cannot be effectively shared and uniformly managed across different test functions, affecting the completeness and accuracy of assertion aggregation. Thirdly, script name recognition is difficult: the association between test scripts and log files relies on manual configuration or conventions, lacking an automated association mechanism. In large projects, maintenance costs are extremely high, and configuration errors can easily lead to association failures. Fourth, the assertion context is lost: when an assertion fails, the creation context and call chain of the assertion cannot be traced, making it difficult for testers to understand the environment state, variable values, and call path when the assertion is executed, increasing the difficulty of problem localization; fifth, the assertion semantics are ambiguous: the test intent behind the assertion cannot be identified, the test purpose is difficult to understand, and the readability and analytical value of the test report are limited; sixth, the assertion dependencies are unclear, the shared variables and dependencies between assertions cannot be identified, and when the test script is changed, it is difficult to assess the scope of the change and to find test silos. Summary of the Invention

[0004] This invention provides an automated test report generation method and system based on the aggregation of multiple assertion instances. Without modifying the test code, it automatically establishes a mapping relationship from instances to scripts and then to reports, and outputs a unified test report at the script level. This effectively solves the technical problem of difficulty in aggregating and statistically analyzing multiple assertion instances in the prior art, and realizes a leap from simple report generation to in-depth analysis at the assertion level.

[0005] This invention provides the following technical solution:

[0006] In a first aspect, the present invention provides an automated test report generation method based on multi-assertor instance aggregation, characterized by comprising the following steps:

[0007] Step 1: When the pytest test session starts, the system creates a class variable dictionary in the assertion class as a global registry, and registers the assertion instances created by the test function into the class variable dictionary with the assertion name as the key and the instance reference as the value, so as to achieve unified management of instances across the pytest scope and generate assertion fingerprints.

[0008] Step 2: Automatically establish the association between assertion instances and test scripts through a dual mechanism of log processor path backtracking and call stack depth tracing; during the test execution phase, the runtest_makereport hook of the pytest framework is triggered in the call phase of each test, traversing the assertion instances in the global registry to collect assertion context information, statistics, fingerprint information and intent information;

[0009] Step 3: When the pytest session ends, intelligent analysis is triggered through the sessionfinish hook. This includes building a behavioral profile for each assertion based on historical execution data, building a dependency network between scripts based on shared variables and co-occurrence relationships, and aggregating statistical data of all assertion instances by script name, outputting a unified test report at the script level.

[0010] Secondly, the present invention provides an automated test report generation system based on multi-assertor instance aggregation, comprising:

[0011] The data acquisition layer includes a context capturer, a statistics capturer, a fingerprint capturer, and an intent capturer, which are used to collect assertion context information, assertion statistics, fingerprint information, and intent information from the test execution process, respectively.

[0012] The data storage layer is used to store the assertion registry, script statistical aggregation data, fingerprint database, and profile database; the assertion registry adopts a class variable dictionary structure to achieve global sharing;

[0013] The intelligent analysis layer includes a fingerprint matching engine, an intent recognition engine, a dependency graph engine, and a profile building engine, which are used to perform fingerprint matching analysis, intent recognition analysis, dependency graph analysis, and profile building analysis on the collected data, respectively.

[0014] The application service layer is used to generate test reports, intent analysis reports, graph reports, and profile summary reports.

[0015] The present invention has the following beneficial effects:

[0016] 1. This invention achieves unified management of assertion instances across test functions and pytest scopes through a global registration mechanism for assertion class variables. The registration operation has low time complexity and requires no modification to any test code, achieving zero-intrusion script-level assertion aggregation. Through a dual mechanism of log processor path backtracking and call stack depth tracing, the association between assertion instances and test scripts is automatically established, avoiding the maintenance costs of manually configuring script names and ensuring the accuracy and completeness of the mapping relationship even in abnormal scenarios.

[0017] 2. This invention utilizes assertion data fingerprinting technology to generate unique and stable assertion fingerprints based on four-tuple fusion, supporting assertion tracking, clustering, and regression analysis across sessions and test runs, providing fundamental identification capabilities for continuous analysis at the assertion level. Through assertion intent recognition technology, this invention automatically infers the test intent behind assertions, generating semantic tags with high intent recognition accuracy, significantly reducing manual annotation costs and improving the readability and analytical value of test reports. Through assertion behavior profiling technology, a multi-dimensional behavioral feature vector is constructed, classifying assertions into five behavioral tags: STABLE, UNSTABLE, FLAKY, CRITICAL, and NEW, supporting assertion-level problem prediction, significantly reducing problem discovery time, and improving profiling problem prediction accuracy. Through assertion dependency graph technology, an inter-script dependency network is constructed based on shared variable semantics, identifying test silos and the scope of change impact, providing decision support for test maintenance. Through pytest hook collaborative aggregation technology, deeply coupled with the pytest lifecycle, unified aggregation output is achieved at the end of the session, achieving 100% aggregation coverage, supporting session-level incremental aggregation, and avoiding memory leaks during long-term operation. Attached Figure Description

[0018] Figure 1 This is a diagram showing the overall architecture of the system of the present invention.

[0019] Figure 2 This is a flowchart illustrating the assertion data fingerprint generation process of the present invention.

[0020] Figure 3 This is a schematic diagram of the bidirectional traceability mechanism in this invention.

[0021] Figure 4 This is a flowchart illustrating the method for constructing an assertion behavior profile according to the present invention.

[0022] Figure 5 This is a flowchart illustrating the assertion intent recognition process of the present invention.

[0023] Figure 6 This is a schematic diagram of the assertion-dependent graph construction mechanism of the present invention.

[0024] Figure 7This is a flowchart of the pytest hook collaboration timing in this invention. Detailed Implementation

[0025] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.

[0026] To facilitate understanding of this invention, the following terms are briefly introduced:

[0027] Pytest is one of the most popular and powerful testing frameworks in the Python ecosystem. It aims to make writing tests simple, efficient, and scalable, handling everything from simple unit tests to complex functional tests. The execution of a test case goes through three phases: setup, call, and teardown. The `pytest_runtest_makereport` hook is called once after each phase. The `runtest_makereport` hook in the Pytest framework acts like an "event listener," automatically invoked at a specific stage of each test case's execution to generate and return a test report object for that stage. The `sessionfinish` hook in the Pytest framework is a crucial "final destination" in the Pytest lifecycle, used to perform some global, one-time cleanup work. It is called after all test cases have finished executing and before Pytest returns the final result (success or failure) to the operating system.

[0028] Example 1

[0029] Please see Figure 1-7 As shown, an automated test report generation method based on multi-assertor instance aggregation includes the following steps:

[0030] Step 1: Global Registration of Class Variables and Generation of Assertion Data Fingerprints: When the pytest test session starts, the system automatically initializes the data acquisition layer, data storage layer, intelligent analysis layer, and application service layer in the four-layer architecture. At the same time, it creates a class variable dictionary in the assertion class as a global registry. When the test function creates an assertion instance, the instance initialization method automatically completes the following operations: initializes instance attributes including the assertion name, creates a log recorder and establishes a log directory, and clears the statistical counter; registers the assertion name as the key and the instance reference as the value in the class variable dictionary, achieving global registration with O(1) time complexity, which is automatically shared across all scopes (function, class, module, session) of pytest; and generates a unique and stable 32-bit hexadecimal assertion fingerprint based on the four-tuple of assertion method name, parameter signature, comparison value hash, and call context hash. This step achieves zero-intrusion design, and the test code does not need any modification, minimizing the intrusion on the existing test architecture and improving the efficiency of project implementation.

[0031] Assertion fingerprint generation employs assertion data fingerprinting technology, which uses a four-tuple of assertion method name, parameter signature, comparison value hash, and call context hash to generate a unique and stable assertion fingerprint. The fingerprint possesses three key characteristics: uniqueness, stability, and traceability. Different assertions will not produce the same fingerprint; multiple executions of the same assertion will produce the same fingerprint; and the fingerprint can be reverse-analyzed to retrieve the original features. The fingerprint is a 32-bit hexadecimal string, facilitating storage and retrieval. It serves as a unique identifier for assertions across sessions and test runs, and supports advanced functions such as assertion clustering, change detection, and regression analysis.

[0032] The fingerprint generation formula is:

[0033] ;

[0034] Where M is the assertion method name, such as equal or true, used to assert different expected test results; P is the type label string sorted by parameter name; V is the comparison value hash, which is obtained by serializing the expected value and the actual value, calculating the SHA256 hash, and taking the first 16 bits; C is the context hash, which is obtained by combining the called function name and the call depth, calculating the SHA256 hash, and taking the first 8 bits. The vertical bar (or bar) is used as the separator. The fingerprint generated by this formula is unique and stable, forming the basis for cross-session assertion tracing. The formula execution can be broken down into three steps. First, the four feature values ​​M, P, V, and C are concatenated into a long string using vertical bars as separators, thus integrating the multi-feature characteristics into a single entity. Second, a SHA256 hash operation is performed on this long string to generate an irreversible, collision-resistant 256-bit (64 hexadecimal characters) digest. Third, the first 32 hexadecimal characters are extracted using Python string slicing to shorten the fingerprint length for easier storage and retrieval. The final FP is a hexadecimal string of length 32.

[0035] Step 2, Two-way tracing and pytest hook collaborative data collection: During the test execution phase, pytest hooks collaboratively collect four types of core data to ensure complete capture of assertion context.

[0036] Log processor path reverse mapping to establish script association: Traverse the log processors of the assertioner instance, extract the parent directory name from the log file path as the script file name, establish a reverse tracing mapping from the assertioner instance to the test script, and provide a foundation for regression analysis in continuous integration scenarios.

[0037] Call stack depth tracing supplements the association: As a supplementary mechanism to log path backtracking, the `inspect` module in Python traverses the call stack, skipping the assertion module itself and extracting the filename of the first external script as the script filename. This ensures that script association can still be completed even in abnormal scenarios such as irregular log directory structures or log processor cleanup. The two methods complement each other, ensuring the integrity and accuracy of the mapping relationship and avoiding the failure risk of a single tracing method in abnormal scenarios.

[0038] The pytest hook extracts data during the call phase: The runtest_makereport hook of the pytest framework is triggered during the call phase of each test. Conditional judgment ensures that processing only occurs during the test execution phase, avoiding repeated triggering during the setup and teardown phases. It traverses all assertion instances in the global registry of class variables, collects assertion context information, assertion statistics, fingerprint information, and intent information, and writes them into the collector of the data collection layer, providing a complete data foundation for subsequent intelligent analysis.

[0039] Step 3, Intelligent Analysis and Comprehensive Report Generation: When the pytest session ends, the sessionfinish hook is triggered, starting the four analysis engines of the intelligent analysis layer and generating a comprehensive report, realizing the leap from simple report generation to in-depth analysis at the assertion level.

[0040] First, behavioral profiling and problem prediction: The profiling engine constructs a behavioral feature vector for each assertion based on historical assertion execution data, including four dimensions: stability score, trend score, execution frequency, and related assertion set; it calculates scores using stability and trend formulas and automatically classifies assertions into at least one behavioral label: STABLE, UNSTABLE, FLAKY, CRITICAL, NEW; it supports problem prediction at the assertion level, reducing problem discovery time by 65% ​​and achieving a profiling problem prediction accuracy of 85%.

[0041] The behavioral profiling employs assertion behavioral profiling technology, which constructs assertion behavioral feature vectors based on historical assertion execution data. This creates a multi-dimensional profile for each assertion, including stability score, trend score, execution frequency, and a set of related assertions. Unlike traditional quality assessments that rely solely on pass rates, assertion profiling focuses on the multi-dimensional behavioral characteristics of assertions, including execution frequency, failure mode, time distribution, and contextual relevance. The profile data can be used for assertion-level problem prediction and test optimization suggestions. The profiling categorizes assertions into five behavioral labels: STABLE, UNSTABLE, FLAKY, CRITICAL, and NEW. STABLE indicates stable assertions, UNSTABLE indicates unstable assertions, FLAKY indicates flaky testing, CRITICAL indicates severe deterioration, and NEW indicates new assertions.

[0042] The formula for calculating the stability score is as follows:

[0043] ;

[0044] in The maximum number of consecutive failures is represented by S, and the total number of executions is represented by N. The stability score ranges from 0 to 1, with S values ​​closer to 1 indicating greater stability and closer to 0 indicating less stability. The stability judgment rules are as follows: S values ​​greater than or equal to 0.8 are considered stable, corresponding to the behavior label STABLE; S values ​​less than 0.5 are considered unstable, corresponding to the behavior label UNSTABLE; values ​​in between are considered basically stable.

[0045] The formula for calculating the trend score is as follows:

[0046] ;

[0047] in The failure rate of the most recent N executions. The failure rate of the first N executions; the trend score ranges from -1 to 1, a T value less than 0 indicates a deteriorating trend, a T value greater than 0.3 indicates an improving trend, and a T value less than -0.5 indicates a severe deterioration, corresponding to the behavioral label CRITICAL.

[0048] Second, dependency graph construction and change impact assessment: The dependency graph engine extracts the set of variables referenced by assertions through static analysis and identifies assertions that co-occur in the same test function through dynamic analysis. It constructs a dependency network between scripts based on shared variables and co-occurrence relationships. The engine calculates the dependency strength between two scripts using edge weight calculation formulas, identifying test silos and the scope of change impact. This mechanism constructs dependency relationships based on shared variable semantics, providing decision support for test maintenance while effectively avoiding AIOps time window correlation analysis patents.

[0049] The dependency graph construction employs assertion dependency graphing technology. This technology extracts the set of variables referenced by assertions through static analysis and identifies assertions that co-occur within the same test function through dynamic analysis. Based on shared variables and co-occurrence relationships, it constructs a dependency network between scripts. Unlike AIOps correlation analysis based on time correlation, this invention constructs dependency relationships based on the semantics of shared variables, effectively circumventing AIOps time window correlation analysis patents. The dependency graph can be used for advanced functions such as identifying true dependencies between test scripts, discovering test silos, and assessing the impact of test changes.

[0050] The formula for calculating the edge weight between two scripts a and b is:

[0051] ;

[0052] in Assert the set of variables referenced by script a. Assert the set of variables referenced by script b; the intersection represents the number of shared variables. This is a co-occurrence indicator function. It takes the value 1 when script a and script b both appear in the test function, and 0 otherwise. A value of 0.7 represents the weight of the shared variable. A co-occurrence weight of 0.3 is used. The edge type determination rules are as follows: edge weights greater than or equal to 1.5 indicate strong dependency; weights between 0.5 and 1.5 indicate medium dependency; and weights less than 0.5 indicate weak dependency. This formula constructs dependency relationships based on semantic variables, which differs significantly from AIOps time correlation analysis.

[0053] Third, session-level aggregation and multi-dimensional report output: Aggregate statistical data of all assertor instances by script name, output a unified test report at the script level, covering script name, assertion data fingerprint, intent label, statistical data and conclusions; generate intent analysis report, count the number and proportion of assertions for each intent category, with intent recognition accuracy reaching 82%, significantly reducing manual annotation costs; generate profile summary report, outputting the distribution of each behavior label; finally, call the cleanup method to release memory resources, achieving 100% aggregation coverage, deeply coupled with the pytest lifecycle, and supporting session-level incremental aggregation.

[0054] Session-level aggregation employs pytest hook-based collaborative aggregation technology. This technology utilizes hook functions provided by the pytest framework to achieve data collection and aggregation at the end of the session. The `runtest_makereport` hook is responsible for extracting statistical data during the call phase of each test, ensuring processing only during the test execution phase through conditional checks to avoid duplicate triggering during the setup and teardown phases. The `sessionfinish` hook is responsible for performing final aggregation and report generation at the end of the session, while simultaneously calling cleanup methods to release memory. Hooks are deeply coupled with the pytest lifecycle to ensure data accuracy and integrity. The aggregation coverage calculation formula is as follows:

[0055] ;

[0056] in The number of assertion instances being aggregated. This represents the total number of assertion instances. This formula is used to evaluate the percentage of assertion instances that were successfully aggregated at the end of the session, with a target value of 100%, meaning that all assertion instances are aggregated into the corresponding script report entries at the end of the session.

[0057] In step 3, the intelligent analysis also includes assertion intent recognition, which employs assertion intent recognition technology. This technology automatically infers the test intent behind the assertions by analyzing their pattern features. Based on the four-dimensional features of the assertion—method name, parameter type, comparison range, and variable naming—a weighted match is performed with a preset intent pattern library to calculate the matching score and output intent labels. The intent recognition results are output in semantic label form, improving the readability of the test report. Intent recognition supports incremental learning, dynamically adjusting pattern weights based on labeled data, significantly reducing manual labeling costs. Intent categories cover six main types: configuration verification, boundary checks, status confirmation, data consistency, permission checks, and dependency verification. The intent label format is "intent category [subcategory]", such as "configuration verification [port]" or "boundary check [timeout]". The formula for calculating the intent matching score is:

[0058] ;

[0059] in A weight of 0.4 is used to name the variable. A value of 0.3 represents the method name weight. A value of 0.3 indicates the parameter type and the comparison range weight. To ensure score normalization, Name the variable to match the score. Score based on method name matching. Scoring is given based on the match between parameter type and comparison range (in engineering practice, parameter type and comparison range are essentially the same thing; that is, an assertion of an integer type must compare values ​​within the integer range). The intent determination rules are as follows: If If the score is greater than or equal to 0.3, the intent category with the highest matching score is taken as the recognition result; if... If the value is less than 0.3, the default intent "General Validation" is used. The weighted matching mechanism of this formula can effectively integrate multi-dimensional features and improve the accuracy of intent recognition.

[0060] Example 2

[0061] Please see Figure 1 As shown, an automated test report generation system based on multi-assertor instance aggregation adopts a four-layer architecture, consisting of a data acquisition layer, a data storage layer, an intelligent analysis layer, and an application service layer from bottom to top. The data acquisition layer is the foundation of the entire system, responsible for collecting assertion context information, assertion statistics, fingerprint information, and intent information from the test execution process, accomplished by a context capturer, a statistics capturer, a fingerprint capturer, and an intent capturer, respectively. The data storage layer stores the assertion registry, script statistical aggregation data, a fingerprint database, and a profile database. The assertion registry uses a dictionary-like variable structure for global sharing, circumventing the fixture scope limitations of pytest. The intelligent analysis layer performs fingerprint matching analysis, intent recognition analysis, dependency graph analysis, and profile construction analysis on the collected data, consisting of four core engines: a fingerprint matching engine, an intent recognition engine, a dependency graph engine, and a profile construction engine. The application service layer, facing the end user, is responsible for generating test reports, intent analysis reports, graph reports, and profile summary reports to meet different analytical needs. The four layers are connected by data flow. The data acquisition layer outputs raw data to the upper layer, the intelligent analysis layer outputs analysis results to the upper layer, and the application service layer generates the final report.

[0062] At the functional level, this invention consists of the following core modules:

[0063] The module includes: a global registration module for assertion class variables, which manages assertion instances uniformly through a class variable dictionary, with a registration operation time complexity of O(1); a log processor path inference module, which extracts script filenames from the log processors of assertion instances and automatically establishes associations using the log directory structure; a call stack tracing module, which extracts script filenames by traversing the call stack using Python's inspect module, serving as a supplementary mechanism for log path inference; an assertion data fingerprint generation module, which generates unique assertion fingerprints based on four-tuple fusion, supporting cross-session assertion tracing; an assertion behavior profiling module, which constructs multi-dimensional behavioral feature vectors, supporting assertion-level problem prediction; an assertion intent recognition module, which automatically infers test intent, significantly reducing manual annotation costs; an assertion dependency graph construction module, which constructs a script dependency network based on shared variables, identifying test silos and the scope of change impact; and a pytest hook collaborative aggregation module, which aggregates output uniformly at the end of a session, achieving session-level incremental aggregation.

[0064] The system achieves the following eight objectives: 1. Without modifying any test code, automatically register multiple assertion instances scattered across different test functions into a globally visible registry, enabling instance management across test functions and pytest scopes; 2. Automatically establish the association between assertion instances and test scripts, avoiding the maintenance costs associated with manually configuring script names; 3. Automatically capture complete call context information during assertion execution, including call stack, filename, function name, call depth, and local variables; 4. Automatically aggregate statistical data of all assertion instances by script name at the end of the pytest session, outputting a unified test report at the script level; 5. Promptly clean up processed instances after report generation to avoid memory leaks during long-term operation; 6. Generate unique and stable fingerprint identifiers based on the multi-dimensional characteristics of assertions, supporting assertion tracing across sessions and test runs; 7. Automatically infer the test intent behind assertions, generating semantic tags such as configuration verification and boundary checks; 8. Construct a dependency network between scripts based on shared variables to identify test silos and the scope of change impact.

[0065] Example 3

[0066] This embodiment further illustrates the invention in the specific scenario of power terminal automation testing: In a power terminal automation testing project, the test script "030_Configuration File_Notes on Test Configuration File.py" contains three test functions (A, B, and C), which respectively test the reading, writing, and verification functions of the configuration file. Each test function in this script creates an independent assertion instance, with the aim of aggregating the statistical data of all assertion instances into a single script-level test report entry.

[0067] Implement according to step 1 (global registration of class variables and generation of assertion data fingerprints):

[0068] After the pytest session starts, the system automatically initializes the data acquisition layer, data storage layer, intelligent analysis layer, and application service layer in the four-layer architecture. Simultaneously, it creates a class variable dictionary in the assertion class as a global registry. When the first test function A is executed, it loads the configuration file to obtain the configuration object and creates the first assertion instance. During assertion instance creation, the following operations are automatically completed: initializing instance attributes including the assertion name, creating a logger and establishing a log directory, and clearing the statistics counter; registering the assertion name as the key and the instance reference as the value in the class variable dictionary, with a registration operation time complexity of O(1); generating a unique and stable 32-bit hexadecimal assertion fingerprint based on the four-tuple of assertion method name, parameter signature, comparison value hash, and call context hash; and automatically establishing the association between the assertion instance and the test script through a dual mechanism of log processor path backtracking and call stack depth tracing. When the second test function B is executed, a second assertion instance is created and the same registration, fingerprint generation, and script association operations are automatically completed. When the third test function C is executed, a third assertion instance is created and the above operations are automatically completed. The registration operations for the three assertion instances all have an O(1) time complexity. They are automatically shared across all scopes (function, class, module, session) of pytest without requiring any modification to the test code, achieving zero-intrusion script-level assertion aggregation.

[0069] Implement according to step 2 (two-way tracing and pytest hook collaborative data collection):

[0070] First, the script association is established by reverse tracing the log processor path: Each of the three assertioner instances establishes a log directory structure upon creation, and the log processor of each assertioner instance stores the corresponding log file path. By traversing the log processors and extracting the parent directory name of the log file path (i.e., "030_Configuration File_Notes on Test Configuration File"), a reverse tracing mapping from assertioner instances to test scripts is automatically established, ensuring the integrity of the mapping relationship between assertioner instances and test scripts.

[0071] Secondly, call stack depth tracing supplements the association: as a supplementary mechanism to log path backtracking, the `inspect` module in Python traverses the call stack, skipping the assertion module itself and extracting the filename of the first external script. This ensures that script association can still be completed even in abnormal scenarios such as irregular log directory structures or cleaned-up log processors. These two methods complement each other, ensuring the accuracy of the mapping relationship and avoiding the risk of failure of a single tracing method in abnormal scenarios.

[0072] Third, pytest hooks extract data during the call phase: During test function execution, the pytest framework's runtest_makereport hook is triggered in the call phase of each test. Conditional checks ensure that processing only occurs during the test execution phase, avoiding duplicate triggering in the setup and teardown phases. After the hook is triggered, it iterates through the three assertor instances in the global registry of class variables, collecting statistical data, fingerprint information, intent tags, and context information for each instance. This data is then written to the collector in the data collection layer, providing a complete data foundation for subsequent intelligent analysis.

[0073] Fourth, Assertion Execution and Intent Recognition: The first test function A executes two assertions to verify port configuration and timeout configuration, both of which pass. The intent recognition result of the first assertion is configuration verification with a confidence level of 1.0, and the intent recognition result of the second assertion is configuration verification with a confidence level of 1.0. The second test function B executes two assertions to verify write results and log content, both of which pass. The intent recognition result of the first assertion is status confirmation with a confidence level of 0.8, and the intent recognition result of the second assertion is data consistency with a confidence level of 0.7. The third test function C executes two assertions to verify numerical range. The first assertion (value greater than zero) fails to execute, with an intent recognition result of boundary check with a confidence level of 0.9. The second assertion (value less than one hundred) passes, with an intent recognition result of boundary check with a confidence level of 0.85.

[0074] Follow step 3 (Intelligent Analysis and Comprehensive Report Generation):

[0075] First, behavioral profile construction and problem prediction: A behavioral profile is constructed for each assertion based on the execution history data of three assertion instances. The first assertion instance has both assertions that passed, with a stability score of 1.0, a trend score of 1.0, and a behavioral label of STABLE. The second assertion instance has both assertions that passed, with a stability score of 1.0, a trend score of 1.0, and a behavioral label of STABLE. The third assertion instance has one assertion that failed, with a stability score of 0.8, a trend score of 0.7, and a behavioral label of UNSTABLE. The profiles are summarized into four stable labels and one unstable label, supporting problem prediction at the assertion level.

[0076] Second, dependency graph construction and change impact assessment: Static analysis identifies shared variable sets such as configuration objects, write results, logs, and numerical variables shared by the three test functions; dynamic analysis identifies read and write functions appearing together in the same script. A dependency network between scripts is constructed based on shared variables and co-occurrence relationships. The dependency strength is calculated using the edge weight calculation formula. An edge weight of 1.0 indicates a medium dependency, providing decision support for testing and maintenance.

[0077] Third, session-level aggregation and multi-dimensional report output: When the pytest session ends, the sessionfinish hook is triggered, aggregating statistical data from three assertion instances by script name, and outputting a unified test report at the script level, covering script name, assertion data fingerprint, intent label, statistical data, and conclusions. The aggregation results are: total assertions 6, pass count 5, failure count 1, pass rate 83.33%, and aggregation coverage reaching 100%. An intent analysis report is generated, with intent distribution statistics as follows: configuration verification 2 times (33.3%), status confirmation 1 time (16.7%), data consistency 1 time (16.7%), and boundary checks 2 times (33.3%). Finally, the cleanup method is called to release memory resources, and a comprehensive test report containing statistical data, fingerprint information, intent labels, and profile data is output.

[0078] As can be seen from the above embodiments, the method of the present invention successfully realizes the automatic aggregation and statistics of multiple assertion instances. Without modifying the test code, it automatically establishes the mapping relationship from instance to script to report, and outputs a comprehensive test report containing statistical data, fingerprint information, intent tags and profile data, effectively solving the technical problem of difficulty in aggregating and statistically analyzing multiple assertion instances in the prior art.

[0079] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. An automated test report generation method based on multi-assertor instance aggregation, characterized in that, Includes the following steps: Step 1: When the pytest test session starts, the system creates a class variable dictionary in the assertion class as a global registry, and registers the assertion instances created by the test function into the class variable dictionary with the assertion name as the key and the instance reference as the value, so as to achieve unified management of instances across the pytest scope and generate assertion fingerprints. Step 2: Automatically establish the association between assertion instances and test scripts through a dual mechanism of log processor path backtracking and call stack depth tracing; during the test execution phase, the runtest_makereport hook of the pytest framework is triggered in the call phase of each test, traversing the assertion instances in the global registry to collect assertion context information, statistics, fingerprint information and intent information; Step 3: When the pytest session ends, intelligent analysis is triggered through the sessionfinish hook. This includes building a behavioral profile for each assertion based on historical execution data, building a dependency network between scripts based on shared variables and co-occurrence relationships, and aggregating statistical data of all assertion instances by script name, outputting a unified test report at the script level.

2. The automated test report generation method based on multi-assertor instance aggregation according to claim 1, characterized in that, In step 1, the formula for asserting fingerprint generation is: ； Where M is the assertion method name, used to assert different expected test results; P is the type label string sorted by parameter name; V is the comparison value hash, which is obtained by serializing the expected value and the actual value, calculating the SHA256 hash, and taking the first 16 bits; C is the context hash, which is obtained by combining the function name and the call depth, calculating the SHA256 hash, and taking the first 8 bits. This is a vertical line separator.

3. The automated test report generation method based on multi-assertor instance aggregation according to claim 1, characterized in that, Step 3, the construction of the assertion behavior profile includes: Based on historical assertion execution data, assertion behavior feature vectors are constructed, and a multi-dimensional profile is created for each assertion, including stability score, trend score, execution frequency, and related assertion set. The profile classifies assertions into five behavioral labels: STABLE, UNSTABLE, FLAKY, CRITICAL, and NEW. STABLE indicates stability, UNSTABLE indicates instability, FLAKY indicates flaky test, CRITICAL indicates severe deterioration, and NEW indicates new assertion.

4. The automated test report generation method based on multi-assertor instance aggregation according to claim 3, characterized in that, The formula for calculating the stability score is: ； in The maximum number of consecutive failures is N, and the total number of executions is N; the stability score ranges from 0 to 1. The formula for calculating the trend score is: ； in The failure rate of the most recent N executions. The failure rate for the first N executions; The trend score ranges from -1 to 1.

5. The automated test report generation method based on multi-assertor instance aggregation according to claim 1, characterized in that, In step 3, the construction of the inter-script dependency network includes: extracting the set of variables referenced by assertions through static analysis, identifying assertions that co-occur in the same test function through dynamic analysis, and constructing the inter-script dependency network based on shared variables and co-occurrence relationships; wherein the edge weight calculation formula between two scripts a and b is: ； in Assert the set of variables referenced by script a. Assert the set of variables referenced by script b; the intersection represents the number of shared variables. This is a co-occurrence indicator function. It takes the value 1 when script a and script b both appear in the test function, and 0 otherwise. A value of 0.7 represents the weight of the shared variable. A co-occurrence weight of 0.3 is given.

6. The automated test report generation method based on multi-assertor instance aggregation according to claim 1, characterized in that, In step 3, the sessionfinish hook calls a cleanup method after triggering intelligent analysis to release the memory resources occupied by the processed assertion instances; the formula for calculating aggregate coverage is: ； in The number of assertion instances being aggregated. This represents the total number of assertion instances.

7. The automated test report generation method based on multi-assertor instance aggregation according to claim 1, characterized in that, Step 3 also includes: Generate an intent analysis report, and count the number and percentage of assertions for each intent category; Generate a profile summary report and output the distribution of each behavior tag; Call the cleanup method to release memory resources.

8. The automated test report generation method based on multi-assertor instance aggregation according to claim 7, characterized in that, In step 3, the intelligent analysis further includes assertion intent recognition: based on the four-dimensional features of the assertion—method name, parameter type, comparison range, and variable naming—a weighted match is performed with a preset intent pattern library to calculate the intent matching score and output the intent label; the formula for calculating the intent matching score is: ； in A weight of 0.4 is used to name the variable. A value of 0.3 represents the method name weight. A value of 0.3 indicates the parameter type and the comparison range weight. , Name the variable to match the score. Score based on method name matching. Score the parameter type and comparison range matching.

9. The automated test report generation method based on multi-assertor instance aggregation according to claim 1, characterized in that, In step 2, The log processor path reverse inference includes: traversing the log processors of the assertion instance and extracting the parent directory name from the log file path as the script file name; The call stack depth tracing includes: traversing the call stack using Python's inspect module, skipping the assertion module itself, and then extracting the filename of the first external script.

10. An automated test report generation system based on multi-assertor instance aggregation, the system being applicable to the method of any one of claims 1-9, characterized in that, include: The data acquisition layer includes a context capturer, a statistics capturer, a fingerprint capturer, and an intent capturer, which are used to collect assertion context information, assertion statistics, fingerprint information, and intent information from the test execution process, respectively. The data storage layer is used to store the assertioner registry, script statistical aggregation data, fingerprint database, and profile database; The assertion registry uses a class variable dictionary structure to achieve global sharing; The intelligent analysis layer includes a fingerprint matching engine, an intent recognition engine, a dependency graph engine, and a profile building engine, which are used to perform fingerprint matching analysis, intent recognition analysis, dependency graph analysis, and profile building analysis on the collected data, respectively. The application service layer is used to generate test reports, intent analysis reports, graph reports, and profile summary reports.

Images