Software test result analysis system

Abstract

The invention discloses a software test result analysis system which comprises a multi-source test data acquisition module, an intelligent anomaly detection engine, a cross-project association analysis module, a visual decision support terminal and a self-adaptive feedback optimization module. The multi-source test data acquisition module integrates heterogeneous test tool data and performs standardization processing; the intelligent anomaly detection engine realizes dynamic anomaly detection by using an LSTM model and a reinforcement learning algorithm; the cross-project association analysis module completes cross-project data association by means of code tracking, a knowledge graph and the like; the visual decision support terminal visually displays an analysis result in the form of a three-dimensional thermodynamic diagram and the like; and the adaptive feedback optimization module forms closed-loop control and dynamically adjusts a test strategy. The system effectively solves the problems that according to a traditional method, multi-source data cannot be fused, anomaly detection is not sensitive, and visual analysis is lacked, efficient analysis of software test results and optimization of the test process are achieved through cooperative work of all the modules, and the test efficiency and the software quality are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing devices, and in particular to a software testing result analysis system. Background Art

[0002] In the field of software testing, test result analysis is a key link in ensuring software quality. Currently, traditional software test result analysis methods have significant limitations. On the one hand, they only focus on the output of a single test tool, making it difficult to integrate multi-source heterogeneous test data such as unit testing, UI testing, and performance testing. They are unable to conduct a comprehensive and integrated analysis and evaluation of the software, making it difficult to discover potential correlations and problems between data. On the other hand, existing anomaly detection generally relies on static threshold rules. When faced with complex situations such as intermittent failures and hidden performance degradation, their sensitivity is seriously insufficient, and important software defects are easily missed. In addition, the presentation of test reports is fragmented, lacking effective integration and trend visualization of cross-version and cross-project data, making it difficult for testers to grasp the changing trends of software quality from a macro perspective, greatly affecting the efficiency and accuracy of test result analysis. These problems seriously restrict the quality and effectiveness of software test result analysis, and a new software test result analysis system is urgently needed to solve the above problems. Summary of the Invention

[0003] (1) Technical issues

[0004] The present invention provides a software test result analysis system that can integrate multi-source heterogeneous test data, realize intelligent dynamic anomaly detection, support cross-project trend visualization analysis, and optimize the test process through adaptive feedback.

[0005] (2) Technical content

[0006] To solve the above technical problems, the technical solution of the present invention is: a software test result analysis system, including a multi-source test data acquisition module; an intelligent anomaly detection engine; a cross-project correlation analysis module; a visual decision support terminal; and an adaptive feedback optimization module; the multi-source test data acquisition module is connected to the intelligent anomaly detection engine through a distributed message queue, the output end of the intelligent anomaly detection engine is connected to the input end of the cross-project correlation analysis module, the output of the cross-project correlation analysis module interacts with the visual decision support terminal through an API gateway, and the adaptive feedback optimization module forms a closed-loop control loop and is bidirectionally connected to each module.

[0007] Furthermore, the multi-source test data acquisition module includes:

[0008] Supports heterogeneous test tool adapters such as Junit, Selenium, or Appium;

[0009] Real-time log sniffer based on regular expressions and natural language processing;

[0010] Environmental parameter capture unit that records CPU, memory, and network metrics during testing;

[0011] A data standardization processor that implements unified conversion between XML, JSON, and CSV formats.

[0012] Furthermore, the intelligent anomaly detection engine adopts: an LSTM-based time series anomaly prediction model; a reinforcement learning-driven threshold dynamic adjustment algorithm; and a multi-dimensional feature cross-validation mechanism, which simultaneously analyzes the correlation between error rate, execution time, and resource consumption.

[0013] Furthermore, the cross-project correlation analysis module includes:

[0014] A code change impact tracker that links Git commit records with test failure cases;

[0015] A defect pattern knowledge graph that stores feature vectors of historical defects and solutions;

[0016] A similar item matcher that computes cross-item test pattern matching via cosine similarity.

[0017] Furthermore, the visual decision support terminal provides: three-dimensional heat map display, root cause deduction path diagram, and interactive test case clustering view.

[0018] Furthermore, the adaptive feedback optimization module implements: dynamic adjustment of test case priorities, i.e., re-ordering based on historical pass rates and defect detection capabilities; elastic allocation strategy for test resources, i.e., automatic increase or decrease in the number of concurrent cloud test nodes; and self-learning filter for false alarm patterns, i.e., reducing repetitive false alarms through continuous training.

[0019] (3) Technical effects

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] 1. The multi-source test data acquisition module, through components such as heterogeneous test tool adapters and real-time log sniffers, effectively integrates test data generated by different tools such as JUnit and Selenium, enabling unified conversion between multiple formats such as XML and JSON. This addresses the inability of traditional methods to integrate multi-source data and lays the foundation for comprehensive and integrated software quality analysis. The intelligent anomaly detection engine utilizes an LSTM time series anomaly prediction model and a reinforcement learning-driven threshold dynamic adjustment algorithm, combined with a multi-dimensional feature cross-validation mechanism. This significantly increases sensitivity to intermittent failures and hidden performance degradation, reducing the probability of missing important software defects.

[0022] 2. The cross-project correlation analysis module uses a code change impact tracker, a defect pattern knowledge graph, and a similar project comparer to link Git commit records with test failures and effectively match cross-project test patterns. This allows testers to draw solutions from historical experience, improving problem location and resolution efficiency. The visual decision support terminal provides visualizations such as three-dimensional heat maps and root cause derivation path maps, integrating fragmented test reports into intuitive and easy-to-understand visual presentations. This helps testers quickly grasp software quality trends across versions and projects, improving analytical decision-making efficiency.

[0023] 3. The adaptive feedback optimization module realizes dynamic adjustment of test case priorities, flexible allocation of test resources and self-learning filtering of false alarm patterns. By forming a closed-loop control loop, it optimizes the test process, improves test resource utilization, reduces repetitive false alarms, reduces test costs and improves test quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The present invention is a system schematic diagram of a software test result analysis system. DETAILED DESCRIPTION

[0025] The present invention will be described in further detail below with reference to the accompanying drawings.

[0026] Combined with attachment Figure 1 A software test result analysis system includes a multi-source test data acquisition module; an intelligent anomaly detection engine; a cross-project correlation analysis module; a visual decision support terminal; and an adaptive feedback optimization module. The multi-source test data acquisition module is connected to the intelligent anomaly detection engine via a distributed message queue, the output of the intelligent anomaly detection engine is connected to the input of the cross-project correlation analysis module, the output of the cross-project correlation analysis module interacts with the visual decision support terminal via an API gateway, and the adaptive feedback optimization module forms a closed-loop control loop and is bidirectionally connected to each module.

[0027] The multi-source test data acquisition module includes: a heterogeneous test tool adapter that supports Junit, Selenium, or Appium; a real-time log sniffer based on regular expressions and natural language processing; an environmental parameter capture unit that records CPU, memory, and network indicators during testing; and a data standardization processor that implements unified conversion between XML, JSON, and CSV formats.

[0028] The intelligent anomaly detection engine adopts: an LSTM-based time series anomaly prediction model; a reinforcement learning-driven threshold dynamic adjustment algorithm; and a multi-dimensional feature cross-validation mechanism, which simultaneously analyzes the correlation between error rate, execution time, and resource consumption.

[0029] The cross-project association analysis module includes: a code change impact tracker that associates Git commit records with test failure cases; a defect pattern knowledge graph that stores feature vectors and solutions of historical defects; and a similar project matcher that calculates the matching degree of cross-project test patterns through cosine similarity.

[0030] The visual decision support terminal provides: three-dimensional heat map display, root cause deduction path diagram, and interactive test case clustering view.

[0031] The adaptive feedback optimization module implements: dynamic adjustment of test case priorities, i.e., re-ordering based on historical pass rates and defect detection capabilities; elastic allocation strategy for test resources, i.e., automatically increasing or decreasing the number of concurrent cloud test nodes; and self-learning filters for false alarm patterns, i.e., reducing repetitive false alarms through continuous training.

[0032] The working principle of the present invention is as follows: based on the modular design concept, the present invention collects various types of heterogeneous test data through a multi-source test data acquisition module, and transmits them to the intelligent anomaly detection engine using a distributed message queue. The engine combines deep learning and reinforcement learning algorithms to perform anomaly detection on the data from multiple dimensions; the detection results are processed through a cross-project association analysis module, which uses code tracing, knowledge graphs and similarity calculation technologies to mine data associations; the analysis results are transmitted to a visual decision support terminal through an API gateway and displayed in intuitive graphics; the adaptive feedback optimization module forms a closed-loop control based on terminal feedback and dynamically adjusts the test strategy. The modules work together to achieve efficient analysis of software test results and optimization of the test process.

[0033] The working process of the present invention is as follows:

[0034] 1. Data Collection

[0035] After the multi-source test data collection module is initialized, the heterogeneous testing tool adapter is launched. For JUnit unit testing tools, the adapter parses the test report file and extracts information such as the test case name, execution result (pass or fail), and execution time. For Selenium or Appium automated testing tools, the adapter establishes a connection with the tool's runtime environment to obtain data such as UI element operation logs, page load time, and element positioning status.

[0036] The real-time log sniffer scans the log files generated during the test line by line based on pre-set regular expression rules, identifies key log information (such as error stacks and warning prompts), and uses natural language processing technology to perform semantic analysis on unstructured log text to extract valid data.

[0037] The environmental parameter capture unit obtains the CPU usage, core load, memory used / available space, network throughput, latency, packet loss rate and other indicator data in real time through the system API interface during the test, and samples them at fixed time intervals (such as once per second).

[0038] After receiving data in these various formats, the data normalization processor extracts target node data using XPath expressions for XML data; performs structured parsing on JSON data using a JSON parsing library; and reads and converts CSV data into a unified data structure by row and column. Ultimately, all data is converted to a standard format recognizable by the system and asynchronously transmitted to the intelligent anomaly detection engine via a distributed message queue (such as Kafka).

[0039] 2. Anomaly Detection

[0040] After receiving data, the intelligent anomaly detection engine first feeds time series data (such as execution time and resource consumption over time) into an LSTM-based time series anomaly prediction model. The LSTM model uses memory cells and gating mechanisms to learn long-term dependencies in the data, predict future data points, and compare the actual data with the predicted value. Any deviation exceeding a set threshold is flagged as a potential anomaly.

[0041] The reinforcement learning-driven threshold dynamic adjustment algorithm models the anomaly detection process as a Markov decision process based on historical data and current anomaly detection results. By continuously trying different threshold settings and learning based on reward mechanisms such as detection accuracy and false alarm rate, the algorithm dynamically adjusts the detection thresholds for each dimension of data to adapt to different test scenarios and data characteristics.

[0042] A multi-dimensional feature cross-validation mechanism is activated, correlating data across multiple dimensions, such as error rate, execution time, and resource consumption. For example, if a test case's execution time suddenly increases, accompanied by a significant increase in CPU usage and error rate, this is comprehensively identified as a true anomaly. If fluctuations occur in only one dimension, further analysis and verification are performed to avoid false positives. The results are collated and output to the cross-project correlation analysis module.

[0043] 3. Correlation Analysis

[0044] After the cross-project correlation analysis module obtains the anomaly detection results, the code change impact tracker interacts with the Git version control system to obtain the code commit records for the time period corresponding to the anomaly test case. The code commits are analyzed for their file paths, modifications, and the functions and classes involved. These are then correlated with the failed test cases to determine whether the code changes were the cause of the test failure.

[0045] The defect pattern knowledge graph is built on a graph database (such as Neo4j) to retrieve feature vectors of historical defects (including error type, occurrence scenario, impact range, etc.) and solutions. The current anomaly detection results are matched with nodes in the knowledge graph. If similar defect patterns are found, the corresponding solutions are extracted for reference.

[0046] When calculating cross-project test pattern matching, the Similar Project Comparator converts the current project's test data (such as test case execution result sequences and resource consumption patterns) into feature vectors. Using the cosine similarity algorithm, it compares these features with those of historical projects to identify projects with high similarity and leverages the testing experience and problem-solving methods of similar projects. The analysis results are then transmitted to the visual decision support terminal via the API gateway.

[0047] 4. Visual display

[0048] After the visual decision support terminal receives the correlation analysis results, the three-dimensional heat map module generates coordinate points for each test case in three-dimensional space based on the data of the three dimensions of test case execution results, execution time and resource consumption. The density of the data is indicated by the depth of color, which intuitively displays the distribution characteristics of the test data and helps users quickly locate areas with concentrated abnormalities.

[0049] The root cause derivation path diagram module builds a derivation logic diagram from anomaly detection and correlation analysis results to the root cause. Nodes represent the problem, intermediate analysis steps, and the final cause, and directed edges connect the nodes, clearly presenting the context and derivation process of the problem.

[0050] The interactive test case clustering view module uses clustering algorithms (such as K-means) to classify test cases. Users can view detailed information about test cases in different clusters by clicking and zooming in and out, including case descriptions, execution results, and relationships with other cases, making it easier to manage and analyze test cases.

[0051] 5. Feedback optimization

[0052] The adaptive feedback optimization module first dynamically adjusts test case priorities based on feedback from the visual decision support terminal. It then establishes a priority assessment model that comprehensively considers factors such as the test case's historical pass rate (lower pass rates, higher priorities), defect detection capabilities (higher defects historically, higher priorities), and business importance. It then assigns a priority value to each test case and reorders its execution order.

[0053] The test resource elastic allocation strategy calculates the required number of concurrent cloud test nodes based on the complexity of the current test task, the estimated execution time, and resource usage. If the task volume increases and resource utilization is low, the platform automatically requests additional test nodes. If the task nears completion or resources are idle, the platform releases excess nodes to achieve efficient resource utilization.

[0054] The false alarm pattern self-learning filter collects false alarm data generated by the system, extracts its features (such as the test scenarios and data patterns that trigger false alarms), trains it using machine learning algorithms (such as Naive Bayes and Support Vector Machines), and updates the false alarm identification model. During subsequent testing, the updated model is used to re-evaluate the test results to reduce repetitive false alarms. The optimized test strategy is fed back to various modules, including the multi-source test data acquisition module and the intelligent anomaly detection engine, forming a closed-loop control circuit to continuously improve the overall efficiency of software testing.

[0055] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A software test result analysis system, characterized in that: Includes multi-source test data acquisition module; intelligent anomaly detection engine; Cross-project correlation analysis module; visual decision support terminal; Adaptive feedback optimization module; The multi-source test data acquisition module is connected to the intelligent anomaly detection engine through a distributed message queue. The output of the intelligent anomaly detection engine is connected to the input of the cross-project association analysis module. The output of the cross-project association analysis module interacts with the visual decision support terminal through the API gateway. The adaptive feedback optimization module forms a closed-loop control loop and is bidirectionally connected to each module.

2. A software test result analysis system according to claim 1, characterized in that: The multi-source test data acquisition module includes: Supports heterogeneous test tool adapters such as Junit, Selenium, or Appium; Real-time log sniffer based on regular expressions and natural language processing; Environmental parameter capture unit that records CPU, memory, and network metrics during testing; A data standardization processor that implements unified conversion between XML, JSON, and CSV formats.

3. A software test result analysis system according to claim 1, characterized in that: The intelligent anomaly detection engine adopts: an LSTM-based time series anomaly prediction model; a reinforcement learning-driven threshold dynamic adjustment algorithm; and a multi-dimensional feature cross-validation mechanism, which simultaneously analyzes the correlation between error rate, execution time, and resource consumption.

4. A software test result analysis system according to claim 1, characterized in that: The cross-project association analysis module includes: A code change impact tracker that links Git commit records with test failure cases; A defect pattern knowledge graph that stores feature vectors of historical defects and solutions; A similar item matcher that computes cross-item test pattern matching via cosine similarity.

5. A software test result analysis system according to claim 1, characterized in that: The visual decision support terminal provides: three-dimensional heat map display, root cause deduction path diagram, and interactive test case clustering view.

6. A software test result analysis system according to claim 1, characterized in that: The adaptive feedback optimization module implements: dynamic adjustment of test case priorities, i.e., re-ordering based on historical pass rates and defect detection capabilities; elastic allocation strategy for test resources, i.e., automatically increasing or decreasing the number of concurrent cloud test nodes; and self-learning filters for false alarm patterns, i.e., reducing repetitive false alarms through continuous training.

Images