Intelligent evaluation system based on database software test result

Abstract

The invention relates to the technical field of software test evaluation, and discloses a database software test result-based intelligent evaluation system, which comprises a data acquisition module, an intelligent analysis module, an evaluation decision module, a display storage module and a scene adaptation module. Multi-source test data is acquired and cleaned through the data acquisition module, the intelligent analysis module constructs a causal atlas by using time sequence grouping intervention, metadata driving association and scene clustering, the evaluation decision module forms a three-dimensional evaluation model, and the display storage module realizes natural language report and visual interaction. Meanwhile, by means of a scene adaptation module and an incremental learning ability dynamic optimization evaluation strategy, the root cause positioning efficiency can be improved, the evaluation error can be reduced, the real-time evaluation requirement can be met, multi-source data integration and model self-evolution are achieved, and a solution is provided for database test evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing and evaluation, and in particular to an intelligent evaluation system based on database software testing results. Background Art

[0002] In the field of database software testing, traditional test result evaluation usually relies on manual analysis or rule-based automation tools. In existing technologies, testers need to manually process the massive amount of indicator data, functional test logs and security scan reports generated by performance testing, and judge whether the test results are qualified based on preset thresholds. Although some automated evaluation systems can perform statistical analysis of basic indicators, such as evaluating performance stability by calculating mean values and standard deviations, or identifying abnormal information in logs through keyword matching, such systems often remain at the independent evaluation level of a single indicator and lack correlation analysis of multi-source test data.

[0003] The main drawback of existing technologies is that they are unable to effectively handle the complex causal relationships in database testing. For example, it is difficult to identify the potential correlation between "increased CPU utilization" and "increased slow queries", resulting in inefficient root cause location. In addition, traditional evaluation systems generally lack scenario adaptability and are unable to dynamically adjust evaluation strategies for different database application scenarios such as online transactions and real-time analysis. "One-size-fits-all" evaluation errors often occur. More importantly, when faced with large-scale test data, existing technologies lack efficient data compression and causal inference algorithms, which can easily lead to evaluation delays and fail to meet real-time requirements, seriously affecting the efficiency of database version iteration. Summary of the Invention

[0004] The technical problem to be solved by the present invention is that the existing technology cannot effectively handle the complex causal relationships in database testing. To this end, we propose an intelligent evaluation system based on database software test results.

[0005] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions: an intelligent evaluation system for software test results based on a database, comprising a data acquisition module, an intelligent analysis module, an evaluation and decision module, and a display and storage module, wherein each module works in coordination through a standardized data flow; The data acquisition module obtains multi-source test data such as performance indicators and function logs, and outputs cleaned feature vectors; The intelligent analysis module generates indicator correlation maps through causal modeling; The assessment and decision-making module outputs risk assessment and root cause location results based on the graph; The display storage module presents results in visualization and natural language reports and stores historical data.

[0006] Preferably, the data acquisition module includes: Multi-source interface unit, connects to MySQL stress test data through JDBC protocol, and obtains MongoDB compatibility test reports through REST API; The data cleaning unit uses discrete wavelet transform to denoise CPU utilization data, processes IOPS outliers through median filtering, and outputs standardized data with a unified sampling frequency of 100 Hz.

[0007] Preferably, the intelligent analysis module includes: a time sequence grouping intervention unit, which divides the concurrent test data into high / low groups according to the threshold value, and calculates the causal strength by the formula: ,in It is expressed as the average value of indicator B when indicator A is in the high intervention group, for example, the average IO waiting time when the concurrency is greater than 200; Indicates the average value of indicator B when indicator A is in the low intervention group, for example, the average IO wait time when the concurrency is ≤ 200; is the standard deviation of indicator B, which is used to normalize the numerator. By comparing the high / low groups, it quantifies the impact of the change in indicator A on indicator B. The larger the absolute value, the more significant the impact of A on B. is the timing constraint factor. When the change of indicator A occurs before the change of indicator B ,otherwise , used to ensure the temporal directionality of the causal relationship and avoid misjudging the reverse causality of "B→A" as "A→B"; the dynamic hysteresis window unit traverses the 1-500ms hysteresis interval and searches for the maximum correlation hysteresis period between IO waiting time and CPU occupancy.

[0008] Preferably, the intelligent analysis module further comprises: a metadata driven association unit, parsing the table structure metadata, learning the indicator association factor through a single-layer neural network , which indicates the domain correlation strength between indicators A and B. The larger the value, the stronger the correlation between the two in terms of database principles. ,in is the input feature vector, 、 The database metadata corresponding to indicators A and B, such as table structure, index configuration, field access frequency, etc. 、 It is a metadata feature extraction function that converts the original metadata into numerical features, such as encoding the index type as 0 / 1 and normalizing the field access frequency to [0,1]. is the number of co-occurrences of indicators A and B in historical test data, reflecting the statistical correlation between the two; W is the weight parameter of the neural network, which is obtained by training the historical causal graph data and is used to quantify the importance of each input feature; activation function For the Sigmoid function: , mapping the output value to the [0,1] interval.

[0009] Preferably, the scene clustering unit uses DBSCAN to cluster the test scene features and generate scene weights Used to adjust the evaluation dimension, , where T represents the current scene feature vector, the scene features extracted from the test data, Represents the historical cluster center, the center vector obtained by clustering historical test scenes using the DBSCAN algorithm. Represents a typical scenario. To calculate the current scene T and the historical cluster center The directional similarity is used to measure the similarity between the current scene and the historical scene. The larger the value, the closer the scene is. The weight of the historical cluster center is determined by the amount of test data contained in the cluster. The larger the amount of data, the higher the weight.

[0010] Preferably, the intelligent analysis module further includes: an evaluation and decision module constructing a three-dimensional evaluation model including risk level, impact range and repair cost based on the cause-effect diagram generated by the intelligent analysis module, wherein: The risk level combines the vulnerability CVSS score and the database business importance weight; The scope of impact identifies cascading effects in upstream and downstream systems through causal chain propagation analysis; Repair costs are predicted based on a repair case library of historical test data.

[0011] Preferably, the display storage module includes: a natural language generation unit that converts the technical indicators of the evaluation and decision-making module into a business-readable report, the report including a natural language description of the causal chain and optimization suggestions; The interactive visualization unit displays the causal diagram in the form of a dynamic graph, allowing users to explore the causal relationship of test data through interactive operations.

[0012] Preferably, a scene adaptation module is also included, which is used to: Extract scenario features from test data, including concurrency patterns, SQL type distribution, or transaction ratios; The corresponding evaluation strategy is automatically loaded based on the scenario characteristics. The evaluation strategy includes indicator weight configuration and threshold adjustment rules.

[0013] Preferably, the scenario adaptation module clusters historical test scenarios through a machine learning model, generates a mapping relationship between scenario feature vectors and evaluation strategies, and realizes dynamic adaptation of the evaluation model under different database application scenarios.

[0014] Preferably, the system has incremental learning capabilities to continuously optimize the evaluation model through: Update the parameters of metadata-driven associated units using the newly added test data; The feature mapping relationship of the scene clustering adaptation unit is adjusted based on the historical evaluation results.

[0015] The technical effects and advantages of the present invention are as follows: In the present invention, in order to address the problems existing in existing database software testing and evaluation, such as the inability to effectively handle complex causal relationships, lack of scenario adaptability, and large-scale data evaluation delays, the data acquisition module obtains and cleans multi-source test data, the intelligent analysis module uses time series grouping intervention, metadata-driven association, and scenario clustering to construct a causal graph, the evaluation decision module forms a three-dimensional evaluation model, and the display storage module realizes natural language reporting and visual interaction. At the same time, with the help of the scenario adaptation module and incremental learning capabilities, the evaluation strategy is dynamically optimized, which can improve the efficiency of root cause location, reduce evaluation errors, meet real-time evaluation needs, realize multi-source data integration and model self-evolution, and provide a solution for database testing and evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The disclosure of the present invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components: Figure 1 It is a module schematic diagram of the present invention. DETAILED DESCRIPTION

[0017] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of the present invention and should not be regarded as the entire invention or as a limitation or restriction of the technical solution of the present invention.

[0018] Reference Figure 1 As shown, the present invention provides a technical solution: an intelligent evaluation system based on database software test results through data collection, intelligent analysis, evaluation decision-making, display storage and scenario adaptation modules. The specific workflow is as follows: The data acquisition module obtains multi-source data such as performance indicators and function logs from testing tools such as JMeter and Selenium, and generates standardized feature vectors after cleaning; The intelligent analysis module performs causal modeling on feature vectors and generates indicator causal graphs through algorithms such as time series grouping intervention and metadata-driven association; The assessment and decision-making module builds a three-dimensional assessment model based on the cause-effect diagram, outputting risk level, root cause location, and remediation suggestions; The display storage module presents evaluation results in natural language reports and interactive visualizations, and stores historical data for model optimization.

[0019] The data collection module includes a multi-source interface unit, which connects to the performance data of the MySQL stress testing tool in real time through the JDBC protocol, including indicators such as QPS, TPS, and number of connections; uses the REST API to obtain MongoDB compatibility test reports and parse non-relational database feature data such as document storage and query efficiency; supports custom plug-in extensions and adapts to the output formats of testing tools for different types of databases such as Oracle and PostgreSQL.

[0020] The data acquisition module also includes a data cleaning unit, which applies discrete wavelet transform to time series data such as CPU utilization and IOPS to eliminate high-frequency noise such as network jitter and hardware fluctuations; uses a median filtering algorithm to process IOPS outliers to avoid single-point failures from interfering with the overall assessment; and uses resampling technology to unify test data of different frequencies to a 100Hz sampling rate to ensure timing consistency for subsequent analysis.

[0021] The intelligent analysis module includes a time series grouping intervention unit, which divides the concurrent test data into high / low intervention groups according to business thresholds, such as 200 concurrent tests, and calculates the causal strength of indicator A on indicator B: When the concurrency is greater than 200, the high group mean of IO waiting time is calculated When the concurrency is ≤ 200, the low group mean of IO waiting time is calculated ; Through the formula Quantifying causal relationships, is the standard deviation of indicator B, which is used to normalize the numerator. By comparing the high / low groups, it quantifies the impact of the change in indicator A on indicator B. The larger the absolute value, the more significant the impact of A on B. is the timing constraint factor. When the change of indicator A occurs before the change of indicator B ,otherwise , which is used to ensure the temporal directionality of the causal relationship and avoid misjudging the reverse causality of "B→A" as "A→B".

[0022] The data acquisition module also includes a dynamic hysteresis window unit, which, for distributed database testing, traverses the 1-500ms hysteresis interval and searches for the maximum correlation period between CPU usage and IO wait time. For example, if an IO bottleneck is found 300ms after the CPU usage increases, the hysteresis period is automatically marked as the causal delay response time to avoid misinterpreting synchronization fluctuations as causal relationships.

[0023] The data collection module also includes a metadata-driven association unit and a scene clustering unit. The metadata-driven association unit parses the table structure metadata and extracts features such as field access frequency and index type: B-tree index is encoded as 1 and hash index is encoded as 0; field access frequency is normalized to the interval [0,1]; indicator association factors are learned through a single-layer neural network , which indicates the domain correlation strength between indicators A and B. The larger the value, the stronger the correlation between the two in terms of database principles. ,in is the input feature vector, 、 The database metadata corresponding to indicators A and B, such as table structure, index configuration, field access frequency, etc. 、 It is a metadata feature extraction function that converts the original metadata into numerical features, such as encoding the index type as 0 / 1 and normalizing the field access frequency to [0,1]. is the number of co-occurrences of indicators A and B in historical test data, reflecting the statistical correlation between the two; W is the weight parameter of the neural network, which is obtained by training the historical causal graph data and is used to quantify the importance of each input feature; activation function For the Sigmoid function: , map the output value to the [0,1] interval; The scene clustering unit uses the DBSCAN algorithm to cluster the historical test scene features, calculates the cosine similarity between the current scene and the historical cluster center, and generates the scene weight Used to adjust the evaluation dimension, , where T represents the current scene feature vector, the scene features extracted from the test data, Represents the historical cluster center, the center vector obtained by clustering historical test scenes using the DBSCAN algorithm. Represents a typical scenario. To calculate the current scene T and the historical cluster center The directional similarity is used to measure the similarity between the current scene and the historical scene. The larger the value, the closer the scene is. The weight of the historical cluster center is determined by the amount of test data contained in the cluster. The larger the amount of data, the higher the weight.

[0024] The assessment and decision-making module constructs a three-dimensional assessment model based on the causal graph generated by the intelligent analysis module, which includes risk level, impact scope and repair cost. Among them, the risk level combines the vulnerability CVSS score and the database business importance weight; the impact scope identifies the cascading effects of upstream and downstream systems through causal chain propagation analysis; and the repair cost is predicted based on the repair case library of historical test data.

[0025] The display and storage module includes a natural language generation unit, which converts the technical indicators of the evaluation and decision-making module into business-readable reports containing natural language descriptions of the causal chain and optimization suggestions; an interactive visualization unit, which displays the causal graph in the form of a dynamic graph, allowing users to explore the causal relationships of test data through interactive operations; and a scenario adaptation module, which extracts scenario features from test data, such as concurrency patterns, SQL type distribution, or transaction ratios; and automatically loads the corresponding evaluation strategy based on these scenario features, which includes indicator weight configuration and threshold adjustment rules.

[0026] The scenario adaptation module clusters historical test scenarios through machine learning models, generates a mapping relationship between scenario feature vectors and evaluation strategies, and realizes dynamic adaptation of the evaluation model under different database application scenarios.

[0027] The system has incremental learning capabilities and continuously optimizes the evaluation model through the following methods: using new test data to update the parameters of metadata-driven association units; and adjusting the feature mapping relationship of scene clustering adaptation units based on historical evaluation results.

[0028] The technical scope of the present invention is not limited to the contents of the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. An intelligent evaluation system based on database software test results, characterized in that: It includes data acquisition module, intelligent analysis module, evaluation and decision module and display storage module, and each module works together through standardized data flow; The data acquisition module acquires multi-source test data including but not limited to performance indicators and function logs, and outputs cleaned feature vectors; The intelligent analysis module generates indicator correlation maps through causal modeling; The assessment and decision-making module outputs risk assessment and root cause location results based on the graph; The display storage module presents results in visualization and natural language reports and stores historical data.

2. The intelligent evaluation system based on database software test results according to claim 1, characterized in that: The data acquisition module includes: Multi-source interface unit, connects to MySQL stress test data through JDBC protocol, and obtains MongoDB compatibility test reports through REST API; The data cleaning unit uses discrete wavelet transform to denoise CPU utilization data, processes IOPS outliers through median filtering, and outputs standardized data with a unified sampling frequency of 100 Hz.

3. The intelligent evaluation system based on database software test results according to claim 2, characterized in that: The intelligent analysis module includes a time-series grouping intervention unit that divides concurrent test data into high / low groups according to thresholds and calculates causal strength using the formula: ,in It is expressed as the average value of indicator B when indicator A is in the high intervention group. It means the average value of indicator B when indicator A is in the low intervention group. is the standard deviation of indicator B, which is used to normalize the numerator. By comparing the high / low groups, it quantifies the impact of the change in indicator A on indicator B. The larger the absolute value, the more significant the impact of A on B. is the timing constraint factor. When the change of indicator A occurs before the change of indicator B ,otherwise , used to ensure the time directionality of causal relationships; dynamic hysteresis window unit, traverses the 1-500ms hysteresis interval and searches for the maximum correlation hysteresis period between IO waiting time and CPU occupancy.

4. The intelligent evaluation system based on database software test results according to claim 3, characterized in that: The intelligent analysis module further comprises: The metadata drives the association unit, parses the table structure metadata, and learns the indicator association factor through a single-layer neural network , which indicates the domain correlation strength between indicators A and B. The larger the value, the stronger the correlation between the two in terms of database principles. ,in is the input feature vector, 、 is the database metadata corresponding to indicators A and B, 、 It is a metadata feature extraction function that converts the original metadata into numerical features. is the number of co-occurrences of indicators A and B in the historical test data, reflecting the statistical correlation between the two; W is the weight parameter of the neural network, which is obtained by training the historical causal graph data and is used to quantify the importance of each input feature; Activation Function For the Sigmoid function: , mapping the output value to the [0,1] interval.

5. The intelligent evaluation system based on database software test results according to claim 4, characterized in that: The scene clustering unit uses DBSCAN to cluster the test scene features. Generate scene weights Used to adjust the evaluation dimension, , where T represents the current scene feature vector, the scene features extracted from the test data, Represents the historical cluster center, the center vector obtained by clustering historical test scenes using the DBSCAN algorithm. Represents a typical scenario. To calculate the current scene T and the historical cluster center The directional similarity is used to measure the similarity between the current scene and the historical scene. The larger the value, the closer the scene is. The weight of the historical cluster center is determined by the amount of test data contained in the cluster. The larger the amount of data, the higher the weight.

6. The intelligent evaluation system based on database software test results according to claim 5, characterized in that: The assessment and decision-making module constructs a three-dimensional assessment model including risk level, impact scope and repair cost based on the cause-effect diagram generated by the intelligent analysis module, wherein: The risk level combines the vulnerability CVSS score and the database business importance weight; The scope of impact identifies cascading effects in upstream and downstream systems through causal chain propagation analysis; Repair costs are predicted based on a repair case library of historical test data.

7. The intelligent evaluation system based on database software test results according to claim 1, characterized in that: The display storage module includes: A natural language generation unit converts the technical indicators of the evaluation and decision-making module into a business-readable report, wherein the report includes a natural language description of the causal chain and optimization suggestions; The interactive visualization unit displays the causal diagram in the form of a dynamic graph, supporting users to explore the causal relationship of test data through interactive operations.

8. The intelligent evaluation system based on database software test results according to claim 1, characterized in that: It also includes a scene adaptation module for: Extract scenario features from test data, including concurrency patterns, SQL type distribution, or transaction ratios; The corresponding evaluation strategy is automatically loaded based on the scenario characteristics, and the evaluation strategy includes indicator weight configuration and threshold adjustment rules.

9. The intelligent evaluation system based on database software test results according to claim 1, characterized in that: The scenario adaptation module clusters historical test scenarios through a machine learning model, generates a mapping relationship between scenario feature vectors and evaluation strategies, and realizes dynamic adaptation of the evaluation model under different database application scenarios.

10. The intelligent evaluation system based on database software test results according to any one of claims 1 to 9, characterized in that: The system has incremental learning capabilities and continuously optimizes the evaluation model through the following methods: Updating the parameters of the metadata-driven association unit using the newly added test data; The feature mapping relationship of the scene clustering adaptation unit is adjusted based on the historical evaluation result.

Images