Intelligent pressure measurement report automatic generation method and system

Abstract

The invention discloses an intelligent pressure measurement report automatic generation method and system. The method comprises the steps of performing multi-source fusion on pressure measurement result data, a historical case library and a domain knowledge graph; identifying a pressure measurement scene according to the multi-dimensional metadata of the pressure measurement task; designing a structured Prompt template, and filling the Prompt template according to a pressure measurement task; after converting the multi-modal data into an input format supported by a large language model, splicing the multi-modal data with a Prompt template to obtain an input text, and triggering the large language model for reasoning to obtain a reasoning result; the reasoning result is converted into executable steps, code examples are labeled, and an intelligent pressure measurement report is generated in combination with a report template embedded with a trend chart and a root cause analysis tree. The reasoning accuracy is improved, unnecessary computing resource consumption is reduced, and the suggestion generation time is shortened. And through a self-adaptive report template and a multi-format export function, the report generation time is shortened, and the labor cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of software performance testing technology, and in particular to a method and system for automatically generating intelligent load testing reports. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of software performance testing, traditional load testing report generation relies on manual analysis of test results data (such as TPS, response time, error rate, etc.), which has the following technical shortcomings: 1. Inefficient: It requires manual correlation of logs, monitoring data and business scenarios, which is time-consuming and prone to missing key anomalies, resulting in a report generation cycle of up to several hours.

[0004] 2. High dependence on experience: Root cause diagnosis relies on the experience of experts, making it difficult for newcomers to quickly locate the problem, and the diagnosis results are affected by subjective factors.

[0005] 3. Report standardization discrepancy: Manually written reports have inconsistent formats and fail to adequately cover the feasibility of optimization suggestions, resulting in inconsistent report quality.

[0006] With the breakthroughs in the field of natural language processing using Large Language Models (LLM), there is an urgent need for a technical solution to deeply integrate LLM with load testing data, thereby automating and intelligently generating reports to address the aforementioned technical shortcomings. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes an intelligent automated stress testing report generation method and system. This system can automatically adjust the prompt content based on the stress testing scenario, improving inference accuracy while reducing unnecessary computational resource consumption and shortening suggestion generation time. Through adaptive report templates and multi-format export functionality, report generation time is reduced, lowering labor costs.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for automatically generating intelligent stress test reports, comprising: The acquired stress test results data, historical case library and domain knowledge graph are spatiotemporally aligned and then fused from multiple sources. Design historical scenario tags and construct a structure diagram of load testing scenarios, load testing task features, and tags to identify load testing scenarios based on the multi-dimensional metadata of load testing tasks. Based on the multi-dimensional metadata of load testing tasks, the historical case library and domain knowledge graph related to the corresponding load testing scenarios, and the specified output format, design a structured Prompt template and populate the variables in the Prompt template with the multi-dimensional metadata of load testing tasks. After converting the fused multimodal data into an input format supported by the large language model, it is concatenated with the Prompt template to obtain the input text, and the large language model is triggered to perform inference to obtain inference results including anomaly type, root cause components and remediation suggestions. The inference results are transformed into executable steps and annotated with code examples. Combined with report templates that embed trend charts and root cause analysis trees, intelligent load testing reports are generated.

[0009] As an optional implementation, the load testing results data includes time-series metrics, logs, and call chain information, which are standardized through a unified data model, defining common field specifications, and enabling cross-data source association. Specifically, logs are used to extract events and mark the timestamps, impact range, and associated metrics of abnormal events. Call chain information is associated by linking all service nodes based on unique IDs to build an end-to-end call chain graph, associating logs and time-series metrics within the same request lifecycle. A historical case library is used to store abnormal patterns, root cause components, and remediation strategies from historical load tests. It is built using a graph database, with node types including abnormal types, root cause components, and remediation strategies, and edge relationships defined as causing and mitigating.

[0010] As an optional implementation method, multi-dimensional metadata includes basic dimensions, behavioral dimensions, environmental dimensions, and target dimensions. Among them, the basic dimensions include the number of concurrent users, test duration, request type, and data volume; the behavioral dimensions include user operation paths, transaction ratio, and thinking time distribution; the environmental dimensions include network topology, geographical distribution, and terminal type; and the target dimensions include SLA requirements and fault tolerance.

[0011] As an optional implementation method, historical scenario tags are designed, including basic tags, extended tags, and associated tags; among them, basic tags include flash sales, high-concurrency read and write, and mixed load; extended tags include hot and cold data separation, distributed transactions, and cache penetration; associated tags are used to establish mapping relationships with monitoring metrics; A graph database is introduced to construct the relationship between load testing scenarios, load testing task features, and labels, and the structure graph is dynamically updated through an incremental learning mechanism.

[0012] As an alternative implementation, a structured Prompt template is designed, including task instructions, background information, additional content, and output requirements. The task instructions field explicitly specifies the tasks that the large language model needs to complete; the background information field provides multi-dimensional metadata of the load testing task; the additional content field loads the domain knowledge base and historical case library related to the load testing scenario; and the output requirements field specifies the format and content of the output results of the large language model. The variables in the Prompt template are populated based on the multi-dimensional metadata of the load testing task. A prompting strategy of starting with fewer prompts and gradually increasing them is adopted to decompose complex problems into multiple sub-problems and populate the Prompt template context step by step.

[0013] As an alternative implementation, the structured layout of the report template includes a test overview, performance bottleneck analysis, and optimization suggestions, and adopts an adaptive layout strategy, loading different CSS files according to the screen width. The trend chart supports line charts and bar charts, and allows configuration of chart parameters; the root cause analysis tree embedding includes building a hierarchical relationship diagram to show the propagation path of abnormal events, and supports node expansion / collapse interaction; The generated intelligent load testing report supports export in multiple formats. Specifically, it uses the Jinja2 template engine to render structured data into Markdown format, supporting code blocks and table syntax. It also uses the reportlab library to convert Markdown content into PDF files, automatically paginating and adding headers / footers, and supports custom page settings, pagination strategies, and header / footer styles.

[0014] Secondly, this invention provides an automated intelligent stress test report generation system, comprising: The fusion module is configured to perform multi-source fusion of the acquired stress test results data, historical case library and domain knowledge graph after spatiotemporal alignment; The Prompt template generation module is configured to design historical scenario tags and construct a structure diagram of load testing scenarios, load testing task features, and tags to identify load testing scenarios based on the multi-dimensional metadata of load testing tasks. Based on the multi-dimensional metadata of load testing tasks, the historical case library and domain knowledge graph related to the corresponding load testing scenarios, and the specified output format, a structured Prompt template is designed, and the variables in the Prompt template are populated based on the multi-dimensional metadata of load testing tasks. The inference module is configured to convert the fused multimodal data into an input format supported by the large language model, concatenate it with the Prompt template to obtain the input text, and trigger the large language model to perform inference to obtain inference results including anomaly type, root cause components and remediation suggestions. The export module is configured to transform inference results into executable steps and annotate code examples, and generate intelligent load testing reports by combining report templates with embedded trend charts and root cause analysis trees.

[0015] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0016] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0017] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Efficiency Improvement: Accelerated data preprocessing: By using the OpenTelemetry unified data model and Bi-LSTM network log event extraction, the data preprocessing time is shortened.

[0019] Dynamic Prompt Generation: Compared to traditional static Prompts, it can automatically adjust the Prompt content according to the load testing scenario, improving the inference accuracy of LLM while reducing unnecessary computational resource consumption.

[0020] Executable suggestion generation: By combining regular expression parameter extraction and domain rule engine validation, suggestion generation time is shortened.

[0021] Automated report generation: Reduce report generation time and lower labor costs through adaptive report templates and multi-format export functionality.

[0022] 2. Improved accuracy: Historical case association: By storing historical cases in a graph database, similar cases can be automatically associated, improving the accuracy of root cause localization.

[0023] Domain knowledge graph: Integrates best practices for performance optimization to improve the feasibility and effectiveness of recommendations.

[0024] Multi-round iterative reasoning: By adopting a prompting strategy from few to many, the prompt context can be gradually filled, reducing the difficulty of LLM reasoning and improving the solution rate of complex problems.

[0025] 3. Knowledge Accumulation: Dynamic knowledge base updates: The case library and knowledge graph are automatically updated with each load test, forming an organization-level knowledge asset for performance optimization.

[0026] Historical data value mining: Through in-depth analysis of historical stress test data, potential performance bottlenecks and optimization points can be discovered.

[0027] 4. Improved report standardization and readability.

[0028] Adaptive report template: Achieves a responsive layout through CSS3 Media Query to ensure consistent report display across different devices.

[0029] Visualization components: Generate trend charts and root cause analysis trees using the ECharts and D3.js libraries, making complex data easier to understand.

[0030] Multi-format export: Supports exporting in multiple formats such as Markdown and PDF to meet the reporting needs of different scenarios.

[0031] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0033] Figure 1 The flowchart is for the automated generation method of intelligent stress test report provided in Embodiment 1 of the present invention. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0036] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0038] Example 1 This embodiment proposes an automated method for generating intelligent load testing reports based on Large Language Model (LLM), such as... Figure 1 As shown, it includes: The acquired stress test results data, historical case library and domain knowledge graph are spatiotemporally aligned and then fused from multiple sources. Design historical scenario tags and construct a structure diagram of load testing scenarios, load testing task features, and tags to identify load testing scenarios based on the multi-dimensional metadata of load testing tasks. Based on the multi-dimensional metadata of load testing tasks, the historical case library and domain knowledge graph related to the corresponding load testing scenarios, and the specified output format, design a structured Prompt template and populate the variables in the Prompt template with the multi-dimensional metadata of load testing tasks. After converting the fused multimodal data into an input format supported by the large language model, it is concatenated with the Prompt template to obtain the input text, and the large language model is triggered to perform inference to obtain inference results including anomaly type, root cause components and remediation suggestions. The inference results are transformed into executable steps and annotated with code examples. Combined with report templates that embed trend charts and root cause analysis trees, intelligent load testing reports are generated.

[0039] This embodiment provides a comprehensive analytical foundation for LLM through multimodal data fusion. During the dynamic prompt generation process, contextual prompts required for LLM inference are automatically generated based on the stress testing scenario (such as flash sales or high-concurrency read / write operations), improving inference accuracy. Based on the LLM output results, optimization schemes containing specific code snippets or configuration parameters are generated in conjunction with the rule engine to ensure the operability of the recommendations. In terms of adaptive report templates, it supports export in multiple formats such as Markdown / PDF and embeds visualization components such as trend charts and root cause analysis trees to enhance the readability and professionalism of the reports.

[0040] The method of this embodiment will be described in detail below.

[0041] I. Multimodal data fusion engine.

[0042] 1. Data source integration, including load testing results data, historical case library and domain knowledge graph.

[0043] Specifically: (1) Load test result data: including time series indicators (such as TPS, P99 latency), logs (including error stacks), Trace (call chain information) and other data, and standardized through the OpenTelemetry unified data model, defining common field specifications (such as trace_id, service.name, timestamp) to achieve cross-data source association.

[0044] Among them, the OpenTelemetry unified data model defines a set of general field specifications covering the entire lifecycle of load testing, including but not limited to: trace_id (globally unique trace ID), service.name (service name), timestamp (timestamp), metric.name (metric name), log.level (log level), etc.

[0045] By using OpenTelemetry's SDK and API, standardized collection and transmission of load testing results data (time series metrics, logs, traces) can be achieved, ensuring data format consistency across different data sources and facilitating subsequent correlation analysis.

[0046] Example: When collecting database connection pool metrics, use db.connection_pool.size as the metric name and associate it with the specific service.name (e.g., user-service) and trace_id to facilitate problem tracking.

[0047] (2) Historical Case Library: Stores abnormal patterns, root cause components and remediation strategies in historical load testing. It is constructed using a graph database (Neo4j). Node types include abnormal types, components and remediation strategies, and edge relationships are defined as causing / mitigating.

[0048] (3) Domain knowledge graph: Integrate best practices for performance optimization, such as database connection pool parameter tuning rules and cache invalidation strategies, and extract structured knowledge from technical documents through knowledge extraction techniques (such as Bi-LSTM-CRF).

[0049] 2. Data preprocessing.

[0050] (1) Log event extraction: Use Bi-LSTM network to extract events from the load test logs and mark the timestamps, impact range and related indicators of abnormal events.

[0051] For example, by using a bidirectional LSTM network, contextual information in logs can be captured, abnormal patterns in logs can be identified (such as connection pool wait queue exceeding limits), and entity boundary recognition can be optimized by combining a CRF layer to improve event extraction accuracy.

[0052] The network structure includes an input layer, a bidirectional LSTM layer, a CRF layer, and an output layer. During training, a large amount of labeled log data is used, and the network parameters are optimized through the backpropagation algorithm to improve the accuracy of anomaly extraction.

[0053] For example, when dealing with connection pool wait queue overrun events in the logs, the Bi-LSTM network can accurately identify the event type, scope of impact, and related indicators, providing strong support for subsequent analysis.

[0054] (2) Trace data association: Based on the unique trace_id, all service nodes are connected to build an end-to-end call chain graph and automatically associate logs and metrics data within the same request lifecycle.

[0055] 3. Data fusion.

[0056] (1) Spatiotemporal alignment engine: Deploy a spatiotemporal alignment engine at the edge node to achieve a unified spatiotemporal reference for data through spatiotemporal synchronization calibration technology (such as NTP protocol) to ensure the alignment of multi-source data in the time dimension (error <5ms).

[0057] The spatiotemporal alignment engine also features data caching and preprocessing capabilities, enabling it to perform preliminary cleaning and format conversion on received data, thereby improving the efficiency of subsequent analysis.

[0058] (2) Spatiotemporal alignment strategy for edge node deployment of heterogeneous computing platform DTS-MAX.

[0059] Based on Mirror Vision's self-developed DTS-MAX heterogeneous computing platform, integrated processing of video stream preprocessing, point cloud processing, and data fusion is achieved. Video stream preprocessing utilizes GPU-accelerated CNN feature extraction technology to perform real-time analysis of the video stream and extract key feature information. Point cloud processing employs a fast planar segmentation algorithm implemented on FPGA to efficiently process point cloud data and extract planar features. Data fusion combines the results of video stream preprocessing and point cloud processing to generate richer environmental perception information, providing strong support for stress testing and analysis.

[0060] Specifically, the platform breaks through the limitations of traditional spatiotemporal synchronization calibration technology through the following innovative process, forming a set of efficient, low-latency, and adaptive multimodal data fusion solutions.

[0061] (1) Core Improvement Logic: From Explicit Calibration to Implicit Alignment of Feature Domain; Traditional spatiotemporal synchronization techniques rely on physical calibration (such as checkerboard calibration) or explicit parameter matching (such as timestamp alignment and spatial coordinate transformation), while DTS-MAX adopts an implicit alignment strategy of feature domain. Its innovation lies in: (1-1) Cross-modal feature decoupling.

[0062] The video stream is preprocessed using GPU-accelerated CNNs (such as an optimized version of ResNet-152) to extract high-level semantic features (such as object contours and motion trajectories). At the same time, FPGA is used to achieve fast planar segmentation of laser point clouds (such as an improved RANSAC algorithm) to transform the 3D point cloud into structured planar parameters.

[0063] The key improvement lies in establishing the association between video and point cloud at the feature level, rather than directly aligning the original data, thus avoiding alignment errors caused by cross-modal data format differences.

[0064] (1-2) Spatiotemporal consistency constraint propagation.

[0065] A spatiotemporal alignment engine is deployed at the edge nodes, and the feature stream is dynamically calibrated using an LSTM network constrained by spatiotemporal continuity. The network takes inter-frame optical flow and point cloud planar motion vectors as inputs to predict the amount of spatiotemporal alignment error compensation (error < 5ms).

[0066] The key improvement lies in transforming spatiotemporal alignment into a recursive prediction problem, optimizing the current alignment accuracy through historical state feedback, rather than relying on static calibration parameters.

[0067] (2) Support for distributed edge computing architecture.

[0068] (2-1) Heterogeneous resource collaborative scheduling; including: Task layering and mapping: The spatiotemporal alignment task is decomposed into three types of sub-tasks: feature extraction, which GPUs excel at; geometric calculation, which FPGAs excel at; and global coordination, which CPUs excel at. These sub-tasks are then dynamically allocated to edge nodes through the task scheduler of the DTS-MAX platform.

[0069] Data stream optimization: Zero-copy transmission technology (such as GPUDirect RDMA) is used to reduce CPU intervention, enabling video stream and point cloud data to be exchanged directly between GPU / FPGA, reducing communication latency.

[0070] (2-2) Deployment of lightweight alignment model; By deploying pruned lightweight CNN and quantized LSTM models at edge nodes, the model size is compressed to 1 / 10 of the original model, and the inference speed is improved by more than 3 times.

[0071] Adaptive parameter adjustment: Automatically switch alignment models based on scene dynamics (such as target movement speed and lighting changes) (e.g., use optical flow for high-speed scenes and feature matching for low-speed scenes).

[0072] (3) Dynamic error compensation mechanism.

[0073] (3-1) Multi-source error modeling: Construct a joint probability model that includes error sources such as sensor noise, clock drift, and motion distortion, and estimate the alignment error distribution in real time through Bayesian inference.

[0074] (3-2) Closed-loop feedback correction: The aligned data is input into the reconstruction algorithm (such as the spatiotemporal continuity constraint network based on LSTM), and the alignment parameters are adjusted by backpropagation based on the residuals of the reconstruction results and the original data, forming a closed-loop optimization of "alignment-reconstruction-calibration".

[0075] (4) The advantages compared with traditional technologies are shown in Table 1.

[0076] Table 1. Comparative Advantages;

[0077] II. Dynamic Prompt Generation Module.

[0078] 1. Load testing scenario identification: Automatic identification of load testing scenarios based on metadata analysis and rule engine, combined with dynamic learning and adaptive optimization to improve the accuracy and efficiency of scenario identification.

[0079] Specifically: 1-1. Standardized collection and preprocessing of metadata.

[0080] (1) Collect multi-dimensional metadata, including: Basic dimensions: number of concurrent users, test duration, request type (GET / POST), and data volume; Behavioral dimensions: user operation path, transaction ratio (e.g., "browse: place order: pay = 5:3:2"), and thought time distribution; Environmental dimensions: network topology (ratio of internal and external networks), geographical distribution, and terminal type (PC / Mobile). Target dimensions: SLA requirements (response time < 500ms), fault tolerance (allowable error rate < 0.1%).

[0081] (2) Data cleaning and feature enhancement, including: Outlier handling: Filter out concurrency numbers that deviate significantly from normal business conditions (e.g., set to mean ±3σ).

[0082] For example: Calculate operation intensity = number of transactions / test duration; generate data skew = hot data access volume / total access volume; extract time series features = peak request period distribution (such as the 0:00 burst during Double 11).

[0083] 1-2. Construction of historical scene knowledge graph.

[0084] (1) Design a scene tagging system, including: Basic tags: flash sale, high-concurrency read / write, mixed load.

[0085] Extended tags: hot and cold data separation, distributed transactions, cache penetration.

[0086] Association tags: Establish a mapping relationship with monitoring metrics (such as CPU utilization, disk IOPS).

[0087] (2) Dynamic updates to the knowledge graph; Introduce a graph database (such as Neo4j) to store the relationship between scene, feature, and label.

[0088] Incremental learning mechanism: Automatic categorization of new scenarios: When no tag is matched, manual review is triggered and the scenario is added to the knowledge base; Tag weight decay: The weight of tags in old scenarios decays over time (e.g., the weight is halved after six months).

[0089] 1-3. Rule Engine Optimization Design: A scenario recognition system based on the rule engine is built, capable of parsing metadata of load testing tasks (such as concurrent users and test duration) and combining it with historical scenario tags (such as flash sales and high-concurrency read / write) to automatically identify load testing scenarios. The rule engine supports dynamic updates and expansion, allowing the addition or modification of scenario recognition rules according to actual needs. Example: When the number of concurrent users in a load testing task exceeds 5000, the rule engine automatically identifies it as a high-concurrency scenario and loads the corresponding domain knowledge graph.

[0090] Specifically: (1) Layered rule matching.

[0091] (1-1) Coarse-grained filtering: Concurrent users > 100,000 → trigger high-concurrency scenarios; Read-to-write ratio > 10:1 → mark as read-heavy and write-light.

[0092] (1-2) Fine-grained reasoning: If the data skewness is >80% and the transaction type includes inventory deduction → it is identified as a flash sale scenario; combine the time series characteristics and operation intensity to determine whether it is wave-like pressure.

[0093] (2) Fuzzy matching and conflict resolution: Dempster-Shafer evidence theory is introduced to deal with uncertain features.

[0094] Example: When the concurrency is 50,000 (medium) but the transaction type includes distributed locks, increase the confidence level for high-concurrency read and write operations; Priority of conflict rules: Business objectives > Behavioral characteristics > Basic parameters.

[0095] 1-4. Adaptive optimization closed loop.

[0096] (1) Feedback learning mechanism; After the load test is completed, compare the automatic identification results with the manually labeled results, calculate the accuracy / recall rate, and adjust the rule thresholds (such as the lower limit of the number of concurrent users in high-concurrency scenarios).

[0097] Example: If the false positive rate in a flash sale scenario is >20%, then refine the data skewness threshold.

[0098] (2) Online optimization and hot update; the rule engine supports dynamic loading of new rules without restarting the service; A / B testing is implemented: new and old rules run in parallel, and the best rule is promoted based on its performance.

[0099] 1-5. Visualization and interpretability.

[0100] (1) Scene recognition report; display the matching rule chain (e.g., concurrent number > 80,000 → high concurrency → successful verification of flash sale feature); confidence score quantifies the contribution of each feature to scene recognition.

[0101] (2) Abnormal feature warning; mark features that are significantly different from historical scenarios (such as an abnormally high read / write ratio in this test) and provide optimization suggestions, such as suggesting adding cache layer stress testing.

[0102] (3) Process advantages; Improved accuracy: By associating features through knowledge graphs, misjudgments based on single parameters are reduced; Dynamic adaptability: A closed-loop learning mechanism ensures that rules continue to evolve; Enhanced interpretability: Visualizing the rule chain aids in problem localization; Context enhancement: Load domain knowledge bases based on scenario type (such as prioritizing the association of inventory deduction, rate limiting strategies, etc. in flash sale scenarios) to enrich the background information of the Prompt.

[0103] 2. Prompt template design: Design a structured prompt template that includes fields such as task instructions, background information, additional content, and output requirements.

[0104] The task instruction field explicitly specifies the task that the LLM needs to complete (such as analyzing system performance bottlenecks); the background information field provides basic information about the load testing task (such as the number of concurrent users and the test duration); the additional content field loads the domain knowledge base and historical case library related to the load testing scenario; and the output requirements field specifies the format and content of the LLM output results (such as a diagnostic report in JSON format).

[0105] For example, a Prompt template designed for flash sale scenarios.

[0106]

[0107] 3. Dynamic filling and optimization.

[0108] Parametric population: Dynamically populate variables in the template (such as the number of concurrent users and the test duration) according to the load testing task to ensure the personalization of the Prompt.

[0109] Multi-round iteration: For complex scenarios, a least-to-most prompt strategy is adopted to decompose complex problems into multiple sub-problems and gradually fill the Prompt context to reduce the difficulty of LLM inference.

[0110] Example: When analyzing system performance bottlenecks in a flash sale scenario, first use Prompt to guide LLM analysis to find the cause of the sudden drop in TPS, then locate the root cause component, and finally generate optimization suggestions.

[0111] III. Executable suggestion generator.

[0112] 1. LLM inference stage; This solution breaks through the static input-output model of traditional LLM inference and proposes a closed-loop architecture based on multimodal perception and dynamic reasoning. The core innovations are reflected in three levels: adaptive input encoding, progressive inference engine, and semantic routing parsing, forming a full-link optimization from data fusion to decision output.

[0113] Specifically: (1) LLM input processing: a dynamic coding layer driven by multimodal perception; converting the fused multimodal data into an input format (such as word sequence) that LLM can understand, combining it with the dynamically generated Prompt template, and concatenating it with the Prompt template to form a complete input text, triggering LLM to perform inference.

[0114] Specifically, it includes: (1-1) Cross-Modal Semantic Alignment Network (CMSAN).

[0115] Technological Breakthrough: Traditional multimodal fusion employs simple splicing or attention mechanisms. This solution designs a hierarchical semantic alignment network, including: Low-level feature alignment: Primary features are extracted by modality-specific encoders (such as Vision Transformer for image processing and BERT for text processing), and contrastive learning is used to force alignment of semantic spaces of different modalities (such as mapping the feature vectors of high-temperature areas in infrared thermal imaging images to the same latent space).

[0116] High-level semantic fusion: A graph attention network (GAT) is used to construct a graph of intermodal relationships, dynamically calculate modal weights (such as equipment vibration data having higher weight in mechanical fault diagnosis), and generate context vectors for modal perception.

[0117] (1-2) Context-Aware Prompt Generator (CPG).

[0118] Meta-learning driven: A prompt template library is trained based on historical task data, and MAML (Model-Agnostic Meta-Learning) is used to achieve rapid adaptation in scenarios with few samples. For example, in wind power equipment diagnosis, the "blade angle anomaly detection dedicated prompt" is automatically selected based on the current equipment model and historical fault modes.

[0119] Reinforcement learning optimization: The effectiveness of the Prompt is used as a reward signal, and the generation strategy is adjusted in real time through the PPO algorithm. For example, when a sudden change in device parameters is detected, an "emergency shutdown threshold check" instruction is automatically inserted into the Prompt.

[0120] (2) Inference engine: Progressive multi-task inference architecture.

[0121] (2-1) Hierarchical Decision Pipeline (HDP); Cascaded reasoning mechanisms include: Coarse-grained screening: Use a lightweight expert model (such as TinyBERT) for initial classification to filter obviously normal samples (such as temperature fluctuations within the threshold) and reduce the computational load of the main model.

[0122] Fine-grained analysis: For suspected anomalous samples, the master LLM (such as GPT-4 Turbo) is activated, and root cause reasoning is performed in conjunction with the knowledge graph. For example, when an abnormal motor current is detected, the topology graph of the linkage equipment is used to trace the status of related components.

[0123] (2-2) Uncertainty-Aware Inference (UAI).

[0124] Confidence level dynamic calibration includes: Multi-dimensional evaluation: It not only outputs prediction results, but also generates confidence scores and dispute point markers (such as the presence of modal aliasing in vibration spectrum analysis, which suggests supplementing with high-frequency sampling data).

[0125] Active learning loop: When the confidence level is lower than the threshold, the data back-collection process is triggered (such as requiring edge nodes to supplement specific sensor data), forming a closed loop of inference-feedback-optimization.

[0126] (3) Output parsing: Semantic routing driven decision tree.

[0127] (3-1) Structured Output Parser (SOP); including multimodal response decoding: Dynamic JSON Schema Adaptation: Automatically validates the integrity of LLM output based on the preset output format in the Prompt (e.g., the diagnostic report should include three parts: exception type, root cause component, and repair recommendations).

[0128] Semantic routing mechanism: When parsing unstructured text into structured data, a rule-based routing strategy is employed. For example, when the keyword "bearing wear" is detected, the corresponding repair process is automatically activated by querying the equipment maintenance knowledge base.

[0129] (3-2) Root Cause Graph Generator (RCGG).

[0130] Visualization of causal relationships: Dynamic graph construction: Extract entity relationships (such as "insufficient lubrication → bearing overheating → abnormal vibration") from the root cause analysis text output by LLM, and use the Neo4j graph database to build the fault propagation chain in real time.

[0131] Interactive diagnostics: Allows users to click on graph nodes to trace back to original data evidence (such as retrieving vibration spectrum graphs at specific time points), thereby improving the interpretability of decisions.

[0132] In this embodiment, three major technological breakthroughs are achieved by constructing a closed-loop system integrating perception, reasoning, and analysis: Dynamic adaptability: Shifting from static model invocation to context-aware proactive reasoning, enabling LLM to possess a diagnostic thought process similar to that of an expert; Efficiency optimization: By using cascaded reasoning and uncertainty awareness, computational resource consumption is reduced by 30%+ while maintaining accuracy; Transparent decision-making: Transform black-box outputs into traceable, graph-based evidence chains to meet the stringent interpretability requirements of industrial scenarios.

[0133] 2. Post-processing stage.

[0134] (1) Regular expression parameter extraction: Extract key parameters from the LLM output using regular expressions (e.g., connection pool values ​​must be integers between 50 and 500). Thoroughly test and optimize the regular expressions to ensure accurate extraction of the required parameters.

[0135] Example: Use the regular expression \\d+ to extract numeric parameters from LLM output. For example, use the regular expression \\d{3}-\\d{3}-\\d{4} to match phone number formats, ensuring parameter validity.

[0136] (2) Domain rule engine verification: Combine the domain rule engine (e.g., the maximum value of the database connection pool should not exceed 500) to verify the compliance of parameters and avoid generating invalid suggestions.

[0137] The rule engine supports dynamic updates and expansion, allowing you to add or modify validation rules as needed. Example: When the extracted connection pool value exceeds 500, the rule engine triggers a validation failure and prompts the user to modify the parameters.

[0138] 3. Recommended generation.

[0139] Code example generation: Transforms natural language suggestions into executable steps and annotates code examples. Provides code examples in multiple programming languages ​​(such as Java and Python) to meet the needs of different users.

[0140] Example: This section provides a Java code example demonstrating how to modify the connection pool configuration in response to suggestions regarding database connection pool exhaustion.

[0141]

[0142] Prioritization: Based on dimensions such as the scope of impact of the anomaly (e.g., the number of users affected) and the cost of repair (e.g., the amount of code modification), the optimization suggestions are prioritized and an executable roadmap is generated.

[0143] IV. Adaptive Report Templates.

[0144] 1. Template design.

[0145] Structured Layout: Define a structured template that includes three parts: test overview, performance bottleneck analysis, and optimization suggestions. Use the CSS3 Media Query module to achieve adaptive layout and load different CSS files (such as tinyScreen.css and smallScreen.css) according to the screen width.

[0146] Responsive design: through <meta name="viewport" content="width=device-width,initial-scale=1"> To make the webpage width adapt to the screen, use relative units (such as em) for font size, and set the maximum width of images to 100%.

[0147] CSS3 Media Query Responsive Design: Achieve adaptive layouts using the CSS3 Media Query module, loading different CSS files (such as tinyScreen.css and smallScreen.css) based on screen width. Example: Load tinyScreen.css on mobile devices, adjusting font size and layout to ensure reports are clearly displayed on small screens.

[0148] 2. Embedding of visual components.

[0149] ECharts Trend Chart Drawing: Use the ECharts library to generate trend charts (such as TPS change curves), supporting various types including line charts and bar charts. Configure chart parameters (such as axis labels, data series, legends, etc.) using the setOption method. Example: When drawing a TPS change curve, set the X-axis to the time axis, the Y-axis to the TPS value, and add a legend.

[0150] Root Cause Analysis Tree: Built using the D3.js library, this hierarchical graph displays the propagation path of anomalies (e.g., from a surge in user login requests to connection pool exhaustion). It supports expand / collapse node interactions for easy viewing of detailed information. Example: In the root cause analysis tree, users can expand / collapse child nodes by clicking on a node to view the detailed propagation path of the anomaly.

[0151] 3. Export in multiple formats.

[0152] Jinja2 Template Engine Markdown Export: Renders structured data into Markdown format using the Jinja2 template engine, supporting syntax such as code blocks and tables. Example: Populate a Markdown template with load testing results, historical case studies, and domain knowledge graph information to generate a structured report.

[0153] Reportlab PDF Generation: Use the Reportlab library to convert Markdown content into PDF files, automatically paginating and adding headers / footers (such as report title and generation time). It supports custom page settings, pagination strategies, and header / footer styles. Example: When generating a PDF report, set the page size to A4, the pagination strategy to automatic pagination, and include the report title and generation time in the header and footer.

[0154] The above-described solution in this embodiment has the following technical advantages.

[0155] 1. Efficiency Improvement: Accelerated data preprocessing: By using the OpenTelemetry unified data model and Bi-LSTM network log event extraction, the data preprocessing time is shortened.

[0156] Dynamic Prompt Generation: Compared to traditional static Prompts, this embodiment can automatically adjust the Prompt content according to the stress testing scenario, improving the inference accuracy of LLM while reducing unnecessary computational resource consumption.

[0157] Executable suggestion generation: By combining regular expression parameter extraction and domain rule engine validation, suggestion generation time is shortened.

[0158] Automated report generation: Reduce report generation time and lower labor costs through adaptive report templates and multi-format export functionality.

[0159] 2. Improved accuracy: Historical Case Association: By storing historical cases through a graph database (Neo4j), similar cases can be automatically associated, improving the accuracy of root cause localization. For example, when a sudden drop in TPS is detected during load testing, the problem of database connection pool exhaustion can be quickly located.

[0160] Domain knowledge graph: Integrates best practices for performance optimization, improving the feasibility and effectiveness of recommendations. For example, recommendations for tuning database connection pool parameters can provide specific code examples and configuration parameters.

[0161] Multi-round iterative reasoning: By adopting a prompting strategy from few to many, the prompt context can be gradually filled, reducing the difficulty of LLM reasoning and improving the solution rate of complex problems.

[0162] 3. Knowledge Accumulation: Dynamic knowledge base updates: The case library and knowledge graph are automatically updated with each load test, forming an organization-level knowledge asset for performance optimization.

[0163] Historical data value mining: Through in-depth analysis of historical stress test data, potential performance bottlenecks and optimization points can be discovered.

[0164] 4. Improved report standardization and readability.

[0165] Adaptive Report Template: Utilizing CSS3 Media Query, this template achieves a responsive layout, ensuring consistent report display across different devices. For example, when viewing a report on a mobile device, the font size and layout automatically adjust to ensure clear and readable content.

[0166] Visualization components: Trend charts and root cause analysis trees are generated using the ECharts and D3.js libraries, making complex data easier to understand. For example, trend charts can visually display the TPS (Transactions Per Second) curve, and root cause analysis trees can clearly show the propagation path of abnormal events.

[0167] Multi-format export: Supports exporting to multiple formats such as Markdown and PDF to meet reporting needs in different scenarios. For example, when collaborating in a team, it can be exported as Markdown for online editing and discussion; when reporting to management, it can be exported as PDF for printing and distribution.

[0168] It should be noted that all data acquisition is conducted in accordance with laws and regulations and with user consent, and the data is used legally.

[0169] Example 2 This embodiment provides an intelligent stress test report automated generation system, including: The fusion module is configured to perform multi-source fusion of the acquired stress test results data, historical case library and domain knowledge graph after spatiotemporal alignment; The Prompt template generation module is configured to design historical scenario tags and construct a structure diagram of load testing scenarios, load testing task features, and tags to identify load testing scenarios based on the multi-dimensional metadata of load testing tasks. Based on the multi-dimensional metadata of load testing tasks, the historical case library and domain knowledge graph related to the corresponding load testing scenarios, and the specified output format, a structured Prompt template is designed, and the variables in the Prompt template are populated based on the multi-dimensional metadata of load testing tasks. The inference module is configured to convert the fused multimodal data into an input format supported by the large language model, concatenate it with the Prompt template to obtain the input text, and trigger the large language model to perform inference to obtain inference results including anomaly type, root cause components and remediation suggestions. The export module is configured to transform inference results into executable steps and annotate code examples, and generate intelligent load testing reports by combining report templates with embedded trend charts and root cause analysis trees.

[0170] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0171] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0172] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0173] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0174] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0175] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0176] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0177] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0178] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0179] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0180] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for automatically generating intelligent stress test reports, characterized in that, include: The acquired stress test results data, historical case library and domain knowledge graph are spatiotemporally aligned and then fused from multiple sources. Design historical scenario tags and construct a structure diagram of load testing scenarios, load testing task features, and tags to identify load testing scenarios based on the multi-dimensional metadata of load testing tasks; Based on the multi-dimensional metadata of the load testing task, the historical case library and domain knowledge graph related to the corresponding load testing scenario, and the specified output format, a structured Prompt template is designed, and the variables in the Prompt template are populated according to the multi-dimensional metadata of the load testing task. After converting the fused multimodal data into an input format supported by the large language model, it is concatenated with the Prompt template to obtain the input text, and the large language model is triggered to perform inference to obtain inference results including anomaly type, root cause components and remediation suggestions. The inference results are transformed into executable steps and annotated with code examples. Combined with report templates that embed trend charts and root cause analysis trees, intelligent load testing reports are generated.

2. The method for automatically generating intelligent stress test reports as described in claim 1, characterized in that, The load test results include time-series metrics, logs, and call chain information, which are standardized through a unified data model, defining common field specifications and enabling cross-data source association. Specifically, logs are used to extract events and mark the timestamps, impact range, and related metrics of abnormal events; data association is performed on the call chain information, that is, all service nodes are linked based on unique IDs to build an end-to-end call chain graph and associate logs and time-series metrics within the same request lifecycle. The historical case library is used to store abnormal patterns, root cause components, and remediation strategies from historical load tests. It is built using a graph database, with node types including abnormal types, root cause components, and remediation strategies, and edge relationships defined as causing and mitigating.

3. The method for automatically generating intelligent stress test reports as described in claim 1, characterized in that, Multi-dimensional metadata includes basic dimensions, behavioral dimensions, environmental dimensions, and target dimensions. Among them, the basic dimensions include the number of concurrent users, test duration, request type, and data volume; the behavioral dimensions include user operation paths, transaction ratio, and thinking time distribution; the environmental dimensions include network topology, geographical distribution, and terminal type; and the target dimensions include SLA requirements and fault tolerance.

4. The method for automatically generating intelligent stress test reports as described in claim 1, characterized in that, The design includes historical scenario tags, such as basic tags, extended tags, and associated tags. The basic tags include flash sales, high-concurrency read / write, and mixed load. The extended tags include hot / cold data separation, distributed transactions, and cache penetration. The associated tags establish a mapping relationship with monitoring metrics. A graph database is introduced to construct the relationship between load testing scenarios, load testing task features, and labels, and the structure graph is dynamically updated through an incremental learning mechanism.

5. The method for automatically generating intelligent stress test reports as described in claim 1, characterized in that, Design a structured Prompt template, including task instructions, background information, additional content, and output requirements. The task instructions field explicitly specifies the tasks that the large language model needs to complete. The background information field provides multi-dimensional metadata of the load testing task. The additional content field loads the domain knowledge base and historical case library related to the load testing scenario. The output requirements field specifies the format and content of the output results of the large language model. The variables in the Prompt template are populated based on the multi-dimensional metadata of the load testing task. A prompting strategy of starting with fewer prompts and gradually increasing them is adopted to decompose complex problems into multiple sub-problems and populate the Prompt template context step by step.

6. The method for automatically generating intelligent stress test reports as described in claim 1, characterized in that, The report template's structured layout includes a test overview, performance bottleneck analysis, and optimization suggestions. It employs an adaptive layout strategy, loading different CSS files based on screen width. The trend chart supports line charts and bar charts, and allows configuration of chart parameters; the root cause analysis tree embedding includes building a hierarchical relationship diagram to show the propagation path of abnormal events, and supports node expansion / collapse interaction; The generated intelligent load testing report supports export in multiple formats. Specifically, it uses the Jinja2 template engine to render structured data into Markdown format, supporting code blocks and table syntax. It also uses the reportlab library to convert Markdown content into PDF files, automatically paginating and adding headers / footers, and supports custom page settings, pagination strategies, and header / footer styles.

7. An intelligent stress test report automated generation system, characterized in that, include: The fusion module is configured to perform multi-source fusion of the acquired stress test results data, historical case library and domain knowledge graph after spatiotemporal alignment; The Prompt template generation module is configured to design historical scenario tags and construct a structure diagram of load testing scenarios, load testing task features, and tags to identify load testing scenarios based on the multi-dimensional metadata of load testing tasks. Based on the multi-dimensional metadata of the load testing task, the historical case library and domain knowledge graph related to the corresponding load testing scenario, and the specified output format, a structured Prompt template is designed, and the variables in the Prompt template are populated according to the multi-dimensional metadata of the load testing task. The inference module is configured to convert the fused multimodal data into an input format supported by the large language model, concatenate it with the Prompt template to obtain the input text, and trigger the large language model to perform inference to obtain inference results including anomaly type, root cause components and remediation suggestions. The export module is configured to transform inference results into executable steps and annotate code examples, and generate intelligent load testing reports by combining report templates with embedded trend charts and root cause analysis trees.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images