Flow management method and device for software testing and medium
By capturing and classifying network traffic in the production environment in real time, and using the generative adversarial network to generate uncovered traffic, the problems of low coverage and high cost in traditional software testing methods are solved, and more efficient and accurate test results are achieved.
Patent Information
- Application Number
- CN202510359508.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-25
- Publication Date
- 2025-07-11
AI Technical Summary
Traditional software testing methods are difficult to cover all possible user scenarios, and the simulated traffic and real behavior are very different, resulting in inaccurate test results and high cost, making it difficult to quickly adapt to software iteration.
Capture network traffic in the production environment in real time, extract multi-dimensional features and classify them, use the generative adversarial network to generate uncovered traffic, ensure that the traffic in the test environment is consistent with the production environment, and generate uncovered network traffic by generating adversarial network models to meet the test needs.
It improves the accuracy and reliability of test results, ensures test coverage, avoids missed testing business scenarios, and improves the fit between the test environment and production scenarios.
Smart Images
Figure CN120295915A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer software technology, and particularly relates to a traffic management method, device, and medium for software testing. Background Art
[0002] Software testing is a key link in the software development life cycle. By running tests, it is checked whether the software system meets the specified requirements. During the testing process, defects in the software and the differences between the expected results and the actual results are discovered as early as possible. The software system is further improved based on the existing defects and differences to ensure that the functions, performance, security, and stability of the software meet the expectations.
[0003] Traditional testing methods include types such as manual testing, unit testing, and integration testing. As the complexity and scale of software systems continue to increase, traditional testing methods often rely on the experience of testers and achieve it by writing and maintaining a large number of test scripts and generating simulated traffic based on the test scripts. However, the written test scripts are difficult to cover all possible user scenarios. Especially in complex interaction scenarios, it is difficult to simulate the real behavior of users with the simulated traffic, and it cannot accurately reflect the actual load and performance bottlenecks in the production environment. Moreover, the economic cost and time for development and maintenance are relatively high, and it is difficult to quickly adapt to frequent software iterations. Summary of the Invention
[0004] To solve the above problems, this application proposes an optimization method for software testing based on traffic analysis, including:
[0005] Real-time capture network traffic in the production environment and extract multi-dimensional features of the network traffic;
[0006] Classify the network traffic according to the multi-dimensional features, obtain the classified traffic corresponding to the test requirements, and determine whether the classified traffic covers the test requirements;
[0007] If not, generate the uncovered network traffic corresponding to the test requirements through a generative adversarial network model;
[0008] Map the classified traffic and the uncovered network traffic to the test environment corresponding to the test requirements and execute the test;
[0009] Obtain the test performance metrics in the test environment after the test is completed, and determine whether the test performance metrics meet the test standards corresponding to the test requirements.
[0010] On the other hand, this application also proposes an optimization device for software testing based on traffic analysis, including:
[0011] At least one processor; and,
[0012] A memory communicatively connected to the at least one processor; wherein,
[0013] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute a software test optimization method based on traffic analysis as described in the above example.
[0014] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are configured as: a software test optimization method based on traffic analysis as described in the above example.
[0015] The software test optimization method based on traffic analysis proposed by the present application can bring the following beneficial effects:
[0016] By capturing and classifying real traffic in the production environment in real time, the real load and traffic patterns in the production environment can be reproduced more accurately, avoiding the deviations that may be generated by traditional simulation tools. The traffic data used in the test is more consistent with the production environment, so the accuracy and reliability of the test results are higher.
[0017] By using a generative adversarial network to make up for traffic types or scenarios not covered by traditional testing methods, it effectively avoids missing some business scenarios or abnormal situations, ensures the comprehensiveness of the test, makes the simulated traffic in the test environment more conform to the actual production scenario, and thus improves the test coverage rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:
[0019] Figure 1 It is a schematic flowchart of a software test optimization method based on traffic analysis in an embodiment of the present application;
[0020] Figure 2 It is a schematic diagram of machine interaction for traffic capture and test execution in an embodiment of the present application;
[0021] Figure 3 It is a schematic diagram of a software test optimization device based on traffic analysis in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0022] To make the objectives, technical solutions, and advantages of this application more clear, the following will clearly and completely describe the technical solutions of this application in combination with specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.
[0023] The following will, in combination with the drawings, detail the technical solutions provided by each embodiment of this application.
[0024] As Figure 1 shown, an embodiment of this application provides a software test optimization method based on traffic analysis, including:
[0025] S101: Capture network traffic in real time in the production environment and extract multi-dimensional features of the network traffic.
[0026] Specifically, a traffic capture tool is pre-deployed in the production environment. Commonly used traffic capture tools include tcpcopy, goreplay, tcpreplay, etc. Network request data is captured in real time through the traffic capture tool to obtain network traffic, such as HTTP requests, TCP connections, database queries, etc. The network request data includes contents such as original data packets, request headers, request lines, and request information. As Figure 2 shown, a tcpcopy process is deployed on the online machine to capture the traffic of the online machine, and the traffic is forwarded to the test machine by modifying the destination address and source address of the network packet.
[0027] Furthermore, multi-dimensional features are extracted from the original network data. The multi-dimensional features include protocol dimension features, time dimension features, and business semantics dimension features. Among them, the protocol dimension features include protocol type, data packet length, interaction frequency, etc.; the time dimension features include traffic peaks, traffic valleys, request latency, and response latency, etc.; the business semantics dimension features include business logic complexity and user behavior patterns, etc.
[0028] Specifically, each network data in the network traffic is parsed. According to the original data packet corresponding to the network data, the corresponding protocol type and data packet length are determined, the interaction frequency between network data with different protocol types is extracted, the fluctuation curve of the network traffic within a preset time period is statistically analyzed to determine the traffic peaks and traffic valleys of the network traffic. According to the arrival time and response time of the original data packet, the request latency and response latency of the network data are calculated. The request information of the network data is analyzed to determine the corresponding business operation steps of the network data. Based on the business operation steps, the corresponding business logic complexity is determined. Based on the user interaction records, the user types in the network traffic are determined, and the user behavior patterns corresponding to each user type are extracted.
[0029] S102: Classify the network traffic according to the multi-dimensional features, obtain the classified traffic corresponding to the test requirements, and determine whether the classified traffic covers the test requirements.
[0030] Specifically, preprocess the captured network traffic, including removing irrelevant traffic (such as broadcast traffic, redundant data, etc.), formatting traffic data (such as sorting by timestamp, removing invalid data packets, etc.). And perform standardization processing on the feature values to ensure that features in different dimensions can be compared on the same scale and prevent features in a certain dimension from dominating the classification process. According to the extracted features, convert each traffic instance into a feature vector, and the dimensions of each feature vector will include: protocol type, packet length, interaction frequency, request delay and response delay, traffic peak and trough periods, business logic complexity, user behavior patterns.
[0031] It should be noted that the test types include performance test types, security test types, and compatibility test types. Before starting the classification, define the requirements for each test type. Specifically, performance test types focus on performance indicators such as traffic load, throughput, and latency, and mainly analyze features such as the protocol type of the traffic, interaction frequency, request / response delay, etc. Security test types focus on potential security hazards in network traffic, such as non-compliant protocol usage, potential attack patterns (such as denial-of-service attacks, SQL injection, etc.), abnormal request behaviors, etc., and mainly focus on protocol layer features, business logic complexity, traffic abnormal behaviors, etc. Compatibility test types focus on compatibility issues in different protocol, operating system, browser, etc. environments, and mainly analyze differences such as protocol version differences, packet lengths, and differences between requests and responses.
[0032] Furthermore, according to the test criteria corresponding to the test type, determine the decision thresholds corresponding to the multi-dimensional features, such as: the request frequency should exceed the first threshold (such as ≥1000 requests per second); the response delay should be lower than the second threshold (such as the response delay requirement in performance testing <200ms); the packet size is defined as high-load traffic when it is greater than the third threshold according to the scenario setting (such as a single packet size exceeding 1MB), etc. According to the decision thresholds, classify the network data corresponding to the feature vectors into the corresponding test types through the classification model.
[0033] It should be noted that for the test requirements corresponding to different test types, a set of quantitative criteria can be defined. In performance testing, set the request frequency, response time, and packet size within specific numerical ranges; in security testing, set specific standard values for the request frequency, attack traffic flag, and abnormal mode; in compatibility testing, determine the traffic difference ranges for different protocols and versions.
[0034] More specifically, a classification model is trained through a machine learning algorithm, and the traffic is trained using a pre-annotated traffic dataset, and finally a classification model is obtained.
[0035] According to the test types included in the test requirements, determine the required feature coverage range of the test requirements. Based on the included test types, collect the required network data to obtain the classified traffic corresponding to the test requirements, calculate the feature coverage range of the classified traffic, and determine whether the difference between the feature coverage range and the required feature coverage range is lower than a preset threshold.
[0036] It should be noted that there are cases of full coverage, partial coverage, and non-coverage. Full coverage means that the features of the classified traffic fully meet the standards of the test requirements. For example, the request frequency of the performance test traffic is greater than 1000 and the response time is less than 200 ms. Partial coverage means that some of the features of the classified traffic meet the requirements. For example, the security test traffic includes DDoS attack traffic, but the frequency is lower than the test requirement standard. Non-coverage means that the features of the classified traffic do not match the test requirements. For example, the protocol version of the compatibility test traffic does not meet the test requirements (such as using HTTP / 1.0 while the requirement is HTTP / 2).
[0037] S103: If not, then generate the uncovered network traffic corresponding to the test requirements through a generative adversarial network model.
[0038] Specifically, when the classified traffic is not sufficient to cover the test requirements, determine the missing features of the test requirements according to the covered features of the classified traffic and the required covered features.
[0039] Input random noise into the generative adversarial network model. Through the generator in the generative adversarial network model, based on the missing features, generate simulated uncovered network traffic. Through the discriminator in the generative adversarial network model, judge whether the generated traffic is similar to the real traffic and evaluate whether it meets the features in the test requirements. Ensure that the generated traffic is significantly different from the existing traffic in terms of features, meets the test requirements, and ensures that the generated traffic meets the required test requirement boundaries.
[0040] It should be noted that in order to ensure that the generated traffic meets specific boundary scenarios, the boundary conditions of the generated traffic need to be clarified. For example: for performance testing, generate request frequencies and response times that reach a certain limit (such as RPS≥10000, response time≤100 ms). For security testing, generate DDoS attack traffic, simulate request frequencies greater than a certain threshold, and with a long duration. For compatibility testing, generate traffic with different protocol versions to ensure protocol version incompatibility or response time differences.
[0041] When the generated simulated uncovered network traffic does not meet the test requirements, the generator continues to generate and judge; when the generated simulated uncovered network traffic meets the test requirements, the simulated uncovered network traffic is output.
[0042] S104: Map the classified traffic and the uncovered network traffic to the test environment corresponding to the test requirements, and execute the test.
[0043] Specifically, the classified traffic and the uncovered network traffic are copied to the test environment in real time, and the test is executed to simulate the behavior of real users.
[0044] It should be noted that it supports playing back traffic proportionally to simulate different load scenarios, and the interfaces of the service must be idempotent. Multiple calls will not produce side effects. Otherwise, traffic playback may lead to data inconsistency or system anomalies.
[0045] S105: Obtain the test performance metrics in the test environment after the test is completed, and judge whether the test performance metrics meet the test standards corresponding to the test requirements.
[0046] Specifically, obtain the corresponding numerical values of the test performance metrics in the test environment after the test is completed, including response time, throughput, request success rate, latency, error rate, resource utilization rate, etc. Obtain the standard values corresponding to the preset performance metrics corresponding to the test requirements, and compare the collected corresponding numerical values of the test performance metrics with the standard values to evaluate whether they meet the expected performance standards.
[0047] If not, then through the dynamic time warping algorithm, compare the performance metric change curve in the production environment with the test performance metric change curve in the test environment, detect the deviation threshold, generate an optimization strategy based on the deviation threshold, and generate a test report based on the optimization strategy.
[0048] Record the data in the entire test process in real time, and generate a detailed result summary after the test is completed, including test data records and performance result records. Among them, the test data records the detailed data in each test, including the response time, success rate, error information, etc. of each request, the type and mode of the generated test traffic (such as attack traffic, normal traffic, boundary scenario traffic, etc.), and the relationship with the actual system response. The performance results are summarized and analyzed by statistically collecting the data during the test process. For example, calculate the average response time, throughput, success rate, etc. of each type of traffic, and use charts and visualization tools to display the test results to help analyze and evaluate whether the test objectives are achieved. For example, response time curve, error rate trend chart, traffic and system load relationship chart, etc.
[0049] By capturing and classifying real traffic in the production environment in real time, it is possible to reproduce the real load and traffic patterns in the production environment more precisely, avoiding the deviations that may occur in traditional simulation tools. The traffic data used in the test is more consistent with the production environment, so the accuracy and reliability of the test results are higher.
[0050] By using a generative adversarial network to make up for the traffic types or scenarios not covered by traditional testing methods, it effectively avoids missing some business scenarios or abnormal situations, ensures the comprehensiveness of the test, makes the traffic simulated in the test environment more conform to the actual production scenario, and thus improves the test coverage rate.
[0051] As Figure 3 shown, the embodiment of the present application also proposes a software test optimization device based on traffic analysis, including:
[0052] At least one processor; and,
[0053] A memory communicatively connected to the at least one processor; wherein,
[0054] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a software test optimization method based on traffic analysis as described in any one of the above embodiments.
[0055] The embodiment of the present application also provides a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are set as: a software test optimization method based on traffic analysis as described in any one of the above embodiments.
[0056] Each embodiment in the present application is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.
[0057] The device and medium provided by the embodiment of the present application correspond one-to-one with the method. Therefore, the device and medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and medium will not be elaborated here.
[0058] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.
[0059] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.
[0060] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.
[0061] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.
[0062] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.
[0063] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.
[0064] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media such as modulated data signals and carrier waves.
[0065] It should also be noted that the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "including a..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0066] The above description is only for the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.
Claims
1. A software test optimization method based on traffic analysis, characterized in that Including: Real-time capture of network traffic in a production environment and extraction of multi-dimensional features of the network traffic; Classify the network traffic according to the multi-dimensional features, obtain the classified traffic corresponding to the test requirements, and determine whether the classified traffic covers the test requirements; If not, generate the uncovered network traffic corresponding to the test requirements through a generative adversarial network model; Map the classified traffic and the uncovered network traffic to the test environment corresponding to the test requirements and execute the test; Obtain the test performance metrics in the test environment after the test is completed, and determine whether the test performance metrics meet the test standards corresponding to the test requirements.
2. The software testing optimization method based on traffic analysis according to claim 1, characterized in that, The multi-dimensional features include protocol dimension features, time dimension features, and business semantics dimension features; The protocol dimension features include protocol type, packet length, and interaction frequency; The time dimension features include traffic peaks, traffic valleys, request latency, and response latency; The business semantics dimension features include business logic complexity and user behavior patterns.
3. The software testing optimization method based on traffic analysis according to claim 2, characterized in that, The extraction of the multi-dimensional features of the network traffic specifically includes: Parse each network data in the network traffic, and determine the corresponding protocol type and packet length according to the original data packet corresponding to the network data; Extract the interaction frequency between network data with different protocol types; Statistically analyze the fluctuation curve of the network traffic within a preset time period to determine the traffic peaks and traffic valleys of the network traffic; Calculate the request latency and response latency of the network data according to the arrival time and response time of the original data packet; Analyze the request information of the network data to determine the corresponding business operation steps of the network data, and determine the corresponding business logic complexity based on the business operation steps; Based on user interaction records, determine the user types in the network traffic, and extract the user behavior patterns corresponding to each user type.
4. The software test optimization method based on traffic analysis according to claim 2, characterized in that, The classification of the network traffic according to the multi-dimensional features specifically includes: Standardize the multi-dimensional features; Based on the standardized multi-dimensional features, convert each network data in the network traffic into a feature vector; Determine the decision threshold corresponding to the multi-dimensional features according to the test standards corresponding to different test types; According to the decision threshold, classify the network data corresponding to the feature vector into the corresponding test type through a classification model.
5. The software test optimization method based on traffic analysis according to claim 4, wherein The obtaining of the classified traffic corresponding to the test requirements and the determination of whether the classified traffic covers the test requirements specifically includes: Determine the required feature coverage range of the test requirements according to the test types included in the test requirements; Based on the test types included, collect the required network data to obtain the classified traffic corresponding to the test requirements; Calculate the feature coverage range of the classified traffic, and determine whether the difference between the feature coverage range and the required feature coverage range is lower than a preset threshold.
6. The software test optimization method based on traffic analysis according to claim 5, characterized in that, The generation of the uncovered network traffic corresponding to the test requirements through a generative adversarial network model according to the multi-dimensional features specifically includes: Determine the missing features of the test requirements according to the coverage features and required coverage features of the classified traffic; Based on the missing features, a generator in the generative adversarial network model generates simulated uncovered network traffic; Through a discriminator in the generative adversarial network model, analyze the authenticity of the simulated uncovered network traffic and determine whether it conforms to the missing features; If not, continue to generate and judge through the generator; If so, output the simulated uncovered network traffic.
7. A software test optimization method based on traffic analysis according to claim 1, characterized in that The judgment of whether the test performance index conforms to the test standard corresponding to the test requirement specifically includes: Obtain the standard value corresponding to the preset performance index corresponding to the test requirement; Compare the corresponding value of the test performance index with the standard value to determine whether it conforms to the test standard corresponding to the test requirement.
8. A software test optimization method based on traffic analysis according to claim 7, characterized in that After judging whether it conforms to the test standard corresponding to the test requirement, the method further includes: If not, use the dynamic time warping algorithm to compare the performance index change curve in the production environment with the test performance index change curve in the test environment to detect the deviation threshold; Generate an optimization strategy according to the deviation threshold and generate a test report based on the optimization strategy.
9. A software test optimization device based on traffic analysis, characterized in that Includes: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a software test optimization method based on traffic analysis as described in any one of claims 1 to 8.
10. A non - volatile computer storage medium stores computer - executable instructions, characterized in that, The computer-executable instructions are set to a software test optimization method based on traffic analysis as described in any one of claims 1 to 8.