Network automation test method for novel reliable transport protocol of multi-modal network

Abstract

The invention discloses an automatic network testing method for a novel reliable transmission protocol of a multi-modal network, which relates to the technical field of network testing and comprises the following steps of: S1, building a testing environment: building an automatic testing platform, configuring hardware, software and a network environment required by testing by using the platform, and setting up the testing environment; the platform is logically divided into an adaptation layer, a function layer, an interface layer and an application layer. S2, designing a test case; s3, executing a test process; and S4, analyzing a test result. According to the automatic network testing method for the novel reliable transmission protocol of the multi-modal network, a set of complete testing system is established and matched with an established automatic testing platform, so that full-process automation from testing environment establishment, testing case design, testing process execution to testing result analysis can be executed; the test consistency and repeatability are ensured, manual intervention is reduced, manual operation errors are effectively reduced, and meanwhile the test efficiency is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network testing, in particular to a network automation testing method for a new reliable transmission protocol of a multi-modal network. BACKGROUND

[0002] With the rapid development of new generation information technologies such as cloud computing, big data and artificial intelligence, as the core infrastructure of digital economy, data centers continue to expand in size and increase in complexity, which puts higher requirements on the performance bearing, operation reliability and security protection capability of the network. The current data center network architecture presents a diversified evolution trend, from the traditional three-layer architecture, leaf-spine architecture to emerging software-defined network (SDN) and other technical solutions, all of which aim to achieve high throughput transmission, low delay response and high reliability guarantee as the core target, providing solid network support for upper-layer business.

[0003] However, the complexity of the architecture also brings double challenges in management and operation and testing and verification. The traditional test mode relying on manual operation has problems such as low efficiency and limited coverage, which is difficult to adapt to the needs of rapid iteration and continuous deployment of business. In contrast, automated testing, through standardized process design and intelligent tool support, not only can significantly improve testing efficiency and reduce human error, but also can realize continuous monitoring and real-time positioning of network throughout the life cycle, providing strong support for rapid troubleshooting and repair of network faults. Under this background, data center network automation testing has become a core means to guarantee network service quality and improve operational efficiency, and plays an irreplaceable important role in maintaining the stable operation of data centers.

[0004] However, in actual application process, with the continuous evolution of data center network architecture, the network composition, operation protocol and the like are increasingly complex and diverse. In terms of network composition, network heterogeneity has become an important means to ensure network reliability, and there are often devices from different manufacturers and different chips & architectures from the same manufacturer in the network, resulting in differences in command lines, especially in servers, storage and the like, and there are also differences in device types, which usually takes time and effort to test, configure and deploy.

[0005] On the other hand, in order to meet the needs of changing applications, network protocols and technologies in the network are becoming more and more rich, and when testing different scenarios, different configurations need to be switched for different network protocols, especially in large-scale networking testing, the whole network configuration needs to be repeatedly deleted and added during manual switching process, which consumes a lot of time for deployment outside testing. SUMMARY

[0006] The present application aims to provide a network automation testing method for a new reliable transmission protocol of a multi-modal network to solve the problems raised in the background.

[0007] To achieve the above object, the present application provides the following technical solutions: a network automation testing method for a new reliable transmission protocol of a multi-modal network, comprising the following steps:

[0008] S1, building of a test environment: building an automated testing platform, using the platform to configure the required hardware, software and network environment for the subsequent testing activities, the platform is logically divided into an adaptation layer, a function layer, an interface layer and an application layer;

[0009] S2, design of test cases: according to the test requirements, design test cases covering network device single machine testing, network state inspection testing, physical networking testing and stress testing and other scenarios, and for each test scenario, clearly define the test target, test steps and expected results;

[0010] S3, execution of the test process: execute the test cases on the automated testing platform, call the test function modules through the API interface, monitor the test process, record the test data and state in real time, such as network performance indicators, device state information, error logs, etc., and use the API interface provided by the interface layer to store the test data in real time into the database, for subsequent data analysis and report generation;

[0011] For different test scenarios and targets, use the API interface provided by the interface layer of the automated testing platform to dynamically modify the test parameters (such as network bandwidth, delay, packet loss rate, etc. to simulate different network environments) and configurations (device parameters in the test environment, including switches, servers, network cards, etc.), and write dynamic change scripts to change the network bandwidth, introduce random packet loss or delay fluctuations, simulate the dynamic changes of the network environment in reality;

[0012] For the same test case or test scenario, perform multiple iterations of testing, clearly define the target, test case, expected result and evaluation standard for each iteration, use the loop execution function of the automated testing platform to automatically perform multiple iterations of testing, adjust different parameters or configurations for each test to comprehensively evaluate the performance of the network under different conditions, and according to the results of the iteration test, optimize the test case or test configuration to improve the accuracy and effectiveness of the test;

[0013] S4, analysis of test results: collect the data and logs generated during the test process, these data include but are not limited to the execution of test items, the input and output of test data, the change of test state and any abnormal or error information, organize and analyze the collected data and logs, and according to the test target and expected result, calculate the pass rate and failure rate of the test, identify the abnormal patterns or trends in the test (use data analysis tools or algorithms to mine the test data to find potential problems or performance bottlenecks).

[0014] Further, the automatic test platform adopts a logically layered framework, from bottom to top, which is an adaptation layer, a function layer, an interface layer and an application layer. Through layered design, the test platform is structured, modularized and scalable, as follows:

[0015] The adaptation layer: shields the differences of the underlying hardware and provides a unified interface for the function layer, including shielding the driving differences of switches, servers, network cards, instruments and other devices, to ensure that the upper layer functions can be independent of specific hardware implementation;

[0016] The function layer: encapsulates the function modules relied on by the upper layer and provides various communication or storage libraries, including providing file libraries, network libraries, database libraries and other library functions, for the interface layer and the application layer to call;

[0017] The interface layer: provides encapsulated API interfaces to the application layer, encapsulates the interfaces required by the application layer, and flexibly extends new interfaces, including parameter configuration, test item configuration, test data storage, test status update and test log recording API;

[0018] The application layer: the highest layer of the automatic test platform, responsible for providing various application modules to users, including but not limited to test item management, test progress update, test report push and log management, and these application modules can directly interact with users to meet the test needs of users.

[0019] Further, the construction process of the automatic test platform: use a programming language (such as Python, Java, etc.) to develop the codes of each layer of the test platform, integrate the codes of each layer, use object-oriented programming techniques such as interfaces and abstract classes to realize decoupling and efficient communication between layers, and perform unit testing, integration testing and system testing to verify whether each function of the test platform meets the expectations, fix problems found during testing, and optimize platform performance.

[0020] Further, in step S1, the test environment specifically includes:

[0021] Hardware environment: configure the required network devices (such as switches, routers, etc., where the switch is used to build the network topology, realize the data exchange and forwarding between different network devices, and is an indispensable basic device in network testing; the router is responsible for data forwarding between different networks, and is the key device for network interconnection), servers (as the target device or test control node of the test, providing computing and storage resources for running test programs, storing test data, etc.), network cards (connecting servers and network devices to realize data transmission and reception functions, and are the bridge of communication between servers and networks), test instruments (such as network analyzers, protocol analyzers, etc., used for monitoring and analyzing network performance to help testers understand the actual running status of the network), etc., and ensure the correct physical connection between them;

[0022] Software environment: install and configure operating systems (such as Linux, Windows, etc., to provide a basic running environment for testing, support the running of test programs and the processing of test data), test tools, network management software (used to configure and manage network devices, monitor network status, such as network topology display, device state monitoring, traffic analysis, etc.), etc., where the test tools include automated test framework, test scripts, test cases;

[0023] Network environment: set up network topology (network structure designed according to testing requirements, including connection relationship between devices, IP address allocation, etc., used to simulate real network environment), configure IP address, port binding, etc., to ensure network connectivity.

[0024] Further, the test tools include:

[0025] Automated test framework: such as Selenium, Appium, etc., used to write and execute test scripts to realize automation of testing;

[0026] Test script: executable script written according to testing requirements, used to simulate user operations, verify network functions, etc.;

[0027] Test case: defines test target, test steps, expected result, etc., and is the basis of testing work.

[0028] Further, in step S2, a test management tool (such as TestRail, Jira, etc.) is used to design and manage test cases, and a test case template is used to quickly write and modify test cases, which includes definition template and reuse template, as follows:

[0029] Definition template: according to testing requirements, define test case template, including test case ID, test title, test target, preconditions, test steps, expected result, actual result, test status, etc.

[0030] Reuse templates: When creating new test cases, reuse templates to quickly fill in necessary information and reduce repetitive work;

[0031] The test case design follows the principles of comprehensiveness, pertinence, repeatability, and maintainability;

[0032] Comprehensiveness: Ensure that test cases cover all key functions and scenarios, including normal processes, abnormal processes, and boundary conditions.

[0033] Pertinence: For each test scenario, design specific test cases to clearly define test objectives and expected results.

[0034] Repeatability: Test cases should describe test steps and expected results in detail to ensure consistent results when executed by different testers or at different times.

[0035] Maintainability: Test cases should be easy to understand and modify to adapt to changes in requirements or testing environments.

[0036] Further, in step S2, the test objectives, test steps, and expected results of each test scenario are as follows:

[0037] 1) Network device standalone testing

[0038] I. Test objectives: Verify whether the basic functions of a single network device (such as switches, routers, servers, etc.) are normal, and check whether the configuration of the device is correct and the performance meets the requirements;

[0039] II. Test steps:

[0040] Preparation phase: Connect the device under test to the automated testing platform and configure the basic network environment;

[0041] Configuration testing: Issue configuration commands through the automated testing platform to verify whether the device can correctly receive and execute the configuration;

[0042] Function testing: Perform a series of function test cases, such as port state checking, VLAN configuration, and routing protocol verification;

[0043] Performance testing: Send simulated traffic to test the throughput, delay, and other performance indicators of the device;

[0044] Log collection: Collect log information of the device during testing for subsequent analysis;

[0045] III. Expected results:

[0046] A. The device can correctly receive and execute configuration commands;

[0047] B. All functional test cases pass, device functions normally;

[0048] C. Performance indicators meet design requirements;

[0049] D. No abnormal error information in the log;

[0050] 2) Network state inspection test

[0051] I. Test goal: Real-time monitoring of network status, timely discovery and reporting of network anomalies, and verification of the accuracy and real-time performance of the network monitoring system;

[0052] II. Test steps:

[0053] Configure monitoring: Configure network monitoring tasks on the automated testing platform, specify monitoring objects and monitoring indicators;

[0054] Simulate anomalies: Trigger the alarm mechanism of the monitoring system by simulating network failures (such as link interruption, device downtime, etc.);

[0055] Data collection: Collect alarm information and data records of the monitoring system when anomalies occur;

[0056] Result verification: Verify whether the monitoring system can accurately and timely report network anomalies, and check the accuracy of the alarm information;

[0057] III. Expected results:

[0058] A. The monitoring system can accurately and timely report network anomalies;

[0059] B. The alarm information contains accurate anomaly types, occurrence times, and impact ranges;

[0060] C. The log records detailed monitoring data and alarm information;

[0061] 3) Physical networking test

[0062] I. Test goal: Verify the feasibility and performance of the physical networking scheme, and check whether the network topology structure is correct and the connection between devices is stable;

[0063] II. Test steps:

[0064] Networking setup: Set up the network environment according to the predetermined physical networking scheme, including device connection, cable layout, etc.;

[0065] Configuration delivery: Deliver network configuration through the automated testing platform to ensure that all device configurations are correct;

[0066] Connectivity test: Execute connectivity test cases to verify whether the communication between devices is normal;

[0067] Performance testing: Test the throughput, latency, and other performance indicators of the entire network by sending simulated traffic.

[0068] Fault simulation: Simulate network faults (such as link faults, device faults, etc.) to test the fault tolerance and recovery capabilities of the network.

[0069] III. Expected results:

[0070] A. All devices communicate normally, and the connectivity test passes.

[0071] B. Network performance indicators meet design requirements.

[0072] C. When simulated faults occur, the network can quickly recover and has good fault tolerance.

[0073] 4) Stress testing

[0074] I. Test goal: Evaluate the stability and performance of the network under high load, and find potential bottlenecks and problems under high load.

[0075] II. Test steps:

[0076] Environment preparation: Set up the stress testing environment, configure the required hardware, software, and network environment for testing.

[0077] Load generation: Generate high-load traffic through an automated testing platform to simulate network load under actual business scenarios.

[0078] Performance monitoring: Monitor network performance indicators such as throughput, latency, and packet loss rate in real time under stress testing.

[0079] Fault injection: Inject faults (such as link interruptions, device downtime, etc.) during stress testing to test the stability and recovery capabilities of the network.

[0080] Result analysis: Collect test data and analyze network performance and fault recovery under high load.

[0081] III. Expected results:

[0082] A. The network remains stable under high load, and performance indicators do not decrease significantly.

[0083] B. After fault injection, the network can quickly recover and has good stability.

[0084] C. Test data records detailed performance indicators and fault recovery conditions, providing a basis for subsequent optimization.

[0085] Further, the specific operations of simulating dynamic changes in the network environment in the real world are:

[0086] Bandwidth adjustment: The script adjusts the bandwidth limit of the network interface according to a preset time interval or random time point to simulate network congestion or bandwidth fluctuations. Specifically, it uses the tc (Traffic Control) tool under Linux to set the bandwidth limit of the network interface, and writes a script to periodically call the tc command to adjust the bandwidth.

[0087] Random packet loss: During data transmission, the script randomly drops a certain percentage of data packets to simulate network instability or transmission errors. The tc tool can also be used in conjunction with the netem (Network Emulator) module to simulate packet loss.

[0088] Latency fluctuations: The script introduces random network latency to simulate latency changes in network transmission, including sudden increases or decreases in latency. Specifically, the netem module is used to simulate latency fluctuations.

[0089] Furthermore, in step S4, for failed cases (i.e. problems found during testing), the problems are located and analyzed in combination with test data, log information, etc., and based on the results of the problem location and analysis, specific improvement suggestions are proposed. The improvement suggestions include, but are not limited to, network architecture optimization, device configuration adjustment, test case improvement or automated testing platform upgrade.

[0090] Problem localization: By comparing test data and logs, the causes of failure are analyzed to determine the specific steps and influencing factors in which the problem occurred;

[0091] Problem Analysis: Combining factors such as network architecture, device configuration, and test case design, a comprehensive analysis of the problem is conducted, and possible causes and hypotheses are proposed.

[0092] Furthermore, in step S4, the pass rate and failure rate of the test are calculated using the total number of test cases, the number of passed test cases, and the number of failed test cases, using the following formula:

[0093] Pass rate = (Number of test cases passed / Total number of test cases) × 100%;

[0094] Failure rate = (Number of failed test cases / Total number of test cases) × 100%.

[0095] Furthermore, in step S4, the data required for anomaly pattern and trend identification includes, but is not limited to, test execution time series data, classification of failed tests (functional modules, error types, etc.), and test environment information, and the analysis methods include:

[0096] Time trend analysis: Check if the failure rate increases / decreases over time;

[0097] Module analysis: identify high failure rate of specific function modules;

[0098] Error type analysis: count the frequency of each type of error;

[0099] Environment correlation: analyze the relationship between failure and environment configuration.

[0100] The present application provides a kind of network automation test method for multi-modal network new reliable transmission protocol, with the following beneficial effects:

[0101] 1, the present application establishes a complete test system, cooperates the automation test platform built, can execute from test environment construction, test case design, test process execution to the whole process automation of test result analysis, ensures the consistency and repeatability of test, reduces manual intervention, effectively reduces human operation error, while significantly improves test efficiency.

[0102] 2, the present application covers network equipment single machine test, network state inspection test, physical networking test and stress test and multiple scenes, coverage reaches more than 80%, can comprehensively evaluate the performance of network under different conditions, to solve the problem of manual switching configuration when testing different scenes in prior art, especially suitable for large-scale networking test, the whole configuration switching process saves time and effort, further improves test efficiency.

[0103] 3, the present application builds a logically layered automation test platform, from bottom to top into adaptation layer, function layer, interface layer and application layer, this structured design makes the overall function more clear, easy to extend and maintain, especially the adaptation layer shields the difference of bottom hardware by providing a unified interface to function layer, simplifies the test configuration process, while ensuring that the upper function can be independent of specific hardware implementation, improves the universality and flexibility of test platform.

[0104] 4, the present application dynamically modifies test parameters and configuration through API interface, and writes dynamic change script, changes network bandwidth, introduces random packet loss or delay fluctuation regularly, simulates the dynamic change of network environment in reality, improves the comprehensiveness and authenticity of test, solves the problem that manual switching configuration is needed when testing different scenes in prior art, and it is difficult to simulate the dynamic change of network environment in reality. BRIEF DESCRIPTION OF DRAWINGS

[0105] Fig. 1 It is the step flow chart of the network automation test method for multi-modal network new reliable transmission protocol of the present application;

[0106] Fig. 2 It is the automation test platform logical layering schematic diagram of the network automation test method for multi-modal network new reliable transmission protocol of the present application. DETAILED DESCRIPTION

[0107] The embodiments of the present application will be further described below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0108] As shown in the drawings, Figs. 1-2 A network automation test method for a new reliable transmission protocol of a multi-modal network includes the following steps:

[0109] S1, building a test environment:

[0110] An automated test platform is built, and the platform is configured with the required hardware, software, and network environment for testing, providing a foundation for subsequent testing activities. The platform is logically divided into an adaptation layer, a function layer, an interface layer, and an application layer.

[0111] In this embodiment, the automated test platform uses a logically layered framework, from bottom to top, the adaptation layer, the function layer, the interface layer, and the application layer. Through layered design, the test platform achieves structure, modularity, and scalability. Specifically:

[0112] Adaptation layer: shields the differences between the underlying hardware and provides a unified interface for the function layer, including shielding the driver differences of switches, servers, network cards, instruments, and other devices to ensure that the upper layer functions can be implemented without relying on specific hardware;

[0113] Function layer: encapsulates the function modules relied on by the upper layer and provides various communication or storage libraries, including providing file libraries, network libraries, database libraries, and other library functions for the interface layer and the application layer to call;

[0114] Interface layer: provides encapsulated API interfaces to the application layer, encapsulates the interfaces required by the application layer, and flexibly extends new interfaces, including parameter configuration, test item configuration, test data storage, test status update, and test log recording APIs;

[0115] Application layer: the highest layer of the automated test platform, responsible for providing various application modules to users, including but not limited to test item management, test progress update, test report push, and log management. These application modules can directly interact with users to meet their testing needs.

[0116] The construction process of the automated test platform: use a programming language (such as Python, Java, etc.) to develop the codes of each layer of the test platform, integrate the codes of each layer, use object-oriented programming techniques such as interfaces and abstract classes to achieve decoupling and efficient communication between layers, and perform unit testing, integration testing, and system testing to verify whether the functions of the test platform meet the expectations, fix problems found during testing, and optimize platform performance.

[0117] In this embodiment, the test environment specifically includes:

[0118] Hardware environment: configure the required network devices (such as switches, routers, etc., where the switch is used to build the network topology, realize the data exchange and forwarding between different network devices, and is an indispensable basic device in network testing; the router is responsible for data forwarding between different networks, and is the key device for network interconnection), servers (as the target device or test control node of the test, providing computing resources and storage resources, for running test programs, storing test data, etc.), network cards (connecting servers and network devices, realizing data receiving and sending functions, and being the bridge for communication between servers and networks), test instruments (such as network analyzers, protocol analyzers, etc., for monitoring and analyzing network performance, helping testers understand the actual running status of the network) and the like, and ensure the correct physical connection between them;

[0119] Software environment: install and configure operating systems (such as Linux, Windows, etc., to provide a basic running environment for testing, support the running of test programs and the processing of test data), test tools, network management software (for configuring and managing network devices, monitoring network status, such as network topology display, device state monitoring, traffic analysis, etc.) and the like, wherein the test tools include:

[0120] Automatic test framework: such as Selenium, Appium, etc., for writing and executing test scripts to realize the automation of testing;

[0121] Test script: an executable script written according to test requirements, for simulating user operations, verifying network functions, etc.;

[0122] Test case: defines test targets, test steps, expected results, etc., and is the basis for testing work;

[0123] Network environment: set up network topology (network structure designed according to test requirements, including connection relationship between devices, IP address allocation, etc., for simulating real network environment), configure IP addresses, port bindings, etc., to ensure network connectivity.

[0124] S2, design of test cases:

[0125] According to the test requirements, design test cases covering network device single machine testing, network state inspection testing, physical networking testing and stress testing and other scenarios, and for each test scenario, clearly define the test target, test steps and expected results.

[0126] In this step, test management tools such as TestRail, Jira, etc. are used to design and manage test cases, and test case templates are used to quickly write and modify test cases. Test case templates include definition templates and reusable templates, as follows:

[0127] Definition template: According to the test requirements, define the test case template, including test case ID, test title, test target, preconditions, test steps, expected results, actual results, test status, etc.

[0128] Reusable template: When creating a new test case, reuse the template to quickly fill in the necessary information and reduce repetitive work.

[0129] And the test case design follows the principles of comprehensiveness, pertinence, repeatability and maintainability, as follows:

[0130] Comprehensiveness: Ensure that test cases cover all key functions and scenarios, including normal processes, abnormal processes and boundary conditions. For network device single machine testing, basic functions, configuration verification, performance testing, etc. of the device should be covered; for network state inspection testing, accuracy and real-time verification of the monitoring system should be included.

[0131] Pertinence: For each test scenario, design specific test cases to clearly define test objectives and expected results. For example, in physical networking testing, test cases can be designed to verify the correctness of network topology structure and the stability of device-to-device connections.

[0132] Repeatability: Test cases should describe test steps and expected results in detail to ensure consistent results when executed by different testers or at different times.

[0133] Maintainability: Test cases should be easy to understand and modify to adapt to changes in requirements or testing environment. Clear naming and annotation, as well as modular design, can improve the maintainability of test cases.

[0134] In this embodiment, the test objectives, test steps and expected results of each test scenario are as follows:

[0135] 1) Network device single machine testing

[0136] I. Test objective: Verify whether the basic functions of a single network device (such as a switch, router, server, etc.) are normal, and check whether the configuration of the device is correct and the performance meets the requirements;

[0137] II. Test steps:

[0138] Preparation phase: Connect the device under test to the automated testing platform and configure the basic network environment;

[0139] Configuration Test: Issue configuration commands through the automated test platform to verify if the device can correctly receive and execute the configuration;

[0140] Function Test: Execute a series of function test cases, such as port state check, VLAN configuration, routing protocol verification, etc.

[0141] Performance Test: Send simulated traffic to test the device's throughput, delay, and other performance indicators.

[0142] Log Collection: Collect log information during the test process for subsequent analysis.

[0143] III. Expected Results:

[0144] A. The device can correctly receive and execute configuration commands.

[0145] B. All function test cases pass, and the device functions normally.

[0146] C. Performance indicators meet design requirements.

[0147] D. There are no abnormal error messages in the log.

[0148] 2) Network State Inspection Test

[0149] I. Test Goal: Real-time monitoring of network status, timely discovery and reporting of network anomalies, and verification of the accuracy and real-time nature of the network monitoring system.

[0150] II. Test Steps:

[0151] Configure Monitoring: Configure network monitoring tasks on the automated test platform, specifying monitoring objects and monitoring indicators.

[0152] Simulate Abnormalities: Simulate network failures (such as link interruption, device downtime, etc.) to trigger the alarm mechanism of the monitoring system.

[0153] Data Collection: Collect alarm information and data records of the monitoring system when anomalies occur.

[0154] Result Verification: Verify if the monitoring system can accurately and timely report network anomalies, and check the accuracy of the alarm information.

[0155] III. Expected Results:

[0156] A. The monitoring system can accurately and timely report network anomalies.

[0157] B. The alarm information contains accurate anomaly types, occurrence times, and impact ranges.

[0158] C. The log records detailed monitoring data and alarm information.

[0159] 3) Physical networking test

[0160] I. Test objectives: Verify the feasibility and performance of the physical networking solution, and check whether the network topology is correct and the connection between devices is stable;

[0161] II. Test steps:

[0162] Network setup: Set up the network environment according to the predetermined physical networking solution, including device connection, cable layout, etc.

[0163] Configuration delivery: Deliver network configuration through the automated testing platform to ensure that all device configurations are correct.

[0164] Connectivity test: Execute connectivity test cases to verify whether the communication between devices is normal.

[0165] Performance test: Test the throughput, delay, and other performance indicators of the entire network by sending simulated traffic.

[0166] Fault simulation: Simulate network faults (such as link faults, device faults, etc.) to test the fault tolerance and recovery capabilities of the network.

[0167] III. Expected results:

[0168] A. All device communication is normal, and the connectivity test passes.

[0169] B. Network performance indicators meet design requirements.

[0170] C. When simulated faults occur, the network can quickly recover and has good fault tolerance.

[0171] 4) Stress test

[0172] I. Test objectives: Evaluate the stability and performance of the network under high load, and find potential bottlenecks and problems under high load.

[0173] II. Test steps:

[0174] Environment preparation: Set up the stress test environment and configure the required hardware, software, and network environment.

[0175] Load generation: Generate high-load traffic through the automated testing platform to simulate network load under actual business scenarios.

[0176] Performance monitoring: Monitor network performance indicators such as throughput, delay, and packet loss rate in real time under stress testing.

[0177] Fault injection: Inject faults (such as link interruption, device downtime, etc.) during stress testing to test the stability and recovery capabilities of the network.

[0178] Result Analysis: Collect test data, analyze network performance and fault recovery under high load;

[0179] III. Expected Results:

[0180] A. The network remains stable under high load, with no significant performance degradation;

[0181] B. The network quickly recovers after fault injection, with good stability;

[0182] C. Detailed performance indicators and fault recovery information are recorded in the test data, providing a basis for subsequent optimization.

[0183] S3, Execution of the Test Process:

[0184] Execute test cases on the automated testing platform, call test function modules through API interfaces provided by the interface layer, which encapsulate various communication and repository functions in the function layer, allowing the application layer to easily call test functions without worrying about underlying implementation details, and monitor the test process, recording test data and status in real time, such as network performance indicators, device status information, error logs, etc., and use the API interfaces provided by the interface layer to store test data in real time into the database, facilitating subsequent data analysis and report generation.

[0185] For different test scenarios and targets, use the API interfaces provided by the interface layer of the automated testing platform to dynamically modify test parameters (such as network bandwidth, delay, packet loss rate, etc. to simulate different network environments) and configurations (device parameters in the test environment, including switches, servers, network cards, etc.), and write dynamic change scripts to change network bandwidth, introduce random packet loss or delay fluctuations, and simulate dynamic changes in real-world network environments. Specific operations for simulating dynamic changes in real-world network environments:

[0186] Bandwidth adjustment: The script adjusts the bandwidth limit of the network interface according to the preset time interval or random time point, simulating network congestion or bandwidth fluctuation. Specifically, use the tc (Traffic Control) tool under Linux to set the bandwidth limit of the network interface, and through scripting, periodically call the tc command to adjust the bandwidth; for example, you can use tc qdisc add dev eth0 root tbf rate 10mbit burst 10k latency 50ms to set the bandwidth limit to 10Mbps, and you can modify the rate parameter through the script to change the bandwidth.

[0187] Random packet loss: During data transmission, the script randomly drops a certain percentage of data packets to simulate network instability or transmission errors. The tc tool can be used in combination with the netem (Network Emulator) module to simulate packet loss. For example, tc qdisc add dev eth0 root netem loss 1% can set a 1% packet loss rate, and the script can modify the loss parameter at regular intervals to introduce different packet loss rates.

[0188] Delay fluctuation: The script introduces random network delays to simulate delay changes in network transmission, including sudden increases or decreases in delay. The netem module is used to simulate delay fluctuations. For example, tc qdisc add dev eth0 root netem delay 100ms 50ms can set the average delay to 100ms and the fluctuation range to ±50ms. The script can modify the delay and jitter (fluctuation range) parameters at regular intervals to simulate different delay conditions.

[0189] For the same test case or test scenario, perform multiple iteration tests, clearly define the target, test case, expected result and evaluation standard of each iteration, use the loop execution function of the automatic test platform to automatically execute multiple iteration tests, adjust different parameters or configurations during each test to comprehensively evaluate the performance of the network under different conditions, and optimize the test case or test configuration based on the results of the iteration test to improve the accuracy and effectiveness of the test.

[0190] S4, analysis of test results:

[0191] Collect data and logs generated during testing, including but not limited to test item execution, test data input and output, test state changes, and any abnormal or error information. Organize and analyze the collected data and logs, and calculate the pass rate and failure rate of the test based on the test target and expected result, identify abnormal patterns or trends in the test (use data analysis tools or algorithms to mine test data and find potential problems or performance bottlenecks).

[0192] For failed cases (i.e. problems found in testing), combine test data, log information, etc. to locate and analyze the problem, and based on the results of problem location and analysis, propose specific improvement suggestions, including but not limited to optimization of network architecture, adjustment of device configuration, improvement of test cases or upgrade of automatic test platform;

[0193] Problem positioning: Compare test data and logs to analyze the failure cause and determine the specific link and influencing factors of the problem;

[0194] Problem analysis: Based on network architecture, device configuration, test case design and other factors, the problem is analyzed comprehensively, and possible reasons and assumptions are proposed.

[0195] In this embodiment, the pass rate and failure rate of the test are calculated using the total number of test cases, the number of passed test cases, and the number of failed test cases, and the calculation formula is as follows:

[0196] Pass rate = (number of passed test cases / total number of test cases) x 100%;

[0197] Failure rate = (number of failed test cases / total number of test cases) x 100%.

[0198] The data required for abnormal pattern and trend identification includes but is not limited to test execution time series data, classification of failed tests (function modules, error types, etc.), test environment information, and the analysis method includes:

[0199] Time trend analysis: Check if the failure rate increases / decreases over time;

[0200] Module analysis: Identify high failure rates for specific function modules;

[0201] Error type analysis: Count the frequency of occurrence of each type of error;

[0202] Environment correlation: Analyze the relationship between failure and environment configuration.

[0203] The embodiments of the present application are given for the purpose of illustration and description, and are not exhaustive or limit the present application to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described to better illustrate the principles and practical applications of the present application, and to enable those of ordinary skill in the art to understand the present application so as to design various embodiments with various modifications suitable for specific purposes.

Claims

1. A network automation testing method for a new reliable transmission protocol for multi-modal networks, characterized in that, Comprise the following steps: S1, test environment building: build an automated testing platform, configure the required hardware, software and network environment for testing, the platform is logically divided into adaptation layer, function layer, interface layer and application layer; S2, test case design: according to the test requirements, design test cases covering network device single machine test, network state inspection test, physical networking test and stress test multiple scenarios, and for each test scenario, the test target, test steps and expected results are clear; S3, test process execution: execute test cases on the automated testing platform, call test function modules through API interface, monitor test process, and record test data and state in real time; For different test scenarios and targets, use API interface to dynamically modify test parameters and configurations, and write dynamic change scripts to change network bandwidth, introduce random packet loss or delay fluctuation, simulate the dynamic changes of network environment in reality; For the same test case or test scenario, multiple iteration tests are performed, different parameters or configurations are adjusted each time, to comprehensively evaluate the performance of the network under different conditions, and according to the results of iteration test, the test case or test configuration is optimized; S4, test result analysis: collect the data and logs generated during the test process, organize and analyze, and according to the test target and expected result, calculate the pass rate and failure rate of the test, identify the abnormal mode or trend in the test.

2. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, The automated testing platform adopts a logical layered framework, from bottom to top, it is adaptation layer, function layer, interface layer and application layer, specifically as follows: Adaptation layer: shield the difference of bottom hardware, provide unified interface for function layer, including shielding the driver difference of switch, server, network card and instrument equipment; Function layer: encapsulate the function modules dependent on the upper layer, and provide various communication or storage library, including providing file library, network library, database library functions, for interface layer and application layer to call; Interface layer: provide encapsulated API interface to application layer, encapsulate the interface required by application layer, and flexibly extend new interface, including parameter configuration, test item configuration, test data storage, test state update and test log recording API; Application layer: the highest layer of automated testing platform, responsible for providing various application modules to users, the application modules include but are not limited to test item management, test progress update, test report pushing and log management, and directly interact with users to meet the test requirements of users.

3. The network automation testing method for the new reliable transmission protocol of multi-modal network according to claim 2, characterized in that, The construction process of the automated testing platform: use programming language to develop test platform code of each layer, integrate the code of each layer, use object-oriented programming technology to realize decoupling and communication between layers, and perform unit testing, integration testing and system testing to verify whether the functions of the test platform meet the expectations, fix the problems found in the test process, and optimize the platform performance.

4. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, In step S1, the test environment specifically includes: Hardware environment: configure the required network equipment, server, network card, test instrument, and ensure the correct physical connection between them; Software environment: install and configure operating system, test tools, network management software, including automated test framework, test scripts, test cases; Network environment: set up network topology, configure IP address, port binding, and ensure network connectivity.

5. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, In step S2, test management tools are used to design and manage test cases, and test case templates are used to write and modify test cases, including definition templates and reuse templates, as follows: Definition template: define test case template according to test requirements, including test case ID, test title, test target, precondition, test step, expected result, actual result, test status field; Reuse template: when creating a new test case, reuse template to quickly fill in necessary information and reduce repetitive work; The test case design follows the principles of comprehensiveness, pertinence, repeatability and maintainability.

6. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, In step S2, the test target, test step and expected result of each test scenario are as follows: 1) Network device single machine test Ⅰ. Test target: verify whether the basic function of a single network device is normal, and check whether the configuration of the device is correct and the performance meets the requirements; Ⅱ. Test steps: Preparation stage: connect the device under test to the automated test platform and configure the basic network environment; Configuration test: issue configuration commands through the automated test platform to verify whether the device can correctly receive and execute the configuration; Function test: execute a series of function test cases; Performance test: test the throughput and delay of the device by sending simulated traffic; Log collection: collect log information of the device during the test process for subsequent analysis; Ⅲ. Expected results: A. The device can correctly receive and execute the configuration command; B. All function test cases pass, and the device function is normal; C. Performance indicators meet design requirements; D. There is no abnormal error information in the log; 2) Network state inspection test Ⅰ. Test target: monitor network status in real time, discover and report network anomalies in a timely manner, and verify the accuracy and real-time performance of the network monitoring system; Ⅱ. Test steps: Configure monitoring: configure network monitoring tasks on the automated test platform, specify monitoring objects and monitoring indicators; Simulate abnormality: trigger the alarm mechanism of the monitoring system by simulating network failure; Data collection: collect alarm information and data records of the monitoring system when abnormality occurs; Result verification: verify whether the monitoring system can accurately and timely report network abnormalities, and check the accuracy of the alarm information; Ⅲ. Expected results: A. The monitoring system can accurately and timely report network abnormalities; B. The alarm information contains accurate abnormal type, occurrence time and impact range; C. The log records detailed monitoring data and alarm information; 3) Physical networking test Ⅰ. Test target: verify the feasibility and performance of the physical networking scheme, and check whether the network topology structure is correct and the connection between devices is stable; Ⅱ. Test steps: Networking setup: build network environment according to the predetermined physical networking scheme, including device connection and cable layout; Configuration delivery: deliver network configuration through the automated test platform to ensure that all devices are configured correctly; Connectivity test: Perform connectivity test cases to verify the communication between devices is normal; Performance test: Test the throughput, latency and other performance indicators of the entire network by sending simulated traffic; Fault simulation: Simulate network faults to test the fault tolerance and recovery ability of the network; III. Expected results: A. All devices communicate normally, and the connectivity test passes; B. Network performance indicators meet design requirements; C. When simulated faults occur, the network can quickly recover, and the fault tolerance is good; 4) Stress test I. Test goal: Evaluate the stability and performance of the network under high load, and find possible bottlenecks and problems under high load; II. Test steps: Environment preparation: Build a stress test environment, configure the required hardware, software and network environment for testing; Load generation: Generate high-load traffic through an automated testing platform to simulate network load in actual business scenarios; Performance monitoring: Monitor network performance indicators in real time under stress testing; Fault injection: Inject faults during stress testing to test the stability and recovery ability of the network; Result analysis: Collect test data and analyze network performance under high load and fault recovery; III. Expected results: A. The network remains stable under high load, and performance indicators do not decrease significantly; B. After fault injection, the network can quickly recover, and the stability is good; C. Test data records detailed performance indicators and fault recovery conditions, providing a basis for subsequent optimization.

7. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, The specific operations of simulating dynamic changes in the network environment in the simulation reality are as follows: Bandwidth adjustment: The script adjusts the bandwidth limit of the network interface according to the preset time interval or random time point to simulate network congestion or bandwidth fluctuation. The tc tool under Linux is used to set the bandwidth limit of the network interface, and the tc command is called regularly through a script to adjust the bandwidth. Random packet loss: During data transmission, the script randomly discards a certain proportion of data packets to simulate network instability or transmission errors. The tc tool can also be used in combination with the netem module to simulate packet loss. Delay fluctuation: The script introduces random network delays to simulate delay changes in network transmission, including sudden increases or decreases in delay. The netem module is used to simulate delay fluctuations.

8. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, In step S4, for failed cases, combine test data and log information to locate and analyze problems, and propose improvement suggestions, including but not limited to optimization of network architecture, adjustment of device configuration, improvement of test cases or upgrade of automated testing platform; Problem positioning: Compare test data and logs to analyze failure causes and determine the specific steps and influencing factors of the problem; Problem analysis: Analyze the problem comprehensively by considering network architecture, device configuration, and test case design, and propose possible causes and assumptions.

9. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, In step S4, the pass rate and failure rate of the test are calculated using the total number of test cases, the number of passed test cases, and the number of failed test cases. The calculation formula is as follows: Pass rate = (number of passed test cases / total number of test cases) x 100%; Failure rate = (number of failed test cases / total number of test cases) x 100%.

10. The network automation test method for the new reliable transmission protocol of multi-modal network according to claim 1, characterized in that, The data required for the step S4 of abnormal pattern and trend identification includes but not limited to test execution time series data, classification of failed tests, test environment information, and the analysis method includes: Time trend analysis: check if the failure rate increases / decreases over time; Module analysis: identify high failure rate for specific functional modules; Error type analysis: count the frequency of occurrence of each type of error; Environment correlation: analyze the relationship between failure and environment configuration.

Images