Network-based test method and device, equipment and medium

Abstract

The embodiment of the invention discloses a network-based test method and device, equipment and a medium. The method comprises the following steps: acquiring routing information of network equipment in a data communication network, and acquiring task information of a controller in the data communication network; injecting the routing information and the task information into a simulation network corresponding to the data communication network to obtain a simulation result related to the simulation network, and calculating an expected result related to the simulation network based on the routing information, the task information and network environment information of the simulation network; and generating a test result for the controller in the simulation network based on the simulation result and the expected result. According to the technical scheme, automatic testing of the controller in the network is achieved, the testing efficiency and accuracy are improved, and the testing reliability based on the network is high.

Description

Technical Field

[0001] This application relates to the field of communication technology, and more specifically, to a network-based testing method, a network-based testing device, an electronic device, and a computer-readable medium. Background Technology

[0002] In related technologies, when it is necessary to test the controller in the network, relevant information is usually constructed manually for testing. Since manual testing requires a lot of manpower and resources and is prone to errors, it reduces the efficiency and accuracy of the test, and thus reduces the reliability of the test.

[0003] Therefore, improving the efficiency and accuracy of controller testing in networks to ensure the reliability of network-based testing is an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of this application provide a network-based testing method, apparatus, device, and medium, which realizes automated testing of controllers in the network, improves testing efficiency and accuracy, and ensures the reliability of network-based testing.

[0005] In a first aspect, embodiments of this application provide a network-based testing method, the method comprising: acquiring routing information of network devices in a data communication network and acquiring task information of a controller in the data communication network; injecting the routing information and the task information into a simulation network corresponding to the data communication network to obtain simulation results related to the simulation network, and calculating expected results related to the simulation network based on the routing information, the task information, and the network environment information of the simulation network; and generating test results for the controller in the simulation network based on the simulation results and the expected results.

[0006] Secondly, embodiments of this application provide a network-based testing device, the device comprising: an acquisition module configured to acquire routing information of network devices in a data communication network and task information of controllers in the data communication network; an injection and calculation module configured to inject the routing information and the task information into a simulation network corresponding to the data communication network to obtain simulation results related to the simulation network, and calculate expected results related to the simulation network based on the routing information, the task information, and the network environment information of the simulation network; and a generation module configured to generate test results for the controllers in the simulation network based on the simulation results and the expected results.

[0007] In one embodiment of this application, based on the foregoing scheme, the injection and calculation module is specifically configured as follows: configuring network devices in the simulation network based on the routing information to obtain network devices in the simulation network configured with the routing information; configuring controllers in the simulation network based on the task information to obtain controllers in the simulation network configured with the task information; and executing tasks matching the task information through the network devices in the simulation network configured with the routing information and the controllers configured with the task information to obtain simulation results related to the simulation network.

[0008] In one embodiment of this application, based on the aforementioned scheme, the network devices in the simulated network include multiple network devices; the injection and calculation module is further specifically configured to: obtain network device identification information; select a target network device from the multiple network devices that matches the network device identification information; configure the target network device based on the routing information to obtain a target network device in the simulated network configured with the routing information.

[0009] In one embodiment of this application, based on the foregoing scheme, the task information includes multiple tasks; the injection and calculation module is further specifically configured to: send each task to a network device in the simulation network that is configured with the routing information through a controller in the simulation network, thereby obtaining the device simulation result corresponding to each task; and combine the device simulation results corresponding to the multiple tasks to obtain the simulation result related to the simulation network.

[0010] In one embodiment of this application, based on the aforementioned scheme, there is an execution order among the multiple tasks; the injection and calculation module is further specifically configured to: traverse the multiple tasks through a controller configured with the task information in the simulation network; send the traversed tasks to a network device configured with the routing information in the simulation network for execution, obtain the device simulation result corresponding to the traversed task, and traverse the next adjacent task until all the multiple tasks have been traversed, and obtain the device simulation result corresponding to each of the multiple tasks.

[0011] In one embodiment of this application, based on the aforementioned scheme, there is no execution order among the multiple tasks, and the controller configured with the task information in the simulation network includes multiple control logics; the injection and calculation module is further specifically configured to: select a task corresponding to each control logic from the multiple tasks; run each control logic in parallel, so that each control logic sends the task corresponding to each control logic to a network device configured with the routing information in the simulation network for execution, thereby obtaining the device simulation result corresponding to each task.

[0012] In one embodiment of this application, based on the aforementioned scheme, the task information includes multiple tasks; the injection and calculation module is specifically configured to: select a target calculation logic from multiple calculation logics that matches the task type of each task; run the target calculation logic so that the target calculation logic performs calculations based on the routing information, the network environment information, and the task corresponding to the target calculation logic to obtain the expected device result corresponding to each task; and combine the expected device results corresponding to the multiple tasks to obtain the expected result related to the simulation network.

[0013] In one embodiment of this application, based on the aforementioned scheme, the multiple tasks have an execution order, and the target computation logic includes multiple tasks; the injection and computation module is further specifically configured to: sort the multiple target computation logics according to the execution order to obtain a computation logic sequence; traverse the multiple target computation logics in the computation logic sequence and run the traversed target computation logics, so that the traversed target computation logics perform computations based on the routing information, the network environment information, and the tasks corresponding to the traversed target computation logics, to obtain the expected device result corresponding to the task matching the traversed target computation logics, and update the network environment information based on the obtained expected device result, and traverse the next adjacent target computation logic until all the multiple target computation logics are traversed, to obtain the expected device result corresponding to each task.

[0014] In one embodiment of this application, based on the aforementioned scheme, there is no execution order among the multiple tasks, and the target calculation logic includes multiple tasks; the injection and calculation module is further specifically configured to: run each target calculation logic in parallel, so that each target calculation logic performs calculations based on the routing information, the network environment information, and the task corresponding to each target calculation logic, to obtain the expected device result corresponding to each task.

[0015] In one embodiment of this application, based on the foregoing solution, the device further includes an extraction module configured to: display an input area in a test interface; extract the plurality of computational logics from the input content in response to the input content in the input area; and display the plurality of computational logics in the test interface.

[0016] In one embodiment of this application, based on the foregoing scheme, the extraction module is further configured to: adjust the selected computational logic in response to a selection operation for the plurality of computational logics; wherein the adjustment includes at least one of modification and deletion.

[0017] In one embodiment of this application, based on the aforementioned scheme, the task information includes multiple tasks, and the same task corresponds to a device simulation result and a device expected result; the generation module is specifically configured to: compare the device simulation result and the device expected result corresponding to the same task to obtain the comparison result of the same task, so as to obtain the comparison result of each task; and generate test results for the controller in the simulation network based on the comparison results corresponding to the multiple tasks respectively.

[0018] In one embodiment of this application, based on the foregoing scheme, the generation module is further specifically configured to: select, based on the comparison results corresponding to the plurality of tasks respectively, a task corresponding to the comparison result used to characterize the comparison matching from the plurality of tasks; if the number of selected tasks is greater than a preset number threshold, then a test result used to characterize that the controller performance in the simulation network is greater than a preset performance threshold is generated; if the number of selected tasks is less than or equal to the preset number threshold, then a test result used to characterize that the controller performance in the simulation network is less than or equal to the preset performance threshold is generated.

[0019] In one embodiment of this application, based on the foregoing scheme, the generation module is further configured to: select comparison results for characterizing comparison mismatch from the comparison results corresponding to the plurality of tasks respectively; generate management information corresponding to the selected comparison results; wherein the management information includes at least one of characterizing the cause of the comparison mismatch and characterizing the solution for the comparison mismatch; and generate test results for the controller in the simulation network based on the comparison results corresponding to the plurality of tasks respectively and the management information corresponding to the selected comparison results.

[0020] In one embodiment of this application, based on the aforementioned scheme, the controller in the data communication network follows the border gateway protocol; the acquisition module is specifically configured to: construct a simulation network corresponding to the data communication network; wherein, the controller in the simulation network follows the border network protocol; inject the routing information and the task information into the simulation network to obtain simulation results related to the simulation network.

[0021] Thirdly, embodiments of this application provide an electronic device, including one or more processors; and a memory for storing one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the network-based testing method described above.

[0022] Fourthly, embodiments of this application provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the network-based testing method described above.

[0023] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the network-based testing method described above.

[0024] In the technical solution provided by the embodiments of this application: by injecting the routing information of network devices in the data communication network and the task information of the controller in the data communication network into the simulation network to obtain the corresponding simulation results, and using the routing information of network devices in the data communication network, the task information of the controller in the data communication network, and the network environment information of the simulation network, the corresponding expected results are calculated, and then the simulation results and expected results are used to generate test results. Thus, automated testing of the controller in the simulation network is achieved. Compared with the manual testing schemes in related technologies, this avoids the low testing efficiency and accuracy caused by manual testing, greatly improving the efficiency and accuracy of testing, and ensuring the reliability of network-based testing.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating an exemplary implementation environment in which the technical solutions of the embodiments of this application can be applied.

[0027] Figure 2 This is a flowchart illustrating a network-based testing method as shown in an exemplary embodiment of this application.

[0028] Figure 3 This is a schematic diagram illustrating a test interface in an exemplary embodiment of this application.

[0029] Figure 4 This is a schematic diagram of a test system illustrated in an exemplary embodiment of this application.

[0030] Figure 5 This is a flowchart illustrating a network-based testing method, which is another exemplary embodiment of this application.

[0031] Figure 6 This is a schematic diagram illustrating the information acquisition stage of an exemplary embodiment of this application.

[0032] Figure 7 This is a schematic diagram illustrating the task playback test phase of an exemplary embodiment of this application.

[0033] Figure 8 This is a schematic diagram illustrating the network task calculation module calculating the expected result in an exemplary embodiment of this application.

[0034] Figure 9This is a block diagram illustrating a network-based testing apparatus as shown in an exemplary embodiment of this application.

[0035] Figure 10 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments identical to those described in this application. Rather, they are merely examples of apparatuses and methods identical to some aspects of this application as detailed in the appended claims.

[0037] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily need to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0040] It should be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0041] In related technologies, when it is necessary to test the controller in the network, relevant information is usually constructed manually for testing. Since manual testing requires a lot of manpower and resources and is prone to errors, it reduces the efficiency and accuracy of the test, and thus reduces the reliability of the test.

[0042] Therefore, to improve the efficiency and accuracy of controller testing in a network and ensure the reliability of network-based testing, this application provides a network-based testing scheme. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of an implementation environment related to this application. The implementation environment mainly includes a terminal device 101, a server 102, a data communication network 103, and a simulation network 104; wherein:

[0043] Terminal devices 101 include, but are not limited to, smartphones, computers (tablets, laptops, desktop computers, etc.), smart home devices (televisions, refrigerators, air conditioners, washing machines, robot vacuums, etc.), and smart wearable devices (wristbands, watches, etc.).

[0044] Server 102 can be a standalone physical server, or a server cluster or distributed system consisting of multiple physical servers. The server cluster or distributed system includes cloud servers used to provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0045] It is understood that the terminal device 101 and the server 102 support the testing system. Specifically, the testing application (including but not limited to applications, applets, or web pages) can be installed on the terminal device 101 as a client for users (such as testers) to use, while the server 102 provides corresponding background services for the testing application.

[0046] Data communication network 103 refers to the actual network infrastructure used for data transmission; optionally, the data communication network can be a local area network (LAN) or a wide area network (WAN). Data communication network 103 includes, but is not limited to, controllers 1031 (which may be one or more, exemplarily one) and network devices 1032 (which may be one or more, exemplarily multiple), wherein controller 1031 includes, but is not limited to, servers, terminal devices, etc., and network devices 1032 include, but are not limited to, routers, switches, wireless access points, load balancers, etc.

[0047] The simulation network 104 corresponds to the data communication network 103. It refers to the network after simulating the data communication network 103 for use in simulating / testing the data communication network 103. Optionally, the simulation network 104 can be a network reconfigured based on the data communication network 103, or a network adjusted based on the data communication network 103 to other network models. Similarly, the simulation network 104 includes, but is not limited to, controllers 1041 (which can be one or more, exemplarily one) and network devices 1042 (which can be one or more, exemplarily multiple). Controller 1041 corresponds to controller 1031 and includes, but is not limited to, servers, terminal devices, etc. Network devices 1042 correspond to network devices 1032 and include, but are not limited to, routers, switches, wireless access points, load balancers, etc.

[0048] In one embodiment of this application, the network-based test can be performed by server 102. Specifically, server 102 obtains routing information of network device 1032 and task information of controller 1031; injects the routing information and task information into the simulation network 104 to obtain simulation results related to the simulation network 104 (optionally, simulation results of network device 1042 and / or controller 1041); and calculates expected results related to the simulation network 104 based on the routing information, task information, and network environment information of the simulation network 104 (optionally, expected results of network device 1042 and / or controller 1041); and generates test results for controller 1041 based on the simulation results and expected results.

[0049] In other embodiments of this application, the network-based testing method can also be executed by the terminal device 101 alone, or by the terminal device 101 and the server 102 together. In practical applications, the executing entity of the network-based testing method can be flexibly adjusted according to the specific application scenario.

[0050] It should be made clear that, Figure 1 The number of terminal devices 101, servers 102, data communication networks 103, and simulation networks 104 in this diagram is merely illustrative. Depending on actual needs, there can be any number of terminal devices 101, servers 102, data communication networks 103, and simulation networks 104.

[0051] It should be noted that in the specific implementation of this application, user-related data is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0052] The following details the various implementation details of the technical solutions in the embodiments of this application:

[0053] Please see Figure 2 , Figure 2 This is a flowchart illustrating a network-based testing method according to an embodiment of this application. Figure 2 As shown, this network-based testing method includes at least S201 to S203, which are described in detail below:

[0054] S201, Obtain routing information of network devices in the data communication network, and obtain task information of the controller in the data communication network.

[0055] For ease of description, network devices in data communication networks are referred to as live network devices, controllers in data communication networks are referred to as live network controllers, network devices in simulated networks are referred to as simulated network devices, and controllers in simulated networks are referred to as simulated network controllers.

[0056] In the embodiments of this application, the routing information of the existing network device refers to the network path and related routing decision information used by the existing network device to guide the transmission of data packets from the source device to the destination device. It usually corresponds to a routing table, which includes, but is not limited to, destination address, next hop, interface, subnet mask, routing protocol, and metric.

[0057] In this application embodiment, the task information of the existing network controller refers to the specific tasks and related data involved in the implementation of task scheduling, management, and control. This includes, but is not limited to, task definitions (e.g., task identifier, task type, number of tasks, etc.), task dependencies (e.g., predecessor tasks, successor tasks, etc.), task parameters (e.g., task configuration, task execution time, etc.), task status (e.g., pending execution, executing, execution successful, execution failed, etc.), and task scheduling strategies (e.g., serial scheduling, parallel scheduling, scheduling according to task priority, etc.).

[0058] S202, the routing information and task information are injected into the simulation network corresponding to the data communication network to obtain the simulation results related to the simulation network, and the expected results related to the simulation network are calculated based on the routing information, task information, and network environment information of the simulation network.

[0059] In this embodiment, the routing information of the existing network devices and the task information of the existing network controller are obtained. Then, the routing information of the existing network devices and the task information of the existing network controller can be injected into the simulation network to obtain the simulation results related to the simulation network. At the same time, the expected results related to the simulation network can be calculated by using the routing information of the existing network devices, the task information of the existing network controller, and the network environment information of the simulation network.

[0060] In this embodiment of the application, the simulation result refers to the result generated after the routing information of the existing network device and the task information of the existing network controller are input into the simulation network for simulation and the task indicated by the task information is executed. The result is related to the device (including the simulated network device and / or the simulated network controller), including but not limited to the device status (e.g., on, off) and the routing table of the device.

[0061] In this application embodiment, the expected result refers to the result related to the device (including the simulated network device and / or simulated network controller) that is calculated using the routing information of the existing network device, the task information of the existing network controller, and the network environment information of the simulated network. Similarly, it includes, but is not limited to, the status of the device (e.g., on, off) and the routing table of the device.

[0062] In one embodiment of this application, the controller in the data communication network conforms to the Border Gateway Protocol (BGP); correspondingly, the process in S202 of injecting routing information and task information into the simulation network corresponding to the data communication network to obtain simulation results related to the simulation network may include:

[0063] Construct a simulation network corresponding to the data communication network; in the simulation network, the controller follows the boundary network protocol;

[0064] By injecting routing and task information into the simulation network, simulation results related to the simulation network are obtained.

[0065] That is, in the optional embodiment, the existing network controller is specifically the existing network BGP controller (also known as the existing network BGP service orchestration controller). Since the simulated network controller corresponds to the existing network controller, the simulated network controller is specifically the simulated network BGP controller (also known as the simulated network BGP service orchestration controller).

[0066] It is understood that, in optional embodiments, the existing network controller may also follow other network protocols, and correspondingly, the simulated network controller should also follow these other network protocols. Optionally, other network protocols include, but are not limited to, at least one of Interior Gateway Protocol (IGP), Multi-Protocol Label Switching (MPLS), Extended Border Gateway Protocol (eBGP), and Internal Border Gateway Protocol (iBGP); wherein the Interior Gateway Protocol includes, but is not limited to, at least one of Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS-IS), and Routing Information Protocol (RIP).

[0067] In one optional embodiment, a simulation network corresponding to the data communication network is constructed with reference to the data communication network, and then the routing information of the existing network devices and the task information of the existing network controller can be injected into the simulation network.

[0068] By implementing this optional embodiment, both the live network controller and the simulated network controller comply with the border gateway protocol, making them suitable for a wide range of testing scenarios.

[0069] In one embodiment of this application, the process of injecting routing information and task information into the simulation network corresponding to the data communication network in step S202 to obtain simulation results related to the simulation network may include:

[0070] The network devices in the simulation network are configured based on routing information to obtain the network devices in the simulation network configured with routing information, and the controllers in the simulation network are configured based on task information to obtain the controllers in the simulation network configured with task information.

[0071] By simulating network devices configured with routing information and controllers configured with task information in the network, tasks matching the task information are executed to obtain simulation results related to the simulation network.

[0072] In other words, in an optional embodiment, the routing information of the existing network devices is used to configure the simulated network devices so that the simulated network devices have the same routing information as the existing network devices. At the same time, the task information of the existing network controller is used to configure the simulated network controller so that the simulated network controller has the same task information as the existing network controller. Then, through the simulated network devices with the same routing information as the existing network devices and the simulated network controller with the same task information as the existing network controller, the tasks that match the task information (also known as the tasks indicated by the task information) are executed, thereby obtaining the simulation results corresponding to the simulated network.

[0073] By implementing this optional embodiment, the routing information of the simulated network device and the actual network device are the same, and the task information of the simulated network controller and the actual network controller are the same. This makes the consistency between the simulated network and the data communication network higher, and the simulation results obtained are more accurate, thereby improving the accuracy of the test.

[0074] In one embodiment of this application, the network devices in the simulated network include multiple devices; correspondingly, the process of configuring the network devices in the simulated network based on routing information to obtain network devices in the simulated network configured with routing information may include:

[0075] Obtain network device identification information;

[0076] Select the target network device that matches the network device identification information from among multiple network devices;

[0077] Configure the target network device based on the routing information to obtain the target network device with the routing information configured in the simulated network.

[0078] That is, in the optional embodiment, when there are multiple simulated network devices, the network device identification information is obtained first, then the target simulated network device that matches the network device identification information is selected from the multiple simulated network devices, and then the routing information is used to configure the target simulated network device so that the target simulated network device has the same routing information as the existing network device.

[0079] In the optional embodiment, the network device identification information obtained is used to uniquely identify the imitation network device, which includes, but is not limited to, the identification number and name of the imitation network device.

[0080] Optionally, the process of obtaining network device identification information may include: displaying a first input area in the test interface; and extracting network device identification information from the first input content in response to the first input content in the first input area. It is understood that when a user (e.g., a tester) has testing needs, they can determine the simulated network device to be tested. Specifically, they only need to enter the network device identification information corresponding to the access network device to be tested in the first input area of ​​the test interface. For example... Figure 3 As shown, the test interface includes a first input area 301, where testers can input network device identification information. This optional embodiment allows for simple and accurate acquisition of network device identification information with good interactivity.

[0081] In one optional embodiment, the target network device refers to a network device that matches the network device identification information selected from multiple network devices. It can be one or more, for example, multiple.

[0082] By implementing this optional embodiment, the routing information can be configured for the simulated network devices that need to participate in the test. This approach is highly targeted, flexible, and applicable to a wide range of testing scenarios.

[0083] In one embodiment of this application, the task information includes multiple tasks; correspondingly, the process of executing tasks matching the task information through network devices configured with routing information and controllers configured with task information in the simulation network to obtain simulation results related to the simulation network may include:

[0084] Each task is sent to a network device with routing information in the simulation network by a controller configured with task information in the simulation network, and the simulation results of the device corresponding to each task are obtained.

[0085] The simulation results of the devices corresponding to multiple tasks are combined to obtain the simulation results related to the simulation network.

[0086] That is, in the optional embodiment, when the task information includes multiple tasks, each task is sent to the simulated network device with the same routing information as the existing network device by the simulated network controller with the same task information as the existing network controller. Each task is executed by the simulated network device with the same routing information as the existing network device to obtain the device simulation result corresponding to each task, that is, to obtain multiple device simulation results. Then, the multiple device simulation results are combined to obtain the simulation result corresponding to the simulated network.

[0087] For example, please refer to Table 1 for the simulation results corresponding to an example simulation network.

[0088] Task 1 Equipment simulation results 1 Task 2 Equipment simulation results 2 Task 3 Equipment simulation results 2 …… ……

[0089] Table 1

[0090] By implementing this optional embodiment, the interaction between the network simulation controller and the network simulation device can be used to easily and accurately simulate the simulation results of the network, providing strong support for testing.

[0091] In one embodiment of this application, multiple tasks have an execution order; correspondingly, the process of sending each task to a network device configured with routing information in the simulation network through a controller configured with task information in the simulation network for execution, and obtaining the device simulation result corresponding to each task, may include:

[0092] Multiple tasks are traversed using a controller configured with task information in a simulation network.

[0093] The traversed tasks are sent to the network devices with routing information in the simulation network for execution, and the device simulation results corresponding to the traversed tasks are obtained. The next adjacent task is traversed until all tasks are traversed, and the device simulation results corresponding to each task are obtained.

[0094] That is, in the optional embodiment, when there is an execution order among multiple tasks, the simulated network controller with the same task information as the existing network controller traverses multiple tasks and sends the traversed tasks to a designated simulated network device with the same routing information as the existing network device. The designated simulated network device with the same routing information as the existing network device executes the received task to obtain the device simulation result corresponding to the received task. Then, the next adjacent task is traversed until all tasks are traversed, thereby obtaining multiple device simulation results.

[0095] In one optional embodiment, the simulated network controller, which has the same task information as the existing network controller, adopts a traversal method to send the traversed tasks to the designated simulated network devices, which have the same routing information as the existing network devices, for execution, so as to obtain simulation results of multiple devices.

[0096] Optionally, when the task information includes multiple tasks with an execution order, the traversal first iterates over the first task in the task information, then iterates over the next task adjacent to the first task, and so on, until the last task in the task information is reached. By implementing this optional embodiment, the existing execution order of the multiple tasks in the task information is utilized to traverse the multiple tasks sequentially, resulting in high traversal efficiency.

[0097] Optionally, if the task information includes multiple tasks but not their execution order, the tasks are sorted according to their execution order to obtain a task sequence. During traversal, the first task in the sequence is traversed first, followed by the next adjacent task, and so on, until the last task in the sequence is reached. This optional embodiment allows for real-time determination of the execution order of multiple tasks, eliminating the need for task information to indicate this order, thus reducing the storage size of task information and saving storage space. Furthermore, by utilizing the execution order of the tasks in the sequence, traversal is performed sequentially, resulting in high traversal efficiency.

[0098] For example, the process of sorting multiple tasks according to their execution order to obtain a task sequence may include: displaying a second input area in the test interface; and extracting the execution order of the multiple tasks from the second input content in response to the second input content. It is understood that when a user (e.g., a tester) has testing needs, they can determine (e.g., adjust) the execution order of multiple tasks; specifically, they only need to enter the execution order of the multiple tasks in the second input area of ​​the test interface. For example... Figure 3 As shown, the test interface includes a second input area 302, where testers can input the execution order. This optional embodiment allows for a simple and accurate determination of the execution order of multiple tasks, offering good interactivity.

[0099] In the optional embodiments, the specified network device refers to at least one of multiple network devices or multiple target network devices, and the specified network devices corresponding to different tasks can be the same or different.

[0100] By implementing this optional embodiment, multiple tasks can be executed sequentially in a traversal manner to obtain multiple device simulation results when there is an execution order among multiple tasks. This avoids execution errors, improves execution accuracy, and thus improves test accuracy.

[0101] In one embodiment of this application, there is no execution order among the multiple tasks, and the controller configured with task information in the simulation network includes multiple control logics; correspondingly, the process of sending each task to a network device configured with routing information in the simulation network for execution through the controller configured with task information in the simulation network, and obtaining the device simulation result corresponding to each task, may include:

[0102] Select the task corresponding to each control logic from multiple tasks;

[0103] Each control logic is run in parallel so that each control logic sends its corresponding task to a network device in the simulation network that is configured with routing information for execution, thereby obtaining the device simulation result for each task.

[0104] That is, in the optional embodiment where there is no execution order among multiple tasks, multiple control logics can be pre-deployed in the simulated network controller with the same task information as the existing network controller. Then, the task corresponding to each control logic is selected from the multiple tasks, and each control logic is run in parallel. Thus, through each running control logic, the task corresponding to each control logic is sent to a designated simulated network device with the same routing information as the existing network device. The received task is then executed by the designated simulated network device with the same routing information as the existing network device corresponding to each control logic, and the device simulation result corresponding to the received task is obtained. Thus, multiple device simulation results are obtained.

[0105] In the optional embodiments, the specified network device refers to at least one of multiple network devices or multiple target network devices, and the specified network devices corresponding to different tasks can be the same or different.

[0106] By implementing this optional embodiment, multiple tasks can be executed in parallel by multiple control logics to obtain multiple device simulation results when there is no specific execution order among the multiple tasks. This improves execution efficiency and thus testing efficiency.

[0107] In one embodiment of this application, the task information includes multiple tasks; correspondingly, the process in S202 of calculating the expected results related to the simulation network based on routing information, task information, and network environment information of the simulation network may include:

[0108] Select the target computation logic that matches the task type of each task from multiple computation logics;

[0109] Run the target computation logic so that the target computation logic performs calculations based on routing information, network environment information, and the tasks corresponding to the target computation logic, and obtains the expected results of the device for each task;

[0110] The expected results of the devices corresponding to multiple tasks are combined to obtain the expected results related to the simulation network.

[0111] That is, in the optional embodiment, when the task information includes multiple tasks, multiple computational logics can be generated in advance, and then a target computational logic matching the task type of each task can be selected from the multiple computational logics. The target computational logic is then run, and the target computational logic is used to perform calculations using routing information, network environment information, and the task corresponding to the target computational logic to obtain the expected results of each task, that is, multiple expected results of devices are obtained. Then, the multiple expected results of devices are combined to obtain the expected results corresponding to the simulated network.

[0112] For example, please refer to Table 2 for the expected results corresponding to an example simulation network.

[0113] Target calculation logic 1 Task 1 Expected Equipment Results 1 Target calculation logic 2 Task 2 Expected Equipment Results 2 Target calculation logic 3 Task 3 Expected Equipment Results 3 …… …… ……

[0114] Table 2

[0115] In one of the optional embodiments, multiple computational logics are pre-written by the user (e.g., a tester) using a programming language; specifically, each computational logic can correspond to a computational logic file, or multiple computational logics can correspond to a single computational logic file.

[0116] Optionally, the process of deriving multiple computational logics may include: displaying a third input area in the test interface; extracting multiple computational logics from the third input content in response to the third input content in the third input area; and displaying the multiple computational logics in the test interface. For example... Figure 3 As shown, the test interface includes a third input area 303, where testers can input multiple calculation logics. This allows for the simple and accurate acquisition of multiple calculation logics through the implementation of this optional embodiment, offering good interactivity.

[0117] For example, the third input area in the test interface includes an upload control. Correspondingly, the process of extracting multiple computational logics from the third input content in response to the third input content of the third input area may include: entering a selection interface in response to a trigger operation on the upload control, the selection interface including multiple locally stored computational logics; and selecting the selected computational logics as the extracted computational logics in response to a selection operation based on the selection interface. Thus, by implementing this optional embodiment, multiple computational logics can be obtained from the local storage with high efficiency. It is understood that the process of deriving multiple computational logics in the optional embodiment is essentially a process of adding computational logics.

[0118] For example, after displaying multiple computational logics in the test interface, the method may further include: adjusting the selected computational logic in response to a selection operation on the multiple computational logics; wherein the adjustment includes at least one of modification and deletion. That is, after adding multiple computational logics, any one of the multiple computational logics can be modified and / or deleted. By implementing this optional embodiment, the added computational logic can be modified and / or deleted in real time, which is highly targeted, flexible, and applicable to many test scenarios.

[0119] In the optional embodiment, the target computation logic refers to the computation logic that matches the task type of the task selected from multiple computation logics. The number of target computation logics corresponds to the number of tasks. That is, if there is one task, there is one target computation logic; if there are multiple tasks, there are multiple target computation logics.

[0120] By implementing this optional embodiment, the pre-written computational logic can autonomously utilize routing information, network environment information of the simulated network, and its corresponding tasks to calculate the expected results corresponding to the simulated network. This results in a high degree of intelligence and a low probability of error, thereby improving the efficiency and accuracy of testing.

[0121] In one embodiment of this application, multiple tasks have an execution order, and the target computation logic includes multiple components. Accordingly, the process of running the target computation logic to perform calculations based on routing information, network environment information, and the tasks corresponding to the target computation logic, to obtain the expected device result for each task, may include:

[0122] The computation logic of multiple targets is sorted according to the order of execution to obtain the computation logic sequence;

[0123] The algorithm iterates through multiple target computation logics in the computation logic sequence and runs the traversed target computation logics. The traversed target computation logics perform calculations based on routing information, network environment information, and the tasks corresponding to the traversed target computation logics. This yields the expected device results for the tasks that match the traversed target computation logics. The network environment information is then updated based on the obtained expected device results. The algorithm then iterates through the next adjacent target computation logic until all multiple target computation logics have been traversed, and the expected device results for each task are obtained.

[0124] That is, in the optional embodiment, when there is an execution order among multiple tasks, the multiple target computation logics are first sorted according to the execution order to obtain a computation logic sequence. Then, the multiple target computation logics in the computation logic sequence are traversed and the traversed target computation logics are run. Thus, the target computation logics run are used to perform calculations using routing information, network environment information, and the tasks corresponding to the run target computation logics to obtain the expected device results corresponding to the tasks matching the run target computation logics. The network environment information is then updated using the obtained expected device results. Then, the next adjacent target computation logic is traversed until all target computation logics are traversed, thereby obtaining multiple expected device results.

[0125] By implementing this optional embodiment, in situations where there is an execution order among multiple tasks, multiple target computation logics are run serially using a traversal approach. This allows for the serial computation of multiple target computation logics to obtain the expected results for multiple devices, avoiding computational errors, improving computational accuracy, and thus enhancing the accuracy of testing.

[0126] In one embodiment of this application, there is no execution order among the multiple tasks, and the target computation logic includes multiple components; accordingly, the process of running the target computation logic to perform calculations based on routing information, network environment information, and the tasks corresponding to the target computation logic to obtain the expected results for each task may include:

[0127] Each target computation logic runs in parallel, so that each target computation logic performs calculations based on routing information, network environment information, and the task corresponding to each target computation logic, and obtains the expected result of the device corresponding to each task.

[0128] That is, in the optional embodiment, when there is no execution order among multiple tasks, each target computing logic is run in parallel. By using routing information, network environment information, and the task corresponding to each target computing logic, the expected results of the device corresponding to the task matching each target computing logic are obtained, thereby obtaining multiple expected results of the device.

[0129] By implementing this optional embodiment, multiple target computation logics can be run in parallel when there is no execution order among multiple tasks. This allows multiple devices to obtain their expected results through parallel computation of multiple target computation logics, thereby improving computational efficiency and thus testing efficiency.

[0130] S203 generates test results for the controller in the simulated network based on the simulation results and expected results.

[0131] In this embodiment of the application, the simulation results corresponding to the simulation network are obtained by simulation, and the expected results corresponding to the simulation network are calculated. Then, the simulation results corresponding to the simulation network and the expected results corresponding to the simulation network can be used to generate test results for the controller in the simulation network.

[0132] In one embodiment of this application, the task information includes multiple tasks, with each task corresponding to both device simulation results and expected device results; correspondingly, the process in S203 of generating test results for the controller in the simulation network based on the simulation results and expected results may include:

[0133] The simulation results and expected results of the equipment corresponding to the same task are compared to obtain the comparison results for the same task, so as to obtain the comparison results for each task.

[0134] Based on the comparison results corresponding to multiple tasks, test results are generated for the controller in the simulation network.

[0135] That is, in the optional embodiment, when the task information includes multiple tasks, each task corresponds to a device simulation result and a device expected result. Then, the device simulation result and the device expected result corresponding to the same task are compared to obtain the comparison result corresponding to the same task, that is, the comparison result of each task is obtained. Then, the comparison results corresponding to multiple tasks are used to generate test results for the controller in the simulation network.

[0136] For example, following the examples in Tables 1 and 2 above, please refer to Table 3, which shows the test results for one example.

[0137] Task 1 Compare the simulation results of equipment 1 with the expected results of equipment 1 Comparison Result 1 Task 2 Compare the simulation results of equipment 2 with the expected results of equipment 2 Comparison Result 2 Task 3 Compare the simulation results of equipment 3 with the expected results of equipment 3 Comparison Result 3 …… …… ……

[0138] Table 3

[0139] By implementing this optional embodiment, the test results of the simulation controller are generated using the comparison results corresponding to multiple tasks, thus avoiding the randomness of the comparison results of a single task and improving the accuracy of the test.

[0140] In one embodiment of this application, the process of generating test results for the controller in the simulation network based on the comparison results corresponding to multiple tasks may include:

[0141] Select alignment results from the alignment results corresponding to multiple tasks to characterize alignment mismatches;

[0142] Generate management information corresponding to the selected comparison results; wherein, the management information includes at least one of the following: a method to characterize the cause of the comparison mismatch and a method to characterize the solution to the comparison mismatch.

[0143] Based on the comparison results corresponding to multiple tasks and the management information corresponding to the selected comparison results, test results are generated for the controller in the simulation network.

[0144] That is, in the optional embodiment, for the task of comparison mismatch, the cause and / or solution of comparison mismatch can be generated, thereby combining the comparison results corresponding to multiple tasks, as well as the generated cause and / or solution of comparison mismatch, to generate the test result of the network controller.

[0145] For example, following the example in Table 3, please refer to Table 4 for the test results of another example.

[0146]

[0147]

[0148] Table 4

[0149] By implementing this optional embodiment, test results are generated using the causes and / or solutions to the mismatches. Users (e.g., testers) can intuitively and clearly understand the causes and / or solutions to the mismatches. This highly intelligent approach avoids the low efficiency and accuracy caused by manual determination, thereby improving the efficiency and accuracy of the test.

[0150] In one embodiment of this application, the process of generating test results for the controller in the simulated network based on simulation results and expected results may include:

[0151] Based on the comparison results corresponding to multiple tasks, select the task corresponding to the comparison result used to characterize the comparison matching from multiple tasks;

[0152] If the number of selected tasks exceeds a preset threshold, test results are generated to characterize the controller performance in the simulation network as exceeding a preset performance threshold.

[0153] If the number of selected tasks is less than or equal to a preset threshold, test results are generated to characterize the performance of the controller in the simulation network as being less than or equal to a preset performance threshold.

[0154] That is, in the optional embodiment, the final test result is obtained based on the number of matching tasks among multiple tasks. If the number of matching tasks is large, the test result indicates that the performance of the network simulation controller is better. If the number of matching tasks is small, the test result indicates that the performance of the network simulation controller is worse.

[0155] In the optional embodiment, "more tasks" means the number of tasks is greater than a preset number threshold, "less tasks" means the number of tasks is less than or equal to a preset number threshold, "better performance" means the performance is greater than a preset performance threshold, and "poor performance" means the performance is less than or equal to a preset performance threshold. In practical applications, the preset number threshold and preset performance threshold can be flexibly adjusted according to specific application scenarios.

[0156] By implementing this optional embodiment, the test results of the simulated network controller can be generated easily and accurately by utilizing the number of matching tasks, making it suitable for a variety of test scenarios.

[0157] In one embodiment of this application, the controller in the simulation network has additional functions compared to the controller in the data communication network; alternatively, after constructing a simulation network corresponding to the data communication network with reference to the data communication network, the function to be tested is deployed to the simulation network so that the simulation network has the added functions.

[0158] Accordingly, after generating test results for the controller in the simulation network based on the simulation results and expected results in S203, the process may further include:

[0159] If the test results indicate that the performance of the controller in the simulation network is greater than the preset performance threshold, then the controller in the data communication network is updated based on the functional data of the newly added function.

[0160] In other words, in the optional embodiment, when the test results indicate that the performance of the simulated network controller is better, the existing network controller can be updated using the functional data of the new function so that the updated existing network controller has the same new function as the simulated network controller.

[0161] By implementing this optional embodiment, when it is necessary to update the function on the existing network controller, the function is first tested and updated only after the test is passed. This avoids the phenomenon of poor user experience caused by directly updating due to function problems, improves the user experience, and ensures the efficiency and accuracy of the update.

[0162] This application embodiment implements automated testing of controllers in simulated networks. Compared with the manual testing schemes in related technologies, it avoids the phenomenon of low testing efficiency and accuracy caused by manual testing, thereby improving the efficiency and accuracy of testing. The network-based testing has high reliability.

[0163] The following provides a detailed description of specific scenarios in the embodiments of this application:

[0164] Please see Figure 4 , Figure 4 This is a schematic diagram of a test system illustrated in one embodiment of this application. Figure 4As shown, the test system includes a network task replay test service, a data communication network (corresponding to a real network environment, referred to as the real environment), and a simulation network (corresponding to a simulated network environment, referred to as the simulation environment, test environment, etc.), wherein:

[0165] The network task replay testing service interfaces with both real and simulation environments. It mainly includes a network task acquisition module, a route acquisition module, a network task calculation module, a network task verification module, and a route injection module.

[0166] Optionally, the network task acquisition module is mainly used to acquire historical and / or existing orchestration task information of the existing network controller. This task information may include task information (e.g., traffic isolation tasks, plane isolation tasks, line isolation tasks, whitelist tasks, etc.), the network device plane on which the task is applied, and the information of the applied device. The main input of the network task acquisition module is the access information of the existing network controller (e.g., address, port, username, and password), and the main output is historical and / or existing orchestration task information (which may include task type, affected plane, device information, and link information).

[0167] Optionally, the route acquisition module is mainly used to obtain routing information of existing network devices. The main inputs to the route acquisition module are network device information (such as IP address, device plane, etc.), login credentials, and acquisition protocols (such as BGP, Simple Network Management Protocol (SNMP), Secure Shell (SSH), etc.). The main output is the routing table of the existing network devices (which may include route entries, destination addresses, next-hop addresses, etc.).

[0168] Optionally, the route injection module is mainly used to inject the collected routing information of the existing network devices and the task information of the existing network controller into the simulation environment. Specifically, the routing information of the existing network devices is injected into the simulated network devices as background test routes, and the task information of the existing network controller is injected into the simulated network controller as background test tasks. The main inputs of the route injection module are the routing information of the existing network devices collected by the route acquisition module, and the device information of the simulated network devices in the simulation environment (such as device IP addresses, login credentials, etc.). The main output is the result of the injection operation (which may include successfully injected route information, failed injection route information, and the reasons for failure).

[0169] Optionally, the network task calculation module is mainly used to calculate the expected state that the routing information in the simulated network devices should maintain after passing the task indicated by the task information, using the routing information of the existing network devices, the task information of the existing network controller, and the network environment information of the simulation environment. These expected states can be the specific state that the network devices in the simulation environment should maintain after a task is successfully executed, or the existence of specific routing information, as well as the state that the controller should maintain (e.g., the network scheduling sub-tasks that should be issued). The main inputs of the network task calculation module are the routing information of the existing network devices, the task information of the existing network controller, and the network environment information of the simulation environment (e.g., topology, link information, device information, etc.). After calculation, the main output is the expected state information (which may include the specific state that the controller should maintain, the specific state that the device should maintain, and the expected routing information).

[0170] Optionally, the network task verification module is mainly used to verify whether the simulation state of the test device (i.e., the simulated network device and / or simulated network controller) matches the expected state after the task indicated by the task information is executed (e.g., it can check whether a specific state such as power on / off is maintained, or whether a specific route exists). The main inputs of the network task verification module are the simulation state information of the test device (also called actual state information, such as actual power on / off state, actual routing information, etc., i.e., the device simulation result in the aforementioned embodiments) and the expected state information (e.g., expected power on / off state, expected routing information, etc., i.e., the expected device result in the aforementioned embodiments). The main output is the verification result (which may include whether the actual state matches the expected state, the differences in cases of mismatch, and possible causes and solutions).

[0171] based on Figure 4 Please refer to the test system shown. Figure 5 , Figure 5 This is a flowchart illustrating a network-based testing method according to an embodiment of this application. Figure 5 As shown, this network-based testing method includes at least S501 to S509, which are detailed below:

[0172] S501, obtains routing information of network devices in the data communication network, and obtains task information of the controller in the data communication network.

[0173] For easier understanding, please refer to Figure 6 This is a schematic diagram of the information collection phase involved in S501. For example... Figure 6 As shown, it mainly includes:

[0174] (1) Testers input the topology information and device information (such as IP address, plane to which the device belongs, login credentials, and data collection protocol) of network devices in the real environment (i.e., live network devices), the topology information and device information (such as IP address, plane to which the device belongs, login credentials, and data collection protocol) of network devices in the simulation environment (i.e., simulated network devices), and the correspondence between the topology information of live network devices and simulated network devices into the information storage module. This provides basic data for subsequent routing information collection and simulation environment setup. At the same time, it ensures the correspondence between the simulation environment and the real environment for better testing.

[0175] (2) Once the tester has completed the data entry and submitted it, the routing information collection will be triggered to ensure that the routing information collection can start in a timely manner after the tester has completed the data entry, so as to prepare for the subsequent task playback test phase.

[0176] (3) The routing acquisition module accesses the existing network devices and obtains their routing information based on the device information (such as Internet Protocol (IP) address, the plane to which the device belongs, etc.), login credentials, and acquisition protocols (such as BGP, SNMP, SSH, etc.) of the existing network devices.

[0177] (4) The routing acquisition module stores the collected routing information of the existing network devices into the information storage module.

[0178] (5) Testers input historical task replay test task information of the controller in the real environment (i.e., the live network controller, such as the BGP service orchestration controller) into the information storage module. This information may include test tasks for a certain period of time that need to be replayed, task types (such as traffic isolation tasks, plane isolation tasks, line isolation tasks, whitelist tasks, etc.), the network device plane on which the task is applied, the device information, and the access information of the live network controller (such as address, port, username and password, etc.).

[0179] (6) When the tester completes the data entry and submits it, the task information collection will be triggered to ensure that the task information collection can start in a timely manner after the tester completes the data entry, so as to prepare for the subsequent task playback test phase.

[0180] (7) The network task acquisition module accesses and obtains the task information of the existing network controller based on the access information of the existing network controller (such as address, port, username and password).

[0181] (8) The network task acquisition module collects the task information of the live network controller and stores it in the information storage module.

[0182] S502, configure the network devices in the simulation network based on routing information to obtain the network devices in the simulation network configured with routing information, and configure the controller in the simulation network based on task information to obtain the controller in the simulation network configured with task information.

[0183] S503 sends each task indicated by the task information to the network device with routing information in the simulation network through the controller configured with task information in the simulation network for execution, and obtains the device simulation result corresponding to each task.

[0184] S504 combines the simulation results of the devices corresponding to multiple tasks to obtain simulation results related to the simulation network.

[0185] S505 selects the target computation logic that matches the task type of each task from multiple computation logics.

[0186] S506, run the target calculation logic so that the target calculation logic performs calculations based on routing information, network environment information, and the tasks corresponding to the target calculation logic, and obtains the expected results of the device corresponding to each task.

[0187] S507 combines the expected results of the devices corresponding to multiple tasks to obtain the expected results related to the simulation network.

[0188] S508 compares the equipment simulation results and the expected equipment results for the same task to obtain the comparison results for the same task, thus obtaining the comparison results for each task.

[0189] S509 generates test results for the controller in the simulation network based on the comparison results corresponding to multiple tasks.

[0190] For easier understanding, please refer to Figure 7 This is a schematic diagram illustrating the task replay testing phases involved in S501 to S509. For example... Figure 7 As shown, it mainly includes:

[0191] (1) The tester triggers the network task replay test command, transmitting the task information to be replayed. The network task calculation module retrieves the task information, routing information, and network environment information of the simulation environment from the information storage module. It can be understood that the routing information and the network environment information of the simulation environment can be referred to as the initial state network information.

[0192] (2) The network task calculation module calculates the expected state of the simulation environment after each task is executed in the task information. This may include the storage of the initial network environment information, all task information that needs to be replayed, the expected results of the task, and the result information of each task's effect on the network.

[0193] For example, please refer to Figure 8 This diagram illustrates the expected results calculated by the network task calculation module. It is understood that the network task calculation module defines a set of mapping functions (i.e., multiple calculation logics in the aforementioned embodiments, which may include plane isolation task mapping functions, line isolation task mapping functions, traffic suppression task mapping functions, etc.), which are written by testers. The input to the mapping function is network information (which may include network topology information such as link information in the simulated network, device connection relationships, network device information such as IP status of network devices, routing information of each network device, etc.), and the corresponding task (which may include task type, parameters to be passed for different tasks, such as whitelist tasks should pass the network segment information corresponding to the whitelist, isolation tasks should pass the corresponding isolated network segment, plane, community attributes, etc.), and the output is the state that should be achieved after executing the corresponding task.

[0194] like Figure 8 As shown, it mainly includes: (1) inputting network device information, routing information, and task information (including multiple tasks, also known as a task information list); (2) selecting the corresponding mapping function according to the task type; (3) outputting the expected state of network information and task information under the action of the mapping function. When mapping the next task, the expected state obtained from this mapping will be used as the network information to be input.

[0195] (3) The network task calculation module stores the expected state into the information storage module.

[0196] (4) The routing injection module obtains the initial network information from the information storage module.

[0197] (5) The routing injection module injects the initial network information into the simulation environment.

[0198] (6) The network task verification module retrieves task information and the expected status of each task in the task information from the information storage module. It can be understood that the task information and the expected status of each task in the task information can be called the revisit test task information.

[0199] Optionally, for each task and the expected state of each task, it may be possible to execute (7-1) to (7-3).

[0200] (7-1) The network task verification module transmits task information and triggers the task issuance by calling the interface of the controller in the simulation environment (i.e., the network simulation controller, such as the BGP service orchestration controller).

[0201] (7-2) The cloning controller sends the task to the designated cloning network device, which then executes the received task to obtain the actual status of the task.

[0202] (7-3) After the task is completed, the network task verification module compares whether the expected state and the actual state of the task match (e.g., whether the routing information before and after the task is executed matches).

[0203] (8) After multiple tasks are completed, the task verification module generates test results (e.g., test result report) based on the comparison results corresponding to the multiple tasks respectively, and returns the test results to the network task calculation module so that testers can view them.

[0204] The embodiments of this application have at least the following technical effects:

[0205] The system enhances the realism of testing and improves the stability of the testing system: By simulating a real environment, the testing system can provide more realistic, comprehensive and accurate test results, enabling testers to better understand the performance of the controller in the actual network and to perceive potential risks in advance, so as to optimize the controller and improve the quality of network services.

[0206] It improves testing efficiency and reduces maintenance costs: The automated collection of routing and task information in the real environment and the automated injection into the simulation environment greatly improve testing efficiency and reduce the workload of manual testing. In addition, the testing system is easy to maintain and reduces maintenance costs.

[0207] Supports rapid upgrades / updates, shortening project cycles: This testing system provides strong support for controller upgrades / updates. By simply modifying the test task calculation module and adding mapping functions, testing can be performed quickly, accelerating the controller development process, shortening the project cycle, and enabling products to be launched to the market faster. At the same time, it can also adapt to changes in controller orchestration methods, offering high flexibility.

[0208] Figure 9 This is a block diagram illustrating a network-based testing apparatus according to one embodiment of this application. Figure 9 As shown, the device includes:

[0209] The acquisition module 901 is configured to acquire routing information of network devices in the data communication network and task information of the controller in the data communication network;

[0210] The injection and calculation module 902 is configured to inject the routing information and the task information into the simulation network corresponding to the data communication network, obtain the simulation results related to the simulation network, and calculate the expected results related to the simulation network based on the routing information, the task information, and the network environment information of the simulation network.

[0211] The generation module 903 is configured to generate test results for the controller in the simulation network based on the simulation results and the expected results.

[0212] In one embodiment of this application, based on the aforementioned scheme, the injection and calculation module 902 is specifically configured as follows: configuring network devices in the simulation network based on the routing information to obtain network devices in the simulation network configured with the routing information; configuring controllers in the simulation network based on the task information to obtain controllers in the simulation network configured with the task information; and executing tasks matching the task information through the network devices in the simulation network configured with the routing information and the controllers configured with the task information to obtain simulation results related to the simulation network.

[0213] In one embodiment of this application, based on the aforementioned scheme, the network devices in the simulated network include multiple network devices; the injection and calculation module 902 is further specifically configured to: obtain network device identification information; select a target network device from the multiple network devices that matches the network device identification information; configure the target network device based on the routing information to obtain a target network device in the simulated network configured with the routing information.

[0214] In one embodiment of this application, based on the foregoing scheme, the task information includes multiple tasks; the injection and calculation module 902 is further specifically configured to: send each task to a network device in the simulation network that is configured with the routing information through a controller in the simulation network, thereby obtaining the device simulation result corresponding to each task; and combine the device simulation results corresponding to the multiple tasks to obtain the simulation result related to the simulation network.

[0215] In one embodiment of this application, based on the aforementioned scheme, there is an execution order among the multiple tasks; the injection and calculation module 902 is further specifically configured to: traverse the multiple tasks through a controller configured with the task information in the simulation network; send the traversed tasks to a network device configured with the routing information in the simulation network for execution, obtain the device simulation result corresponding to the traversed task, and traverse the next adjacent task until all the multiple tasks have been traversed, and obtain the device simulation result corresponding to each of the multiple tasks.

[0216] In one embodiment of this application, based on the aforementioned scheme, there is no execution order among the multiple tasks, and the controller configured with the task information in the simulation network includes multiple control logics; the injection and calculation module 902 is further specifically configured to: select a task corresponding to each control logic from the multiple tasks; run each control logic in parallel, so that each control logic sends the task corresponding to each control logic to a network device configured with the routing information in the simulation network for execution, thereby obtaining the device simulation result corresponding to each task.

[0217] In one embodiment of this application, based on the aforementioned scheme, the task information includes multiple tasks; the injection and calculation module 902 is specifically configured to: select a target calculation logic from multiple calculation logics that matches the task type of each task; run the target calculation logic so that the target calculation logic performs calculations based on the routing information, the network environment information, and the task corresponding to the target calculation logic to obtain the expected device result corresponding to each task; and combine the expected device results corresponding to the multiple tasks to obtain the expected result related to the simulation network.

[0218] In one embodiment of this application, based on the aforementioned scheme, the multiple tasks have an execution order, and the target computation logic includes multiple tasks; the injection and computation module 902 is further specifically configured to: sort the multiple target computation logics according to the execution order to obtain a computation logic sequence; traverse the multiple target computation logics in the computation logic sequence and run the traversed target computation logics, so that the traversed target computation logics perform computations based on the routing information, the network environment information, and the tasks corresponding to the traversed target computation logics, to obtain the expected device result corresponding to the task matching the traversed target computation logics, and update the network environment information based on the obtained expected device result, and traverse the next adjacent target computation logic until all the multiple target computation logics are traversed, to obtain the expected device result corresponding to each task.

[0219] In one embodiment of this application, based on the aforementioned scheme, there is no execution order among the multiple tasks, and the target calculation logic includes multiple tasks; the injection and calculation module 902 is further configured to: run each target calculation logic in parallel, so that each target calculation logic performs calculations based on the routing information, the network environment information, and the task corresponding to each target calculation logic, to obtain the expected result of the device corresponding to each task.

[0220] In one embodiment of this application, based on the foregoing solution, the device further includes an extraction module configured to: display an input area in a test interface; extract the plurality of computational logics from the input content in response to the input content in the input area; and display the plurality of computational logics in the test interface.

[0221] In one embodiment of this application, based on the foregoing scheme, the extraction module is further configured to: adjust the selected computational logic in response to a selection operation for the plurality of computational logics; wherein the adjustment includes at least one of modification and deletion.

[0222] In one embodiment of this application, based on the aforementioned scheme, the task information includes multiple tasks, with each task corresponding to a device simulation result and a device expected result; the generation module 903 is specifically configured to: compare the device simulation result and the device expected result corresponding to the same task to obtain the comparison result of the same task, so as to obtain the comparison result of each task; and generate test results for the controller in the simulation network based on the comparison results corresponding to the multiple tasks respectively.

[0223] In one embodiment of this application, based on the foregoing scheme, the generation module 903 is further specifically configured to: select, based on the comparison results corresponding to the plurality of tasks respectively, a task corresponding to the comparison result used to characterize the comparison matching from the plurality of tasks; if the number of selected tasks is greater than a preset number threshold, then a test result used to characterize that the controller performance in the simulation network is greater than a preset performance threshold is generated; if the number of selected tasks is less than or equal to the preset number threshold, then a test result used to characterize that the controller performance in the simulation network is less than or equal to the preset performance threshold is generated.

[0224] In one embodiment of this application, based on the foregoing scheme, the generation module 903 is further configured to: select comparison results for characterizing comparison mismatch from the comparison results corresponding to the plurality of tasks respectively; generate management information corresponding to the selected comparison results; wherein the management information includes at least one of characterizing the cause of the comparison mismatch and characterizing the solution for the comparison mismatch; and generate test results for the controller in the simulation network based on the comparison results corresponding to the plurality of tasks respectively and the management information corresponding to the selected comparison results.

[0225] In one embodiment of this application, based on the aforementioned scheme, the controller in the data communication network follows the border gateway protocol; the acquisition module 901 is specifically configured to: construct a simulation network corresponding to the data communication network; wherein, the controller in the simulation network follows the border network protocol; inject the routing information and the task information into the simulation network to obtain simulation results related to the simulation network.

[0226] It should be noted that the apparatus provided in the foregoing embodiments and the method provided in the foregoing embodiments belong to the same concept, and the specific way in which each module and unit performs operations has been described in detail in the method embodiments.

[0227] Embodiments of this application also provide an electronic device, including: one or more processors; and a memory for storing one or more computer programs, which, when executed by one or more processors, cause the electronic device to implement the aforementioned network-based testing method.

[0228] Figure 10 It is an electronic device suitable for implementing the embodiments of this application (e.g. Figure 1 The diagram shows the structure of a computer system (terminal device or server).

[0229] It should be noted that, Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0230] like Figure 10 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on a computer program stored in a Read-Only Memory (ROM) 1002 or a computer program loaded from a storage portion 1008 into a Random Access Memory (RAM) 1003. The RAM 1003 also stores various computer programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0231] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0232] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer instructions for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0233] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. For example, a computer-readable medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0234] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0235] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0236] Another aspect of this application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the network-based testing method as described above. This computer-readable medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0237] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable medium. A processor of an electronic device reads the computer instructions from the computer-readable medium and executes the computer instructions, causing the electronic device to perform the network-based testing methods provided in the various embodiments described above.

[0238] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A network-based testing method, characterized in that, include: Obtain routing information of network devices in the data communication network, and obtain task information of the controller in the data communication network; The routing information and the task information are injected into the simulation network corresponding to the data communication network to obtain the simulation results related to the simulation network. Based on the routing information, the task information, and the network environment information of the simulation network, the expected results related to the simulation network are calculated. Based on the simulation results and the expected results, test results are generated for the controller in the simulation network.

2. The method according to claim 1, characterized in that, The step of injecting the routing information and the task information into the simulation network corresponding to the data communication network to obtain simulation results related to the simulation network includes: Based on the routing information, the network devices in the simulation network are configured to obtain the network devices in the simulation network configured with the routing information. Based on the task information, the controllers in the simulation network are configured to obtain the controllers in the simulation network configured with the task information. By using network devices configured with the routing information and controllers configured with the task information in the simulation network, tasks matching the task information are executed to obtain simulation results related to the simulation network.

3. The method according to claim 2, characterized in that, The simulated network includes multiple network devices; configuring the network devices in the simulated network based on the routing information to obtain network devices in the simulated network configured with the routing information includes: Obtain network device identification information; Select a target network device that matches the network device identification information from among multiple network devices; Based on the routing information, the target network device is configured to obtain the target network device in the simulated network configured with the routing information.

4. The method according to claim 2, characterized in that, The task information includes multiple tasks; the process involves executing tasks matching the task information through network devices configured with the routing information and controllers configured with the task information in the simulation network to obtain simulation results related to the simulation network, including: The controller configured with the task information in the simulation network sends each task to a network device configured with the routing information in the simulation network for execution, thereby obtaining the device simulation result corresponding to each task. The simulation results of the devices corresponding to the multiple tasks are combined to obtain the simulation results related to the simulation network.

5. The method according to claim 4, characterized in that, The multiple tasks have an execution order; the process of sending each task to a network device configured with routing information in the simulation network via a controller configured with the task information, and obtaining the device simulation result corresponding to each task, includes: The multiple tasks are traversed through the controller configured with the task information in the simulation network; The traversed tasks are sent to the network devices in the simulation network that are configured with the routing information for execution, so as to obtain the device simulation results corresponding to the traversed tasks. The next adjacent task is traversed until all the tasks are traversed, so as to obtain the device simulation results corresponding to the multiple tasks respectively.

6. The method according to claim 4, characterized in that, There is no execution order among the multiple tasks, and the controller configured with the task information in the simulation network includes multiple control logics; the step of sending each task to a network device configured with the routing information in the simulation network through the controller configured with the task information to obtain the device simulation result corresponding to each task includes: Select the task corresponding to each control logic from the plurality of tasks; Each control logic is run in parallel so that each control logic sends the task corresponding to it to the network device in the simulation network that is configured with the routing information for execution, thereby obtaining the device simulation result corresponding to each task.

7. The method according to claim 1, characterized in that, The task information includes multiple tasks; the calculation of expected results related to the simulation network based on the routing information, the task information, and the network environment information of the simulation network includes: Select the target computation logic that matches the task type of each task from multiple computation logics; The target computation logic is run so that it performs calculations based on the routing information, the network environment information, and the task corresponding to the target computation logic, to obtain the expected result of the device corresponding to each task; The expected results of the devices corresponding to the multiple tasks are combined to obtain the expected results related to the simulation network.

8. The method according to claim 7, characterized in that, The multiple tasks have an execution order, and the target computation logic includes multiple components; running the target computation logic, so that the target computation logic performs calculations based on the routing information, the network environment information, and the task corresponding to the target computation logic, to obtain the expected device result corresponding to each task, includes: The multiple target computation logics are sorted according to the execution order to obtain a computation logic sequence; The system iterates through multiple target computation logics in the computation logic sequence and runs the traversed target computation logics. The traversed target computation logics perform calculations based on the routing information, the network environment information, and the tasks corresponding to the traversed target computation logics. This yields the expected device result for the task that matches the traversed target computation logic. The system then updates the network environment information based on the obtained expected device result and iterates through the next adjacent target computation logic until all multiple target computation logics have been traversed, thus obtaining the expected device result for each task.

9. The method according to claim 7, characterized in that, There is no execution order among the multiple tasks, and the target computation logic includes multiple components; running the target computation logic, so that the target computation logic performs calculations based on the routing information, the network environment information, and the task corresponding to the target computation logic, to obtain the expected device result corresponding to each task, includes: Each target computation logic is run in parallel, so that each target computation logic performs calculations based on the routing information, the network environment information, and the task corresponding to each target computation logic, to obtain the expected result of the device corresponding to each task.

10. The method according to claim 7, characterized in that, Before selecting the target computation logic that matches the task type of each task from multiple computation logics, the method further includes: Display the input area in the test interface; In response to the input content in the input area, the plurality of computational logics are extracted from the input content; The multiple computational logics are displayed in the test interface.

11. The method according to claim 10, characterized in that, After displaying the plurality of computational logics in the test interface, the method further includes: In response to a selection operation for the plurality of computational logics, the selected computational logic is adjusted; wherein the adjustment includes at least one of modification and deletion.

12. The method according to any one of claims 1 to 11, characterized in that, The task information includes multiple tasks, with each task corresponding to both device simulation results and expected device results; the step of generating test results for the controller in the simulation network based on the simulation results and the expected results includes: The simulation results and expected results of the equipment corresponding to the same task are compared to obtain the comparison results for the same task, so as to obtain the comparison results for each task. Based on the comparison results corresponding to the multiple tasks, test results are generated for the controller in the simulation network.

13. The method according to claim 12, characterized in that, The step of generating test results for the controller in the simulation network based on the simulation results and the expected results includes: Based on the comparison results corresponding to the multiple tasks, select the task corresponding to the comparison result used to characterize the comparison matching from the multiple tasks; If the number of selected tasks is greater than a preset threshold, test results are generated to characterize that the controller performance in the simulation network is greater than a preset performance threshold. If the number of selected tasks is less than or equal to the preset number threshold, test results are generated to characterize that the controller performance in the simulation network is less than or equal to the preset performance threshold.

14. The method according to claim 12, characterized in that, The step of generating test results for the controller in the simulation network based on the comparison results corresponding to the multiple tasks includes: Select the alignment results used to characterize the alignment mismatch from the alignment results corresponding to the multiple tasks respectively; Generate management information corresponding to the selected comparison results; wherein, the management information includes at least one of the following: a method for characterizing the cause of the comparison mismatch and a method for characterizing the solution to the comparison mismatch. Based on the comparison results corresponding to the multiple tasks and the management information corresponding to the selected comparison results, test results are generated for the controller in the simulation network.

15. The method according to any one of claims 1 to 11, characterized in that, The controller in the data communication network follows the border gateway protocol; the process of injecting the routing information and the task information into the simulation network corresponding to the data communication network to obtain simulation results related to the simulation network includes: Construct a simulation network corresponding to the data communication network; wherein, the controller in the simulation network follows the boundary network protocol; The routing information and the task information are injected into the simulation network to obtain the simulation results related to the simulation network.

16. A network-based testing device, characterized in that, include: The acquisition module is configured to acquire routing information of network devices in the data communication network and task information of the controller in the data communication network. The injection and calculation module is configured to inject the routing information and the task information into the simulation network corresponding to the data communication network, obtain the simulation results related to the simulation network, and calculate the expected results related to the simulation network based on the routing information, the task information, and the network environment information of the simulation network. The generation module is configured to generate test results for the controller in the simulation network based on the simulation results and the expected results.

17. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more computer programs that, when executed by the electronic device, cause the electronic device to implement the network-based testing method according to any one of claims 1 to 15.

18. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the network-based testing method according to any one of claims 1 to 15.

19. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the network-based testing method according to any one of claims 1 to 15.

Images