Virtual network element testing method, device, electronic device and readable storage medium

Abstract

The embodiments of the present application provide a virtual network element testing method, device, electronic device and readable storage medium, which relate to the field of testing technology. The method includes: receiving a test request, the test request includes test case determination information, determining a target test case that matches the test case information based on the test case determination information, executing each target test case respectively, and obtaining a test result file for each target test case, and the test result file for each target test case may contain the message data of at least one corresponding serviced interface. The virtual network element testing method provided in the present application can determine the target test case based on the test case determination information in the test request, and execute each target test case respectively to obtain the corresponding test result, and generate a test report based on the test result statistics of each target test case, so as to conveniently and quickly realize the automated testing of virtual network elements, improve the test efficiency, and better meet the actual application needs.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to a virtual network element testing method, device, electronic device and readable storage medium. Background Art

[0002] With the development of 5G (Generation Mobile Communication Technology), the development and deployment of 5G core network has become more convenient and faster.

[0003] As a new type of mobile communication network, 5G not only solves the communication between people and provides users with more immersive and ultimate business experience such as augmented reality, virtual reality, and ultra-high-definition (3D) video, but also solves the communication problems between people and things, and between things and things, and meets the needs of Internet of Things applications such as mobile medical care, Internet of Vehicles, smart home, industrial control, and environmental monitoring.

[0004] As 5G technology emerges in all walks of life, a large number of testing requirements for 5G core network elements and related business processes have been generated. However, in the existing test system, the configuration process of the test process is relatively complicated, and the configuration options are not diverse and flexible enough, which makes the testing method of core network elements not convenient enough and the testing efficiency low, making it difficult to meet the actual testing needs. Summary of the invention

[0005] The purpose of the embodiments of the present application is to solve the problem that the testing method is not convenient enough and the testing efficiency is low.

[0006] In a first aspect, a virtual network element testing method is provided, the method comprising:

[0007] receiving a test request for a virtual network element, the test request including test case determination information;

[0008] Based on the test case determination information, at least one target test case matching the test case determination information is determined from preconfigured test cases, each target test case corresponds to at least one service-based interface, and the service-based interface is an interface between virtual network elements;

[0009] Execute each target test case separately to obtain the test result of each target test case, and generate a test report based on the test result statistics of each target test case.

[0010] In an optional embodiment of the first aspect, the test case determination information is at least one of test case storage location information and a keyword, and based on the test case determination information, determining at least one target test case matching the test case determination information from a plurality of test cases comprises:

[0011] If the test case determination information is the test case storage location information, the test case corresponding to the test case storage path is used as the target test case;

[0012] If the test case determination information is a keyword, the keyword is matched with the test case description information of each test case in the preconfigured test cases, and each test case corresponding to the test case description information containing the keyword is determined as a target test case.

[0013] If the test case determination information includes test case storage location information and keywords, determine at least one candidate test case based on the test case storage location information, match the keywords with the test case description information of each candidate test case, and determine the candidate test case corresponding to the test case description information containing the keywords as the target test case.

[0014] In an optional embodiment of the first aspect, the preconfigured test case is obtained by:

[0015] Receive test case configuration information through the test case configuration interface, generate and store test cases based on the test case configuration information;

[0016] For each test case, the test case configuration information includes a test case identifier and a test process file, wherein the test process file includes a test script and business parameters of at least one service-based interface corresponding to the test script.

[0017] In an optional embodiment of the first aspect, the test case configuration interface includes a virtual network element selection control, receiving test case configuration information through the test case configuration interface, and generating a test case based on the test case configuration information, including:

[0018] In response to a trigger operation on a virtual network element selection control, displaying a virtual network element list;

[0019] receiving a selection operation for a virtual network element in the virtual network element list, and determining the selected virtual network element as a target virtual network element;

[0020] Receive test case configuration information for the target virtual network element, and generate a test case corresponding to the target virtual network element based on the test case configuration information.

[0021] In an optional embodiment of the first aspect, the test case configuration interface includes a template selection control, receiving test case configuration information through the test case configuration interface, and generating a test case based on the test case configuration information, including:

[0022] In response to a trigger operation on a template selection control, displaying a template list;

[0023] receiving a selection operation for a test case template in the template list, and determining the selected test case template as a target template;

[0024] Receive test case configuration information based on the target template, and generate test cases according to the target template and the test case configuration information.

[0025] In an optional embodiment of the first aspect, the test case configuration information of each test case further includes at least one of the following:

[0026] The waiting time after any service-oriented interface corresponding to the test case is called;

[0027] Preconfigured interface call conditions for any service-based interface.

[0028] In an optional embodiment of the first aspect, the method further comprises:

[0029] For each target test case, the test result file is compared with the reference result file corresponding to the target test case to obtain a comparison result;

[0030] Generate a test report based on the test result files and comparison results of each target test case.

[0031] In a second aspect, a virtual network element testing device is provided, the device comprising:

[0032] A test request receiving module, used to receive a test request for a virtual network element, wherein the test request includes test case determination information;

[0033] A test case determination module, configured to determine, based on the test case determination information, at least one target test case that matches the test case determination information from preconfigured test cases, each target test case corresponding to at least one service-based interface, which is an interface between virtual network elements;

[0034] The test case execution module is used to execute each target test case respectively and obtain the test result of each target test case;

[0035] The test report generation module is used to generate a test report based on the test result statistics of each target test case.

[0036] In an optional embodiment of the second aspect, the test case determination information is at least one of test case storage location information and a keyword, and the test case determination module, when determining at least one target test case matching the test case determination information from multiple test cases based on the test case determination information, is specifically used to:

[0037] If the test case determination information is the test case storage location information, the test case corresponding to the test case storage path is used as the target test case;

[0038] If the test case determination information is a keyword, the keyword is matched with the test case description information of each test case in the preconfigured test cases, and each test case corresponding to the test case description information containing the keyword is determined as a target test case.

[0039] If the test case determination information includes test case storage location information and keywords, determine at least one candidate test case based on the test case storage location information, match the keywords with the test case description information of each candidate test case, and determine the candidate test case corresponding to the test case description information containing the keywords as the target test case.

[0040] In an optional embodiment of the second aspect, a test case configuration module is further included, which is specifically used to:

[0041] Receive test case configuration information through the test case configuration interface, generate and store test cases based on the test case configuration information;

[0042] For each test case, the test case configuration information includes a test case identifier and a test process file, wherein the test process file includes a test script and business parameters of at least one service-based interface corresponding to the test script.

[0043] In an optional embodiment of the second aspect, the test case configuration interface includes a virtual network element selection control, and the test case configuration module receives the test case configuration information through the test case configuration interface and generates a test case based on the test case configuration information, specifically for:

[0044] In response to a trigger operation on a virtual network element selection control, displaying a virtual network element list;

[0045] receiving a selection operation for a virtual network element in the virtual network element list, and determining the selected virtual network element as a target virtual network element;

[0046] Receive test case configuration information for the target virtual network element, and generate a test case corresponding to the target virtual network element based on the test case configuration information.

[0047] In an optional embodiment of the second aspect, the test case configuration interface includes a template selection control, and the test case configuration module receives the test case configuration information through the test case configuration interface and generates the test case based on the test case configuration information, specifically for:

[0048] In response to a trigger operation on a template selection control, displaying a template list;

[0049] receiving a selection operation for a test case template in the template list, and determining the selected test case template as a target template;

[0050] Receive test case configuration information based on the target template, and generate test cases according to the target template and the test case configuration information.

[0051] In an optional embodiment of the second aspect, the test case configuration information of each test case further includes at least one of the following:

[0052] The waiting time after any service-oriented interface corresponding to the test case is called;

[0053] Preconfigured interface call conditions for any service-based interface.

[0054] In an optional embodiment of the second aspect, a test result comparison module is further included, which is specifically used to:

[0055] For each target test case, the test result file is compared with the reference result file corresponding to the target test case to obtain a comparison result;

[0056] Generate a test report based on the test result files and comparison results of each target test case.

[0057] In a third aspect, an electronic device is provided, the electronic device comprising:

[0058] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the virtual network element testing method of any of the above embodiments is implemented.

[0059] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the virtual network element testing method of any of the above embodiments is implemented.

[0060] In a fifth aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that when the computer device executes the computer instructions, the method provided in the embodiment of the first aspect is implemented.

[0061] The above-mentioned virtual network element testing method receives a test request, and the test request includes test case determination information. Based on the test case determination information, at least one target test case that matches the test case information can be determined from multiple test cases. Each target test case is executed separately to obtain a test result file for each target test case. The test result file for each target test case can contain message data sent by the corresponding at least one serviced interface, and the message data can be a request message and / or a response message. The virtual network element testing method provided in the present application can determine the target test case based on the test case determination information in the test request, and execute each target test case separately to obtain the corresponding test results. A test report is generated based on the test result statistics of each target test case, which conveniently and quickly realizes automated testing of virtual network elements, improves testing efficiency, and better meets actual application needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0063] Figure 1 A schematic diagram of the functional architecture of a system for implementing a virtual network element testing method provided in an embodiment of the present application;

[0064] Figure 2 A flowchart of a virtual network element testing method provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of storage directories and test case identifiers corresponding to each test case in a virtual network element testing method provided in an embodiment of the present application;

[0066] Figure 4 A flowchart of a virtual network element testing method provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of a test report in a virtual network element testing method provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of the structure of a virtual network element testing device provided in an embodiment of the present application;

[0069] Figure 7 A schematic diagram of the structure of an electronic device for virtual network element testing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0071] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0072] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0073] First, several terms involved in this application are introduced and explained:

[0074] The fifth generation of mobile communication technology (i.e. 5G): a new generation of broadband mobile communication technology with high speed, low latency and large connection characteristics, is the network infrastructure for realizing the interconnection of man, machine and things. The core network architecture based on the fifth generation of mobile communication technology breaks the traditional network architecture. It abandons the traditional network architecture mode of dedicated hardware, dedicated communication links and paths, dedicated platforms, and point-to-point interfaces, and is committed to an efficient, software-based, and development-oriented system network architecture.

[0075] 3GPP (3rd Generation Partnership Project): A standards organization that defines complete end-to-end system specifications to ensure compliance with industry requirements, seamless interoperability between different vendors, and the global scale necessary for mobile.

[0076] Core Network: When the mobile network is divided into three parts: base station subsystem, network subsystem and system support part, the core network part is located in the network subsystem. The main function of the core network is to connect the call request or data request from port A to different networks.

[0077] Network element: A device in a network, also called a network entity, which can be composed of one or more disks or frames and can independently complete certain transmission functions. There are many types of granularity for network elements, and different network elements can provide different services.

[0078] Protocol Data Unit (PDU), in a layered network structure, such as in the Open Systems Interconnection model, a protocol data unit is created at each layer of the transmission system. The PDU contains information from the upper layer and additional information from the entity of the current layer. This PDU will be transmitted to the next lower layer. The physical layer actually transmits these PDUs in the form of a framed bit stream, which are constructed by the higher layers of the protocol stack. The receiving system transmits these packets through the protocol stack from bottom to top and separates the relevant information in the PDU at each layer of the protocol stack.

[0079] The User Plane Function (UPF), as a basic component of 5GC, is used to realize the separation of the User Plane (CP) and the Control Plane (UP).

[0080] Access and Mobility Management Function (AMF): Responsible for managing handovers between base stations within the next generation radio access network.

[0081] With the development of 5G mobile communication technology, 5G technology is increasingly used in various application scenarios. In order to improve development efficiency and accelerate R&D iteration, the demand for automated testing of 5G business processes is particularly prominent. However, in the existing testing solutions, there is a lack of testing systems that fully support 5G service-oriented architecture and 3GPP interface specifications, and the testing system is often not flexible and convenient when configuring the test process, resulting in low testing efficiency.

[0082] The virtual network element testing method, device, electronic device and readable storage medium provided in the present application are intended to solve at least one of the above technical problems in the prior art.

[0083] Among them, the embodiment of the present application provides a virtual network element testing method, which can be implemented as a 5G network element automated testing system. Optionally, the functional architecture diagram of the system can be as follows Figure 1As shown, the technical solution of the embodiment of the present application can achieve the technical effect of flexible and convenient automated testing of core network elements.

[0084] Specifically, the main functions of the 5G network element automated testing system may include test case scheduling function and network element test simulator function.

[0085] The test case scheduling function may refer to the ability of the system to schedule execution based on test cases. Figure 1 The automated test case execution function in the example may refer to the system matching the storage directory and test case identifier of all test cases according to the specified test case storage path and keyword, determining the target test case, and then calling the specified command to complete the execution of the target test case; the test case result collection and reading function ( Figure 1 The use case result collection in the test report may mean that the system can collect the test result file of each target test case, and can also compare the test result file with the pre-configured result file of the test case test that is a successful test result (i.e., the reference result file); the automated test report generation function may mean that the system can integrate the result file of each use case into the test report, and can also automatically count the pass rate, time consumption and other information of the entire test process.

[0086] The network element test simulator function can be that the system can combine virtual machine technology to simulate the 5G core network element to initiate service interface access and implement corresponding business process calls, including UE (User Equipment) access, establishment of PDU session, etc. Optional, such as Figure 1 As shown, the network element test simulator function can realize business process configuration, test process scheduling, test information management, test parameter configuration, network element simulation and service interface access, among which: the test parameter configuration function means that the parameter information required by each service interface in the test process can be read from the test case; the business process configuration function can mean that the process scheduling information can be read from the test case, and the process scheduling information includes information such as test steps, whether to wait for interface access completion, and whether to exit upon failure; the test process scheduling function can be to call the service interface one by one according to the parameters in the test case to realize the execution of the entire business process; the network element simulation function can mean that the system can start the corresponding HTTP / 2 service according to the test case to accept service access requests from other network elements; the service interface access function can be to access the corresponding service interface through HTTP / 2 based on the configured interface parameters to complete the interface call; the test information management function can mean to collect the request and response messages of each service interface to generate a test case result file.

[0087] Optional, in Figure 1The test process configuration function and the test case information configuration function on the left may provide optional test case templates to testers, so that testers can write test case source code files based on the test case templates and configure the test process and test case information in the test case source code files.

[0088] In summary, this application Figure 1 The 5G network element automated testing system provided can mainly realize the following functions:

[0089] (1) For a single test case, the service-based interface to be tested can be determined according to the test case, and each service-based interface is tested in turn according to the test process in the test case to obtain the test results of each service-based interface, and the test results of each service-based interface are compared with the reference result file to obtain the corresponding result file;

[0090] (2) For the test case library, there can be many test cases in the test case library. Each test case can be executed in turn to test the service interface involved in each test case, and the result file of each test case can be obtained. Then, the result file of each test case can be integrated into the test report, and the pass rate, time consumption and other information of the entire test process can be automatically counted.

[0091] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0092] The present application provides a method for testing a virtual network element. Figure 2 As shown, the method includes:

[0093] Step S201: receiving a test request for a virtual network element, wherein the test request may include test case determination information.

[0094] In an embodiment of the present application, a test case configuration interface may be provided, and a technician may pre-configure multiple test cases through the test case configuration interface. Each test case may include input data, test steps, expected results, test scripts, etc. for different test objectives and test environments.

[0095] The test case determination information includes but is not limited to the test case storage location information (for example, storage path) of the test case and at least one of the keywords, wherein the keywords can be matched with the test case description information of each test case, and the test case description information may include a test case identifier, a test environment of the test case, or a network element identifier of a virtual network element corresponding to the test case, which is used to filter out test cases of a specified type, such as test cases running in a specified test environment or test cases for a specified virtual network element.

[0096] In step S202, based on the test case determination information, at least one target test case matching the test case determination information can be determined from multiple test cases, each target test case corresponds to at least one service-based interface, and the service-based interface is an interface between virtual network elements.

[0097] In an embodiment of the present application, at least one target test case matching the test case determination information can be determined from a plurality of pre-configured test cases based on the test case determination information. Specifically, each test case can include a configurable test process, and each test process can include a series of test steps, such as a set of 5G service-oriented interface calls or test scripts.

[0098] The service-oriented interface in this application refers to the interface between virtual network elements. The virtual network element may refer to a network element of a simulated core network in a virtual test environment built based on a virtual machine. For example, in one example, a virtual network element may be a base station, an access and mobility management function AMF network element, a session management function SMF network element, or a user plane function UPF network element, etc. Various virtual network elements may be simulated according to actual test application requirements.

[0099] In an embodiment of the present application, each test case may correspond to at least one business process in the 5G communication system, that is, one test case may be used to implement automated testing of at least one business process.

[0100] In an example, each test case may include a configurable test flow file, and the test flow file may include at least one test script and corresponding business parameters of at least one service-based interface.

[0101] Specifically, a test script may include computer-readable instructions for automatically executing a test process (or part of a test process), and these instructions may be executed by an automated testing tool.

[0102] Service-based interface can be an interface exposed by a function body through service registration and service discovery similar to microservice architecture. This interface can be targeted at a single function body. The related function body interacts with other function bodies through the service-based interface, and other function bodies interact with this function body through the interface exposed by the function body. This mechanism provides a many-to-one access mechanism, and because of the use of service registration and service discovery, each party can access the other party without knowing the address of the other party.

[0103] In an embodiment of the present application, the test process file may include a test script and the business parameters of at least one corresponding service-oriented interface. The business parameters may be the parameters required to call the service-oriented interface, including but not limited to the identifier of the virtual network element corresponding to each service-oriented interface and the protocol stack information, manufacturer, type, IP (Internet Protocol) address, port, etc. of the virtual network element.

[0104] When executing a test case, a test script in the test case may be executed, and a business parameter of at least one service-based interface corresponding to the test script may be called.

[0105] It is understandable that, in addition to the test script, the test case may also include a test case description file, which may include business parameters of at least one service-based interface corresponding to the test case for easy viewing by testers.

[0106] Step S203, respectively execute each target test case to obtain a test result file of each target test case; wherein the content contained in the test result file of each target test case can be set according to the test requirements, and different test requirements can be set to return different data.

[0107] Specifically, the message data may be a request message and a response message, and the test result file may be generated based on the request message and the response message of each service-oriented interface corresponding to the target test case. The structure of the message may be determined based on the applicable protocol. As an example, the message structure of the Hyper Text Transfer Protocol (HTTP) may be:

[0108] The start line is used to describe the message;

[0109] The header is used to add some additional information to the message. The header and the protocol work together to determine the functions that the client and server can implement;

[0110] Body, which contains the main part of the data.

[0111] In an embodiment of the present application, for each target test case, the test result file is compared with the reference result file corresponding to the target test case to obtain a comparison result; then, a test report can be generated based on the test result file of each target test case and the comparison result.

[0112] Each test case may have a corresponding reference result file. After executing each target test case and obtaining the test result file of each target test case, for each target test case, the reference result file and the test result file corresponding to the target test case may be compared to obtain a comparison result. The comparison result may include but is not limited to the similarity between the reference result file and the test result file, and the test result for the target test case may be evaluated based on the similarity.

[0113] The test result files and comparison results of each target test case can be integrated to generate a test report for this test and archive it for easy viewing and analysis by testers.

[0114] As a possible implementation method, when multiple target test cases are determined based on the test case determination information in the test request during the detection of the virtual network element, each of the target test cases can be executed separately to obtain a test result file for each of the target test cases, and for each target test case, the reference result file and the test result file corresponding to the target test case can be compared to obtain a comparison result, which can be the similarity between the reference result file and the test result file.

[0115] A similarity threshold can be further set. For each target test case, when the comparison result (the similarity between the reference result file and the test result file) is greater than the similarity threshold, the test result of the target test case is recorded as passed in the test report of this detection. The pass rate, time consumption and other information can be counted in the test report generated at the end to evaluate the test results as a whole.

[0116] Step S204: Generate a test report based on the test result statistics of each target test case.

[0117] In an embodiment of the present application, the result file of each target test case can be integrated into the test report, and the pass rate, time consumption and other information of the entire test process can be automatically counted to realize automated testing of virtual network elements and generate test reports. R&D personnel can clearly and conveniently understand the test results of the virtual network elements from the test report.

[0118] The above-mentioned virtual network element testing method receives a test request, and the test request includes test case determination information. Based on the test case determination information, at least one target test case that matches the test case information can be determined from multiple test cases. Execute each target test case separately to obtain a test result file for each target test case. The test result file for each target test case can contain message data sent by the corresponding at least one service-based interface, and the message data can be a request message and / or a response message. The virtual network element testing method provided in the present application can determine the target test case based on the test case determination information in the test request, and execute each target test case separately to obtain the corresponding test results. A test report is generated based on the test result statistics of each target test case, so as to realize convenient and fast automated testing of virtual network elements, reduce development costs, improve development efficiency, and promote rapid iteration of 5G core network development.

[0119] In an embodiment of the present application, a possible implementation method is provided, where the test case determination information is at least one of the test case storage location information and a keyword. Based on the test case determination information, at least one target test case matching the test case determination information is determined from multiple test cases, which may include the following situations:

[0120] (1) If the test case determination information is the test case storage location information, the test case corresponding to the test case storage path is used as the target test case;

[0121] (2) If the test case determination information is a keyword, the keyword is matched with the test case description information of each test case in the preconfigured test cases, and each test case corresponding to the test case description information containing the keyword is determined as a target test case.

[0122] (3) If the test case determination information includes test case storage location information and keywords, at least one candidate test case is determined based on the test case storage location information, the keywords are matched with the test case description information of each candidate test case, and the candidate test case corresponding to the test case description information containing the keywords is determined as the target test case.

[0123] Among them, the test case description information can be a test case identifier. The specific form of the test case identifier is not limited in the embodiment of the present application. For example, the test case identifier can be the name or ID (Identity document) of the test case, and the keyword can be the characters in the test case identifier. Based on the keyword, the test case corresponding to the test case identifier containing the keyword can be determined from multiple test cases, and the determined test case can be used as the target test case.

[0124] The test case description information may also include a service-based interface identifier and a test environment identifier, etc. The service-based interface identifier in the test case identifier of the test case may be the identifier of the service-based interface to be tested by the test case (including but not limited to the number of the service-based interface, etc.), and the test environment identifier may be the version number or test environment parameter of the test environment applicable to the test case. Correspondingly, the keywords may also be set to the service-based interface identifier and the test environment identifier to determine at least one target test case.

[0125] In one example, if Figure 3 As shown, it is assumed that there are pre-configured test cases A, B, C, D and E, each with its own corresponding test case identifier: "Function One A", "Function Two B", "Function Two C", "Function Three D" and "Function Three E". The test case identifiers describe the functions to be tested for each test case.

[0126] Among them, test case A and test case B are stored in path A, and test case C, test case D and test case E are stored in path B.

[0127] When the test case determination information is a test case storage path (i.e., test case storage location information) and a keyword, assuming that the test case storage path in the test case determination information is path A and the keyword is "function two", test case A and test case B can be determined based on the test case storage path, and then test case B can be determined as a target test case based on the keyword "function two";

[0128] When the test case storage path is path B and the keyword is "function three", test case C, test case D and test case E can be determined based on the test case storage path, and then test case D and test case E can be determined as target test cases according to the keyword "function three".

[0129] A possible implementation method is provided in the embodiment of the present application, and the test case can be obtained in the following way:

[0130] Receive test case configuration information through the test case configuration interface, generate and store test cases based on the test case configuration information; wherein the test case configuration information may be the source code of the test case written by the tester, or may be the configuration information obtained by the tester through the test case interface;

[0131] Specifically, generating test cases based on test case configuration information may include directly obtaining test cases by test personnel according to test case templates. At this time, the test case configuration information may be various parameters that will be used in the test case when testing. The test case may also be a general framework of the test case given by a preconfigured test case template, and the test personnel fill in the test case configuration information in the template to generate a configured test case. The specific implementation method is not limited in this application.

[0132] For each test case, the test case configuration information may include a test case identifier and a test flow file, wherein the test flow file includes a test script and business parameters of at least one service-oriented interface corresponding to the test script. The test script may refer to a series of instructions for a specific test, which can be executed by an automated testing tool.

[0133] In the embodiment of the present application, the business parameters of at least one service-oriented interface may include at least one of the following:

[0134] (1) The waiting time after any service-oriented interface corresponding to the test case is called;

[0135] (2) Preconfigured interface calling conditions for any of the service-based interfaces; specifically, the interface calling conditions can be set based on the parameters in the response message returned by the last called service-based interface. Only when the parameters in the response message returned by the last called service-based interface meet the preset interface calling conditions can the service-based interface corresponding to the interface calling conditions continue to be called.

[0136] (3) The business parameters of each service-oriented interface may include the number corresponding to the service-oriented interface in the test case. The number is used to determine the calling order of each service-oriented interface when executing the test case.

[0137] In the embodiment of the present application, by configuring the business parameters of at least one service-oriented interface in the test case configuration information, the test process corresponding to the test case can be configured. Specifically, the configuration effect achieved can be at least one of the following:

[0138] (1) Repeat the specified test steps in the test process of the test case multiple times; specifically, the process of calling a service-based interface for testing can be used as a test step, and a specified service-based interface can be repeatedly called for testing as needed to achieve repeated execution of specified test steps and increase the flexibility of the test process.

[0139] (2) After the current test step is completed, wait for a specified time before continuing to execute the next test step; during the waiting time, test data fed back after the execution of the current test step can be received;

[0140] (3) Whether to continue the test process can be determined based on whether the test data obtained after the previous step meets the preset conditions corresponding to the current step. If so, continue to execute;

[0141] (4) The calling order between the service interfaces can be determined according to the corresponding numbers of each service interface in the test case, thereby determining the execution order of the test steps.

[0142] In an embodiment of the present application, the test case configuration interface may include a virtual network element selection control, and receiving test case configuration information through the test case configuration interface and generating a test case based on the test case configuration information may include the following steps:

[0143] (1) In response to a trigger operation on a virtual network element selection control, display a virtual network element list; the trigger operation may include but is not limited to a mouse click, a keyboard input, a touch operation, and a voice command; the virtual network element list may display the identifiers of each virtual network element in a preset virtual network element library;

[0144] (2) receiving a selection operation for a virtual network element in the virtual network element list, and determining the selected virtual network element as a target virtual network element; specifically, the target virtual network element may be determined from a preset virtual network element library; the virtual network element library contains virtual network elements that have been pre-configured, and each virtual network element has corresponding basic configuration information (for example, configuration information that is not easy to change, such as manufacturer and type), so that the tester does not need to repeatedly configure the basic configuration information, thereby improving the network element testing efficiency;

[0145] (3) Receive test case configuration information for the target virtual network element, and generate a test case corresponding to the target virtual network element based on the test case configuration information; specifically, the test case configuration information for the target virtual network element here may be configuration information of the target virtual network element other than the basic configuration information that needs to be flexibly configured according to the test content; for example, a virtual network element may have multiple callable service interfaces, and the test case configuration information may include service interface selection information and service interface configuration information, wherein the service interface selection information is used to determine which service interface of the virtual network element to call, and the service interface configuration information may include configuration information for business parameters of the service interface (for example, protocol stack information, IP address and port, etc.), waiting time after the service interface is called, etc.

[0146] Specifically, in the embodiment of the present application, the configuration method of the execution order of the service-oriented interface may include but is not limited to the following:

[0147] The multiple two service interfaces in the test case are numbered respectively. When executing the test case, the numbers of each service interface are read first, and the execution order of each service interface in the test script of the test case is determined according to the numbers; if a service interface needs to be called multiple times, the code segment for calling the service interface can be written multiple times in the test script, and the code segment corresponding to the service interface can be numbered multiple times, or the code segment of the service interface can be written only once, but the code segment corresponds to multiple numbers in the test script. The specific implementation method is not limited in this application;

[0148] A test case configuration interface can be pre-set in the 5G network element automation test system obtained after the virtual network element testing method provided in the present application is implemented, and at least two service interfaces can be sorted based on the test case configuration interface; the specific implementation method can be: first configure the business parameters of each service interface in the test case separately, and then use the identifier of each service interface to refer to the relevant data of each service interface, and sort the execution order of the corresponding service interfaces by sorting the identifiers of each service interface in the test case configuration interface. After the sorting of the identifiers of the service interfaces is completed, the system automatically generates a test case after the execution order is configured.

[0149] (3) Provide testers with preset test case templates. The test case templates provide code writing rules. The code writing rules can prompt testers on how to sort the execution order of each service-based interface when writing code; for example, each service-based interface can have a corresponding service-based interface identifier, and a parameter can be set as "next service-based interface identifier" in the business parameters of each service-based interface. The value of the parameter is set to the identifier of the next service-based interface in the execution order, thereby connecting the service-based interfaces in series.

[0150] In an embodiment of the present application, a possible implementation method is provided. The test case configuration interface includes a template selection control. Then, test case configuration information is received through the test case configuration interface, and a test case is generated based on the test case configuration information. The following steps may be included:

[0151] (1) In response to a trigger operation on a template selection control, displaying a template list; wherein the template list may display each test case template in a preset test case template library;

[0152] (2) receiving a selection operation for a test case template in the template list, and determining the selected test case template as a target template;

[0153] (3) Receive test case configuration information based on the target template, and generate test cases according to the target template and the test case configuration information.

[0154] In order to more clearly explain the virtual network element testing method of the present application, the virtual network element testing method will be further described below with reference to specific examples.

[0155] In one example, if Figure 4 As shown, the virtual network element testing method provided in this application may include the following steps:

[0156] Step S401, accepting a test request, the test request may include test case determination information; the test case determination information may include a test case storage path (i.e., test case storage location information) and a keyword;

[0157] Step S402, traverse each test case and determine the storage directory and test case identifier of each test case;

[0158] Step S403, for the current test case, first determine whether the test case storage path in the test case determination information matches the storage directory of the test case, if so, proceed to step S404, otherwise, proceed to step S402 to match the next test case;

[0159] Step S404, for the current test case, matching the keyword in the test case determination information with the test case identifier of the test case, if the test case identifier matches the keyword, proceeding to step S405, otherwise proceeding to step S402;

[0160] Step S405, executing the current test case to obtain a test result file;

[0161] Step S406, comparing the test result file with the corresponding reference result file to obtain a comparison result;

[0162] Step S407, updating the test report based on the comparison result and the test result file;

[0163] Step S408, check whether the test case traversal is complete, if yes, go to step S409, otherwise go to step S402;

[0164] Step S409: generate a test report based on statistics.

[0165] In one example, a test report may include: Figure 5 The items shown and the specific contents of each item, for example, Figure 5 The "start" in the test is the test start time, which records the test start time, that is, the time when the test case starts to execute.

[0166] Specifically, the comparison result can be similarity, and a similarity threshold can be further set. For each target test case, when the comparison result (the similarity between the reference result file and the test result file) is greater than the similarity threshold, the test result of the target test case is recorded as passed in the test report of this detection. The pass rate, time consumption and other information can be counted in the test report generated at the end to evaluate the test results as a whole.

[0167] The present application embodiment provides a virtual network element testing device, such as Figure 6 As shown, the virtual network element testing device 60 may include: a test request receiving module 601, a test case determining module 602, a test case executing module 603 and a test report generating module 604, wherein:

[0168] A test request receiving module, used to receive a test request, wherein the test request includes test case determination information;

[0169] A test case determination module, configured to determine, based on the test case determination information, at least one target test case that matches the test case determination information from a plurality of test cases, each of the target test cases corresponding to at least one service-based interface, the service-based interface being an interface between virtual network elements;

[0170] A test case execution module, used to execute each of the target test cases respectively and obtain the test results of each of the target test cases;

[0171] The test report generation module is used to generate a test report based on the test result statistics of each target test case.

[0172] The above-mentioned virtual network element testing method receives a test request, and the test request includes test case determination information. Based on the test case determination information, at least one target test case that matches the test case information can be determined from multiple test cases. Each target test case is executed separately to obtain a test result file for each target test case. The test result file for each target test case can contain message data sent by the corresponding at least one service-based interface, and the message data can be a request message and / or a response message. The virtual network element testing method provided in the present application can determine the target test case based on the test case determination information in the test request, and execute each target test case separately to obtain the corresponding test result file, so as to realize convenient and fast automated testing of virtual network elements, reduce development costs, improve development efficiency, and promote the rapid iteration of 5G core network development.

[0173] In an embodiment of the present application, the test case determination information is at least one of the test case storage location information and the keyword. When the test case determination module determines at least one target test case matching the test case determination information from multiple test cases based on the test case determination information, it is specifically used to:

[0174] If the test case determination information is the test case storage location information, the test case corresponding to the test case storage path is used as the target test case;

[0175] If the test case determination information is a keyword, the keyword is matched with the test case description information of each test case in the preconfigured test cases, and each test case corresponding to the test case description information containing the keyword is determined as a target test case.

[0176] If the test case determination information includes test case storage location information and keywords, determine at least one candidate test case based on the test case storage location information, match the keywords with the test case description information of each candidate test case, and determine the candidate test case corresponding to the test case description information containing the keywords as the target test case.

[0177] In the embodiment of the present application, a test case configuration module is also included, which is specifically used to:

[0178] Receive test case configuration information through the test case configuration interface, generate and store test cases based on the test case configuration information;

[0179] For each test case, the test case configuration information includes a test case identifier and a test process file, wherein the test process file includes a test script and business parameters of at least one service-based interface corresponding to the test script.

[0180] In an embodiment of the present application, the test case configuration interface includes a virtual network element selection control, and the test case configuration module receives the test case configuration information through the test case configuration interface and generates a test case based on the test case configuration information, specifically for:

[0181] In response to a trigger operation on a virtual network element selection control, displaying a virtual network element list;

[0182] receiving a selection operation for a virtual network element in the virtual network element list, and determining the selected virtual network element as a target virtual network element;

[0183] Receive test case configuration information for the target virtual network element, and generate a test case corresponding to the target virtual network element based on the test case configuration information.

[0184] In an embodiment of the present application, the test case configuration interface includes a template selection control, and the test case configuration module receives the test case configuration information through the test case configuration interface and generates a test case based on the test case configuration information, specifically for:

[0185] In response to a trigger operation on a template selection control, displaying a template list;

[0186] receiving a selection operation for a test case template in the template list, and determining the selected test case template as a target template;

[0187] Receive test case configuration information based on the target template, and generate test cases according to the target template and the test case configuration information.

[0188] In the embodiment of the present application, the test case configuration information of each test case further includes at least one of the following:

[0189] The waiting time after any service-oriented interface corresponding to the test case is called;

[0190] Preconfigured interface call conditions for any service-based interface.

[0191] In the embodiment of the present application, a test result comparison module is also included, which is specifically used to:

[0192] For each target test case, the test result file is compared with the reference result file corresponding to the target test case to obtain a comparison result;

[0193] Generate a test report based on the test result files and comparison results of each target test case.

[0194] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0195] An electronic device (computer device / equipment / system) is provided in an embodiment of the present application, including a memory, a processor and a computer program stored in the memory. The processor executes the above-mentioned computer program to implement the steps of the virtual network element testing method. Compared with the related technology, it can achieve: convenient and fast automated testing of virtual network elements, improve testing efficiency, and better meet actual application needs.

[0196] In an alternative embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0197] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0198] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0199] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0200] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0201] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, notebook computers, PADs, etc. and fixed terminals such as digital TVs, desktop computers, etc.

[0202] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0203] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0204] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0205] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0206] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A virtual network element testing method, characterized in that: include: Receiving a test request for a virtual network element, wherein the test request includes test case determination information; Based on the test case determination information, determine at least one target test case matching the test case determination information from preconfigured test cases, each of the target test cases corresponding to at least one service-based interface, where the service-based interface is an interface between virtual network elements; Execute each of the target test cases respectively to obtain the test results of each of the target test cases; generate a test report based on the test results of each of the target test cases; The preconfigured test cases are obtained in the following way: Receiving test case configuration information through a test case configuration interface, generating and storing a test case based on the test case configuration information; wherein the test case configuration information is the source code of the test case or configuration information obtained by configuring through the test case interface; For each test case, the test case configuration information includes a test case identifier and a test process file, wherein the test process file includes a test script and business parameters of at least one service-based interface corresponding to the test script; The test case configuration interface includes a virtual network element selection control, and the test case configuration information is received through the test case configuration interface, and the test case is generated based on the test case configuration information, including: In response to a triggering operation on the virtual network element selection control, displaying a virtual network element list; Receiving a selection operation for a virtual network element in the virtual network element list, and determining a virtual network element selected from a preset virtual network element library as a target virtual network element; the virtual network element library includes a plurality of pre-configured virtual network elements, each of which includes corresponding basic configuration information; Receive test case configuration information for the target virtual network element, and generate a test case corresponding to the target virtual network element based on the test case configuration information; the test case configuration information includes service-oriented interface selection information and service-oriented interface configuration information; Before executing each of the target test cases respectively, the method further includes: Configure the business parameters of each service-oriented interface in the test case, and refer to the relevant data of each service-oriented interface through the identifier of each service-oriented interface; In the test case configuration interface, the execution order of each corresponding service-oriented interface is sorted based on the identifier of each service-oriented interface.

2. The virtual network element testing method according to claim 1, characterized in that: The test case determination information is at least one of test case storage location information and a keyword, and determining at least one target test case matching the test case determination information from a plurality of test cases based on the test case determination information includes: If the test case determination information is test case storage location information, taking the test case corresponding to the test case storage path as the target test case; If the test case determination information is a keyword, matching the keyword with the test case description information of each test case in the preconfigured test cases, and determining each test case corresponding to the test case description information containing the keyword as the target test case; If the test case determination information includes test case storage location information and keywords, at least one candidate test case is determined based on the test case storage location information, the keywords are matched with the test case description information of each candidate test case, and the candidate test case corresponding to the test case description information containing the keywords is determined as the target test case.

3. The method according to claim 1, characterized in that The test case configuration interface includes a template selection control, and the test case configuration information is received through the test case configuration interface, and the test case is generated based on the test case configuration information, including: In response to a trigger operation on the template selection control, displaying a template list; receiving a selection operation for a test case template in the template list, and determining the selected test case template as a target template; The test case configuration information is received based on the target template, and the test case is generated according to the target template and the test case configuration information.

4. The virtual network element testing method according to claim 1, characterized in that: The test case configuration information of each test case further includes at least one of the following: The waiting time after any service-oriented interface corresponding to the test case is called; The interface calling conditions preconfigured for any of the service-based interfaces.

5. The virtual network element testing method according to claim 1, characterized in that: The method further comprises: For each target test case, the test result file is compared with the reference result file corresponding to the target test case to obtain a comparison result; Generate a test report based on the test result files and comparison results of each target test case.

6. A virtual network element testing device, characterized in that: include: A test request receiving module, used to receive a test request for a virtual network element, wherein the test request includes test case determination information; A test case determination module, configured to determine, based on the test case determination information, at least one target test case that matches the test case determination information from preconfigured test cases, each of the target test cases corresponding to at least one service-based interface, the service-based interface being an interface between virtual network elements; A test case execution module, used to execute each of the target test cases respectively and obtain the test results of each of the target test cases; A test report generation module, used to generate a test report based on the test result statistics of each target test case; A test case configuration module, used to receive test case configuration information through a test case configuration interface, generate and store test cases based on the test case configuration information; wherein the test case configuration information is the source code of the test case or the configuration information obtained by configuring through the test case interface; For each test case, the test case configuration information includes a test case identifier and a test process file, wherein the test process file includes a test script and business parameters of at least one service-based interface corresponding to the test script; The test case configuration interface includes a virtual network element selection control, and the test case configuration module is specifically used to: In response to a triggering operation on the virtual network element selection control, displaying a virtual network element list; Receiving a selection operation for a virtual network element in the virtual network element list, and determining a virtual network element selected from a preset virtual network element library as a target virtual network element; the virtual network element library includes a plurality of pre-configured virtual network elements, each of which includes corresponding basic configuration information; Receive test case configuration information for the target virtual network element, and generate a test case corresponding to the target virtual network element based on the test case configuration information; the test case configuration information includes service-oriented interface selection information and service-oriented interface configuration information; An execution order determination module is used to configure the business parameters of each service-oriented interface in the test case, and to refer to the relevant data of each service-oriented interface through the identifier of each service-oriented interface; In the test case configuration interface, the execution order of each corresponding service-oriented interface is sorted based on the identifier of each service-oriented interface.

7. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the virtual network element testing method according to any one of claims 1 to 5 are implemented.

Images