Network management test method, network management device, and computer-readable storage medium

Abstract

The present application belongs to the technical field of communications. Provided are a network management test method, a network management device, and a computer-readable storage medium. The method comprises: acquiring a preset test model and a preset test unit, and acquiring an operation log of a user; processing the operation log according to the preset test model and the preset test unit, so as to obtain a test data file, wherein the test data file includes test data; and associating the test data file with a preset initial test case, so as to test network management.

Description

Network management testing method, network management device and computer readable storage medium

[0001] Cross-references

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on June 29, 2023, with application number 202310791699.7 and invention name “Network Management Testing Method, Network Management Equipment and Computer-readable Storage Medium”. The entire contents of the application are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of communication technology, and in particular to a network management testing method, a network management device, and a computer-readable storage medium. Background Art

[0004] In the communications field, network management (NM) is a software system responsible for remotely controlling communication transmissions between base stations and devices to ensure the normal operation of the entire communications system. It also monitors their behavior to ensure that one user equipment (UE) can communicate properly with other UEs. NM R&D efforts are focused on providing users with a simple and convenient system interface, clear business processing logic, and efficient and reliable operations.

[0005] During NMS development, test cases are typically managed based on the NMS's characteristics. These characteristics are linked to market demand, which is then further broken down into product requirements. Test cases are then designed based on these product requirements, generating a collection of test cases, known as a test case set. This set of test cases is then used to test and protect the NMS. If the NMS passes the test, the NMS is then used to manage the communications system. However, during NMS development, the NMS R&D team rarely has opportunities to communicate with live network users. Market demands for live network users are largely passed down by planners, resulting in a very limited and subjective understanding of live users' habits in NMS usage. Consequently, the lack of test case requirements analysis in the context of live network user scenarios further leads to the test case system either completely ignoring user acceptance testing, or test designers writing tests based on their own understanding, which deviates from actual user usage scenarios. This results in the NMS's core test points not being covered. Alternatively, a very small number of test designers consider the corresponding user scenarios but have no idea where to start writing test cases, or the test cases they produce are poorly executable and difficult to ensure effectiveness. Furthermore, user scenarios with multiple requirements are often omitted. Furthermore, the test case system typically has separate test case sets for different test dimensions / purposes, and the test cases and test data are coupled together. This results in a large number of test cases. As the NMS version evolves, requirements change, or capability indicators are adjusted, the NMS's test cases and automation need to be maintained simultaneously, resulting in a very high maintenance workload.

[0006] Therefore, in traditional network management R&D, it is impossible to efficiently construct and update network management test cases based on the actual situation of existing network users.

[0007] Summary of the Invention

[0008] The present application provides a network management testing method, a network management device, and a computer-readable storage medium.

[0009] In the first aspect, the present application provides a network management testing method, including: obtaining a preset test model and a preset test unit, and obtaining a user's operation log; processing the operation log according to the preset test model and the preset test unit to obtain a test data file; and associating the test data file with a preset initial test case to test the network management.

[0010] In a second aspect, the present application also provides a network management device, which includes a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of any network management testing method provided in the present application are implemented.

[0011] In a third aspect, the present application also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any network management testing method provided in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] FIG1 is a flow chart of a network management testing method according to an embodiment of the present application;

[0014] FIG2 is a schematic diagram of the system composition of the network management testing method provided in an embodiment of the present application;

[0015] FIG3 is a schematic diagram of a preset test model of a network management test method provided in an embodiment of the present application;

[0016] FIG4 is a schematic diagram of a network management test dimension framework of a network management test method provided in an embodiment of the present application;

[0017] FIG5 is a test logic diagram of a network management test method provided in an embodiment of the present application;

[0018] FIG6 is a schematic diagram of a test framework of a network management test method provided in an embodiment of the present application;

[0019] FIG7 is a schematic diagram of a first sub-flow of the network management testing method provided in an embodiment of the present application;

[0020] FIG8 is a schematic block diagram of the structure of a network management device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0023] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] The present invention provides a network management testing method, a network management device, and a computer-readable storage medium. The network management testing method can be applied to, but is not limited to, network management products and software systems that have complete user operation records and can obtain user operation logs, production environments that require environmental isolation and remote server deployment, and network management devices including but not limited to servers.

[0025] Faced with the problem that traditional technologies cannot efficiently construct and update network management test cases according to the actual situation of existing network users, the network management testing method proposed in the embodiment of the present application has the following core idea: based on the technical concept of decoupling test cases and test data, user behavior is abstracted to establish a test model, and according to the test model, the user's operation log is processed to obtain a test data file, and the initial test case is pre-constructed, that is, a preset initial test case is constructed, so that the preset initial test case and the test data file can be constructed and updated separately, and then the test data file is associated with the preset initial test case to test the network manager, which can realize testing of the network manager according to the actual situation of existing network users, and because the preset initial test case is decoupled from the test data file, the efficient construction and update of the network management test case can be realized.

[0026] The following embodiments of the present application are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0027] Please refer to Figures 1 to 6, wherein Figure 1 is a flow chart of a network management testing method provided by an embodiment of the present application. As shown in Figure 1, the method includes but is not limited to the following steps S11 to S13.

[0028] S11. Obtain a preset test model and a preset test unit, and obtain a user's operation log.

[0029] Explanatory, a pre-built test model, namely a preset test model, is constructed for network management products based on user behavior and the characteristics of the network management. The preset test model includes a preset N-tuple, which is a sequence of N parameters and can be represented by (X1, X2, ..., XN), where X1 is the first parameter, ..., XN is the Nth parameter, and N is a natural number. This closely integrates the preset test model with the user behavior of the network management. The preset test model, namely the user model (Persona), can also be called a persona or user role. It abstracts and models the actual situation of live network users based on the characteristics of the network management and user behavior. Different live network users can have different preset test models, thereby establishing a preset test model based on the user behavior of live network users, so that the preset test model can reflect the actual situation of live network users, and thus closely align the network management test with the actual situation of live network users. For example, please refer to Figure 3, which is a schematic diagram of a preset test model in the network management test method provided in an embodiment of the present application. In this example, within a period of time, several users performed a functional operation Y. The preset test model can use parameters containing the following four-tuple to describe the user operation Y = (U, O, F, D) = {number of users, number of operation objects, operation frequency, operation parameter}, where user operation Y represents the functional operation Y performed by the user, the number of users U represents the number of concurrent users who perform the functional operation during the period, the number of operation objects O represents the number of operation objects performed by the user, the operation frequency F represents the frequency of the functional operation, and the operation parameter D represents the parameter for performing the functional operation. It should be noted that the preset test model includes but is not limited to the above-mentioned four-tuple parameters. The preset test model can be set according to the actual situation of the user or the needs of the network management test. The preset test model can also be a preset N-tuple such as but not limited to a triple parameter, a five-tuple parameter, a sextuple parameter, etc.

[0030] According to the purpose or dimension of testing the network management, a test unit for testing the network management is pre-constructed, namely a preset test unit. The preset test unit is an independent test unit for testing the network management constructed to achieve the corresponding network management test purpose. The preset test unit represents a calling process of calling the corresponding preset initial test case based on the preset test model or an execution process of executing the preset initial test case. The preset test unit includes a test blueprint and parameter definitions for parameters included in the preset test model. The parameter definition is a method for determining parameter values. The specific values ​​corresponding to the parameters determined according to the parameter definition are test data, which are also the inputs of the preset initial test case. The test blueprint represents scheduling information for scheduling the preset initial test case based on the preset test model or a technical plan for using the preset test model and the preset initial test case. Different preset test units correspond to different test scenarios to achieve different test purposes. All preset test units constitute the test unit system of the embodiment of this application. The test unit system shown in Figure 2 includes preset test unit 1, preset test unit 2,..., preset test unit n. For example, please continue to refer to Figure 2. When the test case system of the network management includes functional testing, commercial stress playback testing, performance limit testing and reliability testing, and the above-mentioned four-tuple is used to describe the preset test model, the preset test unit corresponding to the functional test can be (1,1,1,*), where the number of users U is set to 1, the number of operation objects O is set to 1, the operation frequency F is set to 1, the operation parameters are represented by *, and are related to specific services. The specific values ​​of the operation parameters are defined by the specific services, so that the preset test unit (1,1,1,*) is used to perform functional testing on the network management, mainly to test whether the functions of the network management can operate normally, that is, to test whether the processing flow corresponding to the functions of the network management can be executed smoothly.

[0031] The user's operation log is a log that records the user's operation behavior in the communication system to which the network manager belongs. The network manager has a module for recording user operation events. Based on this module, the user's operation log can be collected and obtained. Therefore, the embodiment of the present application is applicable to communication systems including but not limited to those with complete user operation records. It should be noted that the embodiment of the present application is applicable to situations where the collection and desensitization of operation logs comply with the requirements of the corresponding information security protection laws of various countries or regions such as the EU's GDPR (General Data Protection Regulation) or China's "Personal Information Protection Law" after the logs are desensitized.

[0032] Based on the above settings, the preset test model and preset test unit are obtained, and the user's operation log is obtained. For example, please refer to Figure 2 again. The data lake can store the operation logs of different live network users. The test unit system contains different preset test units. The user's operation log, preset test model and preset test unit are obtained from the data lake, and then aggregated into the log analysis system.

[0033] S12: Process the operation log according to the preset test model and the preset test unit to obtain a test data file.

[0034] Explanatory note: Based on the preset test model, the operation log is first pre-processed to organize the operation log into pre-processed data corresponding to the data format of the preset test model. For example, when the preset test model is the aforementioned four-tuple, the operation log is first organized into a data format corresponding to the user, operation object, operation frequency, and operation parameter based on the number of users, number of operation objects, operation frequency, and operation parameters contained in the four-tuple.

[0035] Then, based on the preset test unit and the preprocessed data, the numerical values ​​required by the preset test unit are counted. The numerical values ​​required by the test unit are defined based on the preset test model corresponding to the quadruple, so as to process the operation log, and obtain the data that conforms to the preset test model and is required by the preset test unit from the operation log, that is, the test data, which can also be called test values. The test data represents the specific values ​​required for network management testing, and the test data is combined with the preset test unit to generate a test data file. The test data file contains a test blueprint and specific test values, that is, the test data file contains specific test data that drives the execution of the preset initial test case and the scheduling strategy information of different preset initial test cases. Therefore, the test data file is data determined according to the actual situation of the existing network users.

[0036] For example, please continue to refer to Figure 2. As shown in Figure 2, when the above-mentioned preset test model is described by a four-tuple and the preset test unit corresponding to the functional test is (1,1,1,*), that is, when the preset test unit defines the required test data based on the four-tuple, the operation log is processed to obtain the specific value of each * that conforms to the four-tuple from the operation log. That is, different services correspond to different values ​​of *, thereby obtaining (1,1,1,*) corresponding to different services, and generating a test data file based on the corresponding (1,1,1,*). Specifically, (1,1,1,*) is included in the test data file. At the same time, the test data file also includes the execution process or processing process of the network management test using the specific value corresponding to (1,1,1,*). Therefore, the preset test model clarifies that the operation logs stored in the data lake are pre-processed using the four-tuple and its corresponding parameters. That is, the preset test model clarifies how to perform preliminary parsing and processing on the operation logs stored in the data lake, and uses the preset test model and the preset test unit to process the operation logs to generate specific test data files and specific test cases.

[0037] S13: Associating the test data file with a preset initial test case to test the network management.

[0038] Explanatory speaking, the system test of network management products is to simulate the operation behavior of different users at the same time and verify whether each user gets the expected results. Based on this, when the preset test model is established based on user behavior, that is, the preset test model is used to describe the user's operation behavior, the test of network management products is to test according to the preset test model while paying attention to the execution results of the test and the impact of the execution on the network management. Therefore, the test cases for network management testing can describe the input according to the preset test model and add corresponding input verification to construct test cases for network management testing.

[0039] Based on the above concept, according to the purpose of testing the network management and the corresponding key functional operations, an initial test case is pre-constructed, that is, a preset initial test case. The preset initial test case is a test case that does not contain specific test values, and the preset initial test case is used to define how the functional operation and the preset test model work together to perform the test. The test data contained in the test data file is the input of the preset initial test case, and the output of the preset initial test case is the test result of testing the network management. The preset initial test case can be reused because it does not contain specific test values. Several preset initial test cases can form a test case library, so that multiple sets of test data can be combined with a set of test cases to complete network management tests with multiple test dimensions or test purposes, that is, the same preset initial test case can be combined with different test data to achieve network management tests with corresponding test dimensions or test purposes. For example, in the case where the above-mentioned preset test model is described by a four-tuple, the input of the preset initial test case is the above-mentioned four-tuple parameter description, and the value of the four-tuple parameter is the specific test data contained in the test data file.

[0040] When testing the network manager, the functional operation items (i.e., test blueprint) with arranged execution order contained in the test data file are associated and driven to the corresponding preset initial test cases in the test case library, and the corresponding test data is passed. The test data is a specific numerical value that conforms to the preset test model (including but not limited to the preset N-tuple), so that the network manager can be tested and the test results can be output, thereby associating the test data file with the preset initial test case to test the network manager.

[0041] The embodiment of the present application is based on the technical concept of testing the network manager by decoupling the preset initial test case from the test data file. By obtaining the preset test model and the preset test unit, and obtaining the user's operation log, the operation log is processed according to the preset test model and the preset test unit to obtain the test data file. The test data file contains test data, so as to obtain the test data according to the actual situation of the existing network users, and the test data file is associated with the preset initial test case to test the network manager. Since the network manager is tested according to the actual situation of the existing network users and the preset initial test case is decoupled from the test data file, the test case of the network manager can be efficiently constructed and updated, which is beneficial to the management and expansion of the test case, and can make the network manager that passes the test more consistent with the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0042] In one embodiment, please refer to Figure 7, which is a schematic diagram of the first sub-process of the network management testing method provided in an embodiment of the present application. As shown in Figure 7, in this embodiment, the operation log is processed according to the preset test model and the preset test unit to obtain a test data file, including: S71, pre-processing the operation log according to the preset N-tuple to obtain pre-processed data with a data format corresponding to the preset N-tuple; S72, obtaining the numerical value required by the preset test unit according to the pre-processed data, and assigning the numerical value to the preset test unit to obtain a test data file, wherein the preset test unit is constructed based on the preset N-tuple.

[0043] Explanatory note: The operation log is preprocessed based on a preset N-tuple, and the operation log is sorted into preprocessed data corresponding to the data format of the preset test model. For example, if the preset test model uses parameters containing the following four-tuple to describe user operations: {number of users, number of operation objects, operation frequency, operation parameter}, the operation log is first sorted into a data format corresponding to the users, operation objects, operation frequency, and operation parameters based on the four-tuple containing the number of users, number of operation objects, operation frequency, and operation parameters, thereby obtaining preprocessed data corresponding to the four-tuple.

[0044] Then, based on the preset test unit and the preprocessed data, the numerical values ​​required by the preset test unit are counted. The numerical values ​​required by the test unit are defined based on a preset test model corresponding to, but not limited to, a quaternary group. Thus, data based on the preset test model and required by the preset test unit, i.e., test data, is counted from the operation log. The test data represents the specific numerical values ​​required for network management testing. The test data is combined with the preset test unit to generate a test data file. Thus, the test data file is data determined based on the actual conditions of users in the existing network. For example, in the case where the preset test model is the aforementioned quaternary group, the numerical values ​​corresponding to the number of users, number of operation objects, operation frequency, and operation parameters contained in the operation log are counted to obtain the quaternary parameter value. The quaternary parameter value represents the specific numerical values ​​corresponding to the parameters "number of users, number of operation objects, operation frequency, and operation parameters" contained in the quaternary group. Then, since the preset test unit represents the execution process or calling process of the preset initial test case based on the quadruple, after obtaining the specific numerical value corresponding to the quadruple parameter, that is, after obtaining the quadruple parameter value, the quadruple parameter value is substituted into the preset test unit to obtain a test data file containing the specific test numerical value, thereby combining the quadruple parameter value with the preset test unit to obtain a test data file.

[0045] In an embodiment of the present application, the operation log is preprocessed according to a preset N-tuple to obtain preprocessed data with a data format corresponding to the preset N-tuple. Based on the preprocessed data, the numerical value required by the preset test unit is obtained, and the numerical value is assigned to the preset test unit to obtain a test data file, thereby realizing the technical concept of decoupling the preset initial test case from the test data file to obtain a test data file independent of the preset initial test case. The test data file is obtained based on the operation log, so that the test data file is obtained according to the actual situation of the existing network users, which can make the network management that passes the test more consistent with the actual situation of the existing network, thereby improving the effectiveness of the network management.

[0046] In one embodiment, a plurality of preset test units are included, and the preset test units include at least one of the following: a functional test unit, a commercial stress playback test unit, a performance limit test unit, and a reliability test unit.

[0047] For explanation, please refer to Figure 4, which is a schematic diagram of the network management test dimension framework of the network management test method provided in an embodiment of the present application. As shown in Figure 4, multiple preset test units can be used in the network management test, that is, the network management test contains several preset test units, and different preset test units realize corresponding test purposes and test functions, and form a test framework. Then, according to different test dimensions or test purposes, the corresponding preset initial test cases are called from the preset test case library, and the preset initial test cases are combined with the corresponding test data files to obtain different target test cases, and the target test cases are used to test the network management to achieve different test strategies and test purposes, and the preset initial test cases can be reused in different test schemes, that is, the same preset initial test case is called in different test data files, and then the test data file is determined by the preset test unit. Different test data files can call the same preset initial test case to achieve the corresponding test dimension or test purpose, and realize that multiple sets of test data are combined with a set of test cases to complete network management tests with multiple test dimensions or test purposes. Among them, the preset initial test cases include at least one of the following: functional test unit, performance efficiency test model, reliability test unit. Furthermore, the performance efficiency test model may include a commercial stress playback test unit and a performance limit test unit (i.e., business PT-performance limit test unit), which can improve the adaptability of network management testing to daily operation and maintenance or user operating habits in security scenarios, thereby improving the effectiveness of network management testing.

[0048] Furthermore, several test data files are used to perform network management tests according to a pre-set test logic, thereby achieving network management tests of different test dimensions or test purposes. Please continue to refer to Figure 5, which is a test logic diagram of the network management test method provided by an embodiment of the present application. As shown in Figure 5, a functional test unit is first used to test the function of the network management, and then a performance efficiency test model (commercial stress playback test, business PI test) is used to test the performance and efficiency of the network management. Finally, a reliability test unit is used to test the reliability of the network management, so as to achieve a complete set of tests on the network management that meet the corresponding test strategies and test purposes, which can improve the effectiveness of the network management test.

[0049] In one embodiment, the preset test unit is a functional test unit, and the preset N-tuple is an N-tuple in which all parameters except the operating parameter have a value of 1.

[0050] Explanatory, since the preset test unit determines the test data file, and the test data file determines the test purpose or test function, therefore, when the preset test unit is a functional test unit and the preset test model is a preset N-tuple, all parameters in the preset N-tuple except the operation parameters are set to 1, and the preset N-tuple after the value is set to 1 is used as the preset test unit corresponding to the functional test unit. Exemplarily, as described above, when the preset test model is a four-tuple {number of users, number of operation objects, operation frequency, operation parameter}, the number of users, number of operation objects, and operation frequency are all set to 1, resulting in {1, 1, 1, operation parameter}, where the operation parameter can be described by *, and the operation parameter is related to the specific business, thereby obtaining (1, 1, 1, *), and (1, 1, 1, *) is used as the preset test unit corresponding to the functional test.

[0051] In the embodiment of the present application, when the preset test unit is a functional test unit, the preset test unit is set to a preset N-tuple obtained by taking the value of 1 for all parameters contained in the preset test model except the operation parameters. This not only configures the functional test based on the preset test model of user behavior, thereby performing functional testing on the network manager according to the actual situation of existing network users, but also focuses on testing whether the corresponding functions of the network manager are normal from a functional perspective, so as to make the test of the network manager targeted and single-purpose as much as possible without involving other factors, avoid complicating the test of the network manager, and when problems occur during the test process, timely locate and maintain the problems, thereby improving the efficiency of the network management test.

[0052] In one embodiment, the preset test unit is the commercial stress playback test unit, and the preset test model is a preset N-tuple, the preset N-tuple includes at least one of the following: number of users, number of operation objects, operation frequency, and operation parameters, and the preset test unit is defined corresponding to the preset N-tuple as follows: the number of users is the number of users who initiate operations on a single functional operation within a preset time granularity; the number of operation objects is the average of the sum of the number of objects operated by all users within the preset time granularity for a single functional operation; the operation frequency is the average of the sum of the operation frequencies of all users within the preset time granularity for a single functional operation; the operation parameter is the operation content corresponding to the user within the preset time granularity for a single functional operation.

[0053] Explanatory, please continue to refer to Figure 4. As shown in Figure 4, the preset test unit is a commercial stress playback test unit. The commercial stress playback test unit represents the simulation of actual commercial business (i.e., operation log) based on the production environment, so as to test the network management in the form of replaying the actual commercial business. The commercial stress playback test unit belongs to the performance efficiency test. In the case where the preset test model is a preset N-tuple, the preset N-tuple may include at least one of the following: the number of users, the number of operation objects, the operation frequency, and the operation parameters. Since the preset test unit is constructed based on the preset test model, based on the above-mentioned preset N-tuple, under the concept of designing the parameter values ​​contained in the preset N-tuple of each functional operation to be positioned at the average level, the preset test unit corresponding to the preset N-tuple is defined as follows.

[0054] 1) The number of users refers to the number of users who initiate operations on a single functional operation within a preset time granularity. The number of users represents the number of concurrent users who initiate operations on the same functional operation within the preset time granularity. A large number of users indicates high concurrency, while a small number of users indicates low concurrency. For example, to determine the corresponding number of users for a single functional operation, with a preset time granularity including but not limited to 15 minutes, for a single functional operation A, the number of users who initiate operations on functional operation A every 15 minutes is counted. The number of users represents the number of concurrent users who initiate operations on functional operation A every 15 minutes.

[0055] 2) The number of objects operated is the average of the sum of the number of objects operated by all users within the preset time granularity for a single function operation. That is, for a single function operation, the ratio of the sum of the number of objects operated by all users within the preset time granularity is calculated to the total number of users. For example, for function operation A, which includes several objects, within a 15-minute time granularity, if user A operates 20 objects, user B operates 30 objects, and user C operates 10 objects, then the number of users = (20 + 30 + 10) / 3 = 20.

[0056] 3) Operation frequency is the average of the sum of the operation frequencies of all users within a preset time granularity for a single function operation. That is, for a single function operation, the ratio of the sum of the operation frequencies of all users within the preset time granularity is calculated to the total number of users. The operation frequency represents the frequency and number of times a user performs an operation within the preset time granularity. For example, for function operation A, within a time granularity of 15 minutes, if user A performs the operation 15 times, user B performs the operation 30 times, and user C performs the operation 15 times, then the operation frequency = (15 + 30 + 15) / 3 = 20.

[0057] 4) Operation parameters refer to the operation content corresponding to the user within the preset time granularity for a single functional operation. The operation content can also be called operation details. For example, when the functional operation is an information interaction operation, the operation content can be specific communication methods such as video, voice, and text. The operation parameters will be extracted from the operation log according to the specific business corresponding to the functional operation.

[0058] The embodiment of the present application defines corresponding preset test units based on the preset N-tuple corresponding to the preset test model, thereby realizing the construction of corresponding preset test units based on user behavior, and then performing commercial stress playback testing on the network manager based on the actual situation of the existing network users. The network manager that passes the test can be more closely matched with the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0059] Furthermore, the preset test unit includes at least one of the following: a daily playback test unit, a weekly playback test unit, and a nighttime playback test unit; wherein the daily playback test unit is a test strategy for performing commercial stress playback testing on network managers based on the preset test unit and taking each day as the time unit; the weekly playback test unit is a test strategy for performing commercial stress playback testing on network managers based on the preset test unit and taking each week as the time unit; the nighttime playback test unit is a test strategy for performing commercial stress playback testing on network managers based on the preset test unit and taking the preset time period at night every day as the time unit.

[0060] Explanatoryally, the embodiments of the present application design different test time dimensions, and design corresponding playback test units from the three dimensions of day, week, and night, namely, the day-level playback test unit, the weekly playback test unit, and the night-level playback test unit, so as to test the network manager from different dimensions, and the different dimensions can be combined to test the network manager, that is, the preset test unit includes at least one of the following: the day-level playback test unit, the weekly playback test unit, and the night-level playback test unit; wherein, the day-level playback test unit is a test strategy for performing commercial stress playback testing on the network manager based on the preset test unit with each day as the time unit; the weekly playback test unit is a test strategy for performing commercial stress playback testing on the network manager based on the preset test unit with each week as the time unit. Due to the weekly time characteristics of the network manager, the weekly playback test unit is used to test the network manager, which can make the network manager that passes the test more consistent with the actual situation of the existing network, thereby improving the effectiveness of the network manager test; the night-time playback test unit is a test strategy for performing commercial stress playback testing on the network manager based on the preset test unit with the preset time period of each night as the time unit.

[0061] Exemplarily, based on the definition of the above-mentioned preset test units corresponding to the preset N-tuples, the daily playback test unit, the weekly playback test unit, and the nighttime playback test unit are respectively described as follows.

[0062] 1) For the daily playback test unit, first design a commercial stress playback test unit, then extract 7 busy hours per day according to the on-site operation and maintenance frequency, and determine the 7 busy hours as 9:00-12:00 and 14:00-18:00 on each working day. Then, for the playback data of each busy hour, the 15-minute time granularity with the largest number of users within the busy hour can be selected for data equivalence, that is, the four-tuple parameter value of each functional operation within each busy hour is the value corresponding to the 15-minute time granularity with the largest number of users for the corresponding functional operation within this busy hour. Finally, the four-tuple parameter value of the 7 busy hours is selected for the daily playback. For any busy hour, the busy hour with the largest number of function operation items among the busy hours on different days is selected as the busy hour data for daily playback. For example, for daily playback of the busy hour of 9:00-10:00, the entire log time range is 3 days. There are 15 function operation items from 9:00-10:00 on the first day, 14 function operation items from 9:00-10:00 on the second day, and 17 function operation items from 9:00-10:00 on the third day. Therefore, the four-tuple parameter value of the busy hour of 9:00-10:00 for daily playback is the four-tuple parameter value of 9:00-10:00 on the third day.

[0063] 2) For the weekly playback test unit, the goal is to replay the operational behavior of the live network for a consecutive week. The corresponding playback model is based on the commercial stress playback test unit corresponding to the daily playback test unit, and is expanded to replay the live network data from Monday to Friday. Playback can also be performed according to the seven busy hours from Monday to Friday, but the daily playback data will be selected within the same day of the week within the log range. For example, the playback data for Monday 9:00-10:00 is selected based on the four-tuple parameter data of 9:00-10:00 with the largest number of functional operation items among all Monday 9:00-10:00 within the log range. The same applies from Tuesday to Friday. In this way, weekly playback data is generated. Due to the weekly time characteristics of network management, using the weekly playback test unit to test network management can make the network management that passes the test more closely match the actual situation of the live network, thereby improving the effectiveness of network management testing.

[0064] 3) For the night-time playback test unit, since major operations such as upgrades and capacity expansions may occur at night, the six busy hours of 21:00-24:00 and 00:00-3:00 can be selected. The value strategy for each busy hour can refer to the value corresponding to the daily playback test unit.

[0065] The embodiment of the present application designs different test time dimensions, designs corresponding playback test units from the three dimensions of day, week, and night, and can combine different dimensions to test the network manager. It can test the network manager according to the actual situation of the existing network users at different times, and can make the network manager that passes the test more closely match the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0066] In one embodiment, the preset initial test case is the performance limit test unit, and the preset initial test case includes at least one of the following: the preset initial test case includes all functional operations and the corresponding preset test models; the preset initial test case includes all functional operations corresponding to the functions operated by the user within the preset time granularity and the corresponding preset test models; the preset initial test case includes all functional operations that depend on the same preset service and the corresponding preset test models.

[0067] For illustrative purposes, please refer to Figure 4. As shown in Figure 4, the preset initial test case is a performance limit test unit. This unit represents a model for testing the performance limits of the network management system and belongs to the performance efficiency test. For performance limit testing, layered testing can be employed, combining different levels of testing. Specifically, the preset initial test case includes at least one of the following:

[0068] 1) The preset initial test case includes all functional operations and the corresponding preset test models, that is, all functional operations are arranged into test files. Each test file consists of different functional operations and the corresponding preset N-tuple parameter values. Different functional operations can be executed sequentially.

[0069] 2) The preset initial test case includes all functional operations corresponding to the functions performed by users within the preset time granularity and the corresponding preset test models, that is, all functional operations corresponding to the functions performed by users within the time granularity including but not limited to 15 minutes and the corresponding preset N-tuple parameter values ​​are selected, and different functional operations can be executed concurrently.

[0070] 3) The preset initial test case includes all functional operations that depend on the same preset service and the corresponding preset test models. The test file can be compiled based on the list L of functional operations that depend on a certain service. The content of the test file consists of L and the corresponding preset N-tuple parameter values. Different functional operations can be executed concurrently.

[0071] In the embodiment of the present application, the performance limit test is designed as different layered tests, and different layered tests can be combined to test the network manager. The performance limit test of the network manager can be performed according to the actual situation of different performance angles of the existing network users, and the network manager that passes the test can be more closely matched with the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0072] Further, please continue to refer to Figure 5. The preset test model is a preset N-tuple, and the preset N-tuple includes at least one of the following items: the number of users, the number of operation objects, the operation frequency, and the operation parameter, and the preset test unit corresponding to the preset N-tuple is defined as follows: the number of users is the maximum value of the concurrent number of users in all time periods determined according to the preset time granularity for a single functional operation within the scope of the operation log; the number of operation objects is the maximum value of the sum of all objects operated by a single user for a single functional operation within the scope of the operation log; the operation frequency is 1; and the operation parameter is the operation content of the user corresponding to the number of users.

[0073] Explanatory note: when the preset test model is a preset N-tuple, the preset N-tuple includes at least one of the following: number of users, number of operation objects, operation frequency, and operation parameters. Because a preset test unit represents the execution process or specific usage process of the preset test model, based on the aforementioned preset N-tuple, and with the parameter values ​​contained in the preset N-tuple for each functional operation positioned at maximum levels, for performance limit testing, the preset test units corresponding to the preset N-tuple are defined as follows.

[0074] 1) The number of users is the maximum number of concurrent users for a single functional operation within the scope of the operation log, determined according to the preset time granularity. For example, with a time granularity of 15 minutes, within the time range defined by the operation log, for a single functional operation A, if the number of users performing functional operation A in one 15-minute period is M, and the number of users performing functional operation A in each of the remaining 15-minute periods can be uniformly represented by N, and N < M, then M is determined as the number of users.

[0075] 2) The number of objects operated is the maximum value of the sum of all objects operated by a single user within the scope of the operation log for a single functional operation. For example, within the data range defined by the operation log, for a single functional operation A, if user B1 operates on M = object m1 + object m2 + ... + object mm, and other users B2, ..., Bn operate on N = object n1 + object n2 + ... + object nn, where N < M, then M is determined as the number of objects operated.

[0076] 3) The operation frequency is set to 1. In the performance limit test, the operation frequency is fixed to 1.

[0077] 4) The operation parameter is the operation content of the user corresponding to the user number. The operation content can also be called operation details. For example, when the functional operation is an information interaction operation, the operation content can be a specific communication method such as video, voice, and text. The operation parameters will be extracted from the operation log according to the specific business corresponding to the functional operation.

[0078] The embodiment of the present application sets corresponding preset test units according to the preset N-tuple corresponding to the preset test model, and positions the parameter values ​​contained in the preset N-tuple of each functional operation at the maximum value level, thereby setting corresponding preset test units based on user behavior, and then performing extreme performance testing on the network manager based on the actual situation of the existing network users. The network manager that passes the test can be more closely matched with the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0079] In one embodiment, after the test data file is associated with a preset initial test case to test the network manager, the method further includes: obtaining a preset scale mapping table and a target scale; adjusting scale-related values ​​contained in the test data file according to the preset scale mapping table and the target scale; and testing the network manager using the adjusted test data file.

[0080] Explanatoryally, a scale mapping table, i.e., a preset scale mapping table, is pre-constructed. The preset scale mapping table describes an adjustment mapping relationship for scaling up or down the scale-related values ​​in the test data file based on a comparison between the target scale and the current scale, based on a test data file obtained from the current operation log as a benchmark test file. The benchmark test file represents a file corresponding to a commonly used scale under normal circumstances. For example, assuming the current scale is A, the target scale B for the test is first determined. If A>B, the scale-related values ​​in the test data file are scaled down. If A<B, the scale-related values ​​in the test data file are scaled up. The preset scale mapping table describes a mapping relationship for scaling up or down the scale-related values ​​in the test data file under different scale conditions, where the target scale B is used. The mapping relationship may include scaling up or down, as well as the magnitude or degree of scaling up or down.

[0081] Based on the construction of the above-mentioned preset scale mapping table, when the network manager faces larger or other scale operation and maintenance requirements and needs to be retested, the target scale is first determined and the preset scale mapping table is obtained. According to the preset scale mapping table and the target scale, the scale-related values ​​contained in the test data file are adjusted, and the network manager is tested using the adjusted test data file.

[0082] In an embodiment of the present application, a preset scale mapping table and a target scale are obtained, and scale-related values ​​contained in a test data file are adjusted according to the preset scale mapping table and the target scale. The adjusted test data file is used to test the network management. When the scale faced by the network management changes, test data of any scale can be adaptively obtained, thereby improving the effectiveness and efficiency of the network management test.

[0083] In one embodiment, after processing the operation log, it includes: processing the operation log to obtain the user function operation corresponding to the user operation; comparing the user function operation with the preset initial test case contained in the preset test case library; if the preset initial test case corresponding to the user function operation does not exist, issuing a prompt that the preset initial test case is missing.

[0084] Explanatoryally, please continue to refer to Figures 2 and 6 to process the operation log. This step can be the same step as processing the operation log to obtain the test data file, and obtain the user function operation corresponding to the user's operation. The user function operation is the output of the log analysis system. When the preset test model is a preset N-tuple, the user function operation is the preset N-tuple corresponding to the preset test model. Multiple user function operations can be presented in a function operation list and can be included in the atomic operation analysis report, where the atomic operation analysis report is a file generated by processing the operation log to analyze the user's operation.

[0085] Pre-build several preset initial test cases, and organize all the preset initial test cases into a test case library, namely the preset test case library. Please refer to the preset test case library in Figure 6. The preset test case library generally includes multiple preset initial test cases. Then, compare the user function operation with the different preset initial test cases contained in the preset test case library. The operation item identifier corresponding to the user function operation, including but not limited to the operation item name, can be compared with the operation item identifier corresponding to the preset initial test case contained in the preset test case library, including but not limited to the operation item name. This includes but is not limited to judging whether there is a preset initial test case corresponding to the user function operation through the preset test case library, and in the case that there is no preset initial test case corresponding to the user function operation, issuing a prompt that the preset initial test case is missing, thereby prompting relevant personnel to add the corresponding preset initial test case.

[0086] In an embodiment of the present application, the operation log is processed to obtain the user function operation corresponding to the user's operation, and the user function operation is compared with different preset initial test cases. If there is no preset initial test case corresponding to the user function operation, a prompt is issued that the preset initial test case is missing. The preset initial test case can be improved according to the actual situation of the users in the existing network, so that the network management that passes the test is more consistent with the actual situation of the existing network, thereby improving the effectiveness of the network management test.

[0087] In one embodiment, the test data file is associated with a preset initial test case to test the network manager, including: calling the preset initial test case from the preset test case library according to the functional operation items with arranged execution order contained in the test data file; inputting the test data contained in the test data file into the preset initial test case to test the network manager.

[0088] Explanatory, according to the functional operation items with arranged execution order contained in the test data file, the preset initial test case is called from the preset test case library, and the same preset initial test case in the preset test case library can be called by different test data files to achieve different test purposes or test functions, so as to realize the reuse of the same preset initial test case, and then the test data contained in the test data file is input into the preset initial test case, so as to associate the decoupled test data file with the preset initial test case, and pass the corresponding test data to the preset initial test case. When the preset test model is a preset N-tuple, the test data is the value of the preset N-tuple, to test the network manager, and after completing the test of the network manager, the corresponding test result is output.

[0089] The embodiment of the present application tests the network manager by calling the preset initial test case from the preset test case library according to the functional operation items with arranged execution order contained in the test data file, and inputting the test data contained in the test data file into the preset initial test case, thereby realizing the technical concept of decoupling the preset initial test case from the test data file to test the network manager, which can realize the efficient construction and update of the test case of the network manager, is beneficial to the management and expansion of the test case, and can improve the flexibility of the network manager test.

[0090] Please refer to Figure 8, which is a schematic block diagram of the structure of a network management device provided in an embodiment of the present application. As shown in Figure 8, the network management device 300 includes a processor 301 and a memory 302, and the processor 301 and the memory 302 are connected via a bus 303, such as an I2C (Inter-Integrated Circuit) bus.

[0091] Specifically, processor 301 is used to provide computing and control capabilities to support the operation of the entire network management device. Processor 301 can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0092] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0093] Those skilled in the art will understand that the structure shown in Figure 8 is merely a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the network management device to which the embodiment of the present application is applied. The specific server may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0094] The processor is configured to run a computer program stored in a memory, and implement any one of the network management testing methods provided in the embodiments of the present application when executing the computer program.

[0095] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the network management device described above can refer to the corresponding process in the aforementioned network management test method embodiment, and will not be repeated here.

[0096] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any network management testing method provided in the description of the embodiment of the present application.

[0097] The storage medium may be an internal storage unit of the network management device described in the aforementioned embodiment, such as a memory of the network management device. The storage medium may also be an external storage device of the network management device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the network management device.

[0098] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0099] It should be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0100] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection of the claims.

Claims

1. A network management testing method, comprising: Obtain the preset test model and preset test unit, and obtain the user's operation log; Processing the operation log according to the preset test model and the preset test unit to obtain a test data file; The test data file is associated with a preset initial test case to test the network management.

2. The network management testing method according to claim 1, wherein: The preset test model includes a preset N-tuple, and the preset N-tuple includes at least one of the following: the number of users, the number of operation objects, the operation frequency, and the operation parameter, wherein N is a natural number.

3. The network management testing method according to claim 2, wherein: According to the preset test model and the preset test unit, the operation log is processed to obtain a test data file, including: Preprocessing the operation log according to the preset N-tuple to obtain preprocessed data whose data format corresponds to the preset N-tuple; According to the preprocessed data, a numerical value required by the preset test unit is obtained, and the numerical value is assigned to the preset test unit to obtain a test data file, wherein the preset test unit is constructed based on the preset N-tuple.

4. The network management testing method according to claim 3, wherein: It comprises a plurality of the preset test units, and the preset test units include at least one of the following: a functional test unit, a commercial pressure playback test unit, a performance limit test unit, and a reliability test unit.

5. The network management testing method according to claim 4, wherein: The preset test unit is the functional test unit, and the preset N-tuple is an N-tuple in which all parameters except the operation parameters are 1.

6. The network management testing method according to claim 4, wherein: The preset test unit is the commercial pressure playback test unit, and the preset test unit corresponds to the preset N-tuple and is defined as follows: The number of users is the number of users who initiate operations on a single functional operation within a preset time granularity; The number of operation objects is the average of the sum of the numbers of objects operated by all users within a preset time granularity for a single function operation; The operation frequency is the average of the sum of the operation frequencies of all users within a preset time granularity for a single function operation; The operation parameter is the operation content corresponding to the user within a preset time granularity for a single functional operation.

7. The network management testing method according to any one of claims 4 to 6, wherein: The preset test unit is the performance limit test unit, and the preset test unit includes at least one of the following: The preset test unit includes all functional operations and corresponding preset test models; The preset test unit includes all functional operations corresponding to the functions operated by the user within the preset time granularity and the corresponding preset test models; The preset test unit includes all functional operations that depend on the same preset service and the corresponding preset test models.

8. The network management testing method according to claim 7, wherein: The preset test unit corresponding to the preset N-tuple is defined as follows: The number of users is the maximum value of the number of concurrent users in all time periods determined according to a preset time granularity within the scope of the operation log for a single functional operation; The number of operation objects is the maximum value of the sum of all objects operated by a single user within the scope of the operation log for a single function operation; The operation frequency is 1; The operation parameter is the operation content of the user corresponding to the user number.

9. The network management testing method according to claim 1, wherein: After the test data file is associated with a preset initial test case to test the network management, the method further includes: Get the preset scale mapping table and the target scale; According to the preset scale mapping table and the target scale, adjusting the scale-related values ​​contained in the test data file; The network management is tested using the adjusted test data file.

10. The network management testing method according to claim 1, wherein: Processing the operation log further includes: Processing the operation log to obtain the user function operation corresponding to the user operation; Compare the user function operation with the preset initial test cases contained in the preset test case library; In the case where there is no preset initial test case corresponding to the user function operation, a prompt indicating that the preset initial test case is missing is issued.

11. The network management testing method according to claim 1, wherein: The test data file is associated with a preset initial test case to test the network management, including: According to the functional operation items in the test data file and arranged in execution order, calling the preset initial test case from the preset test case library; The test data contained in the test data file is input into the preset initial test case to test the network management.

12. A network management device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of the network management testing method as described in any one of claims 1 to 11 are implemented.

13. A computer-readable storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the network management testing method according to any one of claims 1 to 11.

Images