Network management test method, network management equipment and computer readable storage medium

Abstract

The invention provides a network management test method, network management equipment and a computer readable storage medium, and belongs to the technical field of communication. In order to solve the problem that network management test cases cannot be constructed and updated according to actual conditions of current network users in the prior art, based on the technical concept of testing network management by decoupling a preset initial test case and a test data file, the following technical scheme is adopted: obtaining a preset test model and a preset test unit, the operation log of the user is obtained, the operation log is processed according to the preset test model and the preset test unit, the test data file is obtained, the test data file comprises the test data, and the test data is obtained according to the actual situation of the current network user; and the test data file is associated with the preset initial test case to test the webmaster, so that efficient construction and updating of the test case of the webmaster can be realized, and the effect of testing the webmaster is improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a network management testing method, a network management device and a computer-readable storage medium. Background Art

[0002] In the field of communications, network management is a software system that is responsible for remotely controlling the communication transmission between base stations and devices to ensure that the entire communication system can operate normally and monitor its behavior to ensure that a UE can communicate normally with other UEs. The research and development responsibility of the network management is to provide users who use the network management with a simple and convenient system interface, clear business processing logic and efficient and reliable operation.

[0003] In the development of network management, the test cases of network management are generally managed according to the characteristics of network management. The characteristics of network management are related to market demand, and product demand is further divided under market demand. Then, test cases are designed based on product demand, and a collection of test cases, namely test case sets, is generated. The test case sets are used to test and protect the network management. If the network management test passes, the network management is used to manage the communication system. However, in the development of network management, there are very few opportunities for communication between the network management development team and the existing network users. The market demand of the existing network users is basically passed down by the planning personnel, which leads to the R&D team's understanding of the existing network users' habits in using network management. This is very limited and subjective. Therefore, the lack of demand analysis of test cases in the dimension of existing network user scenarios further leads to the fact that the user acceptance angle test in the test case system is either completely ignored, or the test designers write according to their own understanding, which deviates from the actual user usage scenario, resulting in the core test points of the network management not being covered, or a very small number of test designers consider the corresponding user scenarios, but have no idea how to write test cases or the output test cases are poorly executable and difficult to ensure effectiveness. Among them, the user scenarios with multiple requirements superimposed are basically omitted. At the same time, the test case system generally corresponds to a set of test cases for different test dimensions / purposes, and the test cases and test data are coupled together, which leads to a large number of test cases. With the evolution of the network management version, changes in requirements or adjustments to capability indicators, the network management test cases and automation need to be maintained synchronously, and the maintenance workload is very large.

[0004] Therefore, in traditional network management research and development, it is impossible to efficiently build and update network management test cases based on the actual situation of existing network users. Summary of the invention

[0005] The present invention provides a network management test method, a network management device and a computer-readable storage medium, which can efficiently construct and update the test cases of the network management according to the actual situation of the existing network users.

[0006] In a first aspect, the present invention provides a network management testing method, comprising: obtaining a preset test model and a preset test unit, and obtaining a user's operation log; processing the operation log according to the preset test model and the preset test unit to obtain a test data file; and associating the test data file with a preset initial test case to test the network management.

[0007] In a second aspect, the present invention also provides a network management device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of any network management testing method provided by the present invention are implemented.

[0008] In a third aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any network management testing method provided by the present invention.

[0009] The present invention provides a network management test method, a network management device and a computer-readable storage medium. The technical concept of testing the network management based on the decoupling of the preset initial test case and the test data file is that the method obtains the preset test model and the preset test unit, obtains the user's operation log, processes the operation log according to the preset test model and the preset test unit, obtains the test data file, and the test data file contains the test data, thereby obtaining the test data according to the actual situation of the existing network user, and the test data file is associated with the preset initial test case to test the network management. Since the network management is tested according to the actual situation of the existing network users, and the preset initial test case is decoupled from the test data file, the test case of the network management can be efficiently constructed and updated, which is conducive to the management and expansion of the test case, and can make the network management that passes the test more matched with the actual situation of the existing network, thereby improving the effectiveness of the network management test. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A schematic diagram of a flow chart of a network management testing method provided by an embodiment of the present invention;

[0012] Figure 2A schematic diagram of the system composition of the network management testing method provided by an embodiment of the present invention;

[0013] Figure 3 A schematic diagram of a preset test model of a network management test method provided in an embodiment of the present invention;

[0014] Figure 4 A schematic diagram of a network management test dimension framework of a network management test method provided in an embodiment of the present invention;

[0015] Figure 5 A test logic diagram of a network management test method provided by an embodiment of the present invention;

[0016] Figure 6 A schematic diagram of a test framework of a network management test method provided by an embodiment of the present invention;

[0017] Figure 7 A schematic diagram of a first sub-flow chart of a network management testing method provided by an embodiment of the present invention;

[0018] Figure 8 A schematic block diagram of the structure of a network management device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0021] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0022] The embodiment of the present invention provides a network management test method, a network management device and a computer-readable storage medium. The network management test method can be applied to, but not limited to, network management products and software systems with complete user operation records and capable of obtaining user operation record logs, production environments for environmental isolation and remote deployment of servers, and can be applied to network management devices including but not limited to servers.

[0023] In the face of the problem that traditional technologies cannot efficiently construct and update network management test cases according to the actual situation of existing network users, the inventors propose a network management testing method according to an embodiment of the present invention. The core idea of ​​the embodiment of the present invention is: based on the technical concept of decoupling test cases from test data, user behaviors are abstracted to establish a test model, and according to the test model, the user's operation log is processed to obtain a test data file, and an initial test case is pre-constructed, that is, a preset initial test case is constructed, so that the preset initial test case and the test data file can be separately constructed and updated, and then the test data file is associated with the preset initial test case to test the network manager, which can realize testing of the network manager according to the actual situation of existing network users, and because the preset initial test case is decoupled from the test data file, the efficient construction and update of the network manager's test case can be realized.

[0024] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0025] See also Figures 1 to 6 ,in, Figure 1 The flowchart of the network management testing method provided by the embodiment of the present invention is as follows. Figure 1 As shown, the method includes but is not limited to the following steps S11 to S13:

[0026] S11. Obtain a preset test model and a preset test unit, and obtain a user's operation log.

[0027] Explanatory, a test model is pre-built, i.e., a preset test model. For network management products, based on user behavior and network management characteristics, the preset test model includes a preset N-tuple, where the N-tuple is a sequence containing N parameters, which can be (X1, X2, ..., X N ), where X1 is the first parameter, …, X Nis the Nth parameter, N is a natural number, thereby closely combining the preset test model with the user behavior of the network manager. The preset test model, i.e., the user model (Persona), can also be called a persona or user role. It abstracts and models the actual situation of the existing network users based on the characteristics of the network manager and the characteristics of the user behavior. Different existing network users may have different preset test models, thereby establishing a preset test model based on the user behavior of the existing network users, so that the preset test model can reflect the actual situation of the existing network users, and thus make the network manager's test closely fit the actual situation of the existing network users. For example, please refer to Figure 3 , Figure 3 A schematic diagram of a preset test model in a network management test method provided in an embodiment of the present invention. In this example, within a period of time, several users perform a functional operation Y. The preset test model can use parameters containing the following four-tuple to describe the user operation Y = (U, O, F, D) = {number of users, number of operation objects, operation frequency, operation parameter}, wherein user operation Y represents the functional operation Y performed by the user, the number of users U represents the number of concurrent users who perform the functional operation within the period of time, the number of operation objects O represents the number of operation objects performed by the user, the operation frequency F represents the frequency of the functional operation, and the operation parameter D represents the parameter for performing the functional operation. It should be noted that the preset test model includes but is not limited to the above-mentioned four-tuple parameters, and the preset test model can be set according to the actual situation of the user or the needs of the network management test. The preset test model can also be a preset N-tuple such as but not limited to a triple parameter, a quintuple parameter, a sextuple parameter, etc.

[0028] According to the purpose or dimension of testing the network manager, a test unit for testing the network manager is pre-constructed, that is, a preset test unit. The preset test unit is an independent test unit for testing the network manager constructed to achieve the corresponding network manager testing purpose. The preset test unit represents a calling process of calling the corresponding preset initial test case based on the preset test model or an execution process of executing the preset initial test case. The preset test unit includes a test blueprint and a parameter definition of the parameters included in the preset test model. The parameter definition is a method for determining the parameter value. The specific value corresponding to the parameter determined according to the parameter definition is the test data, which is also the input of the preset initial test case. The test blueprint represents the scheduling information for scheduling the preset initial test case based on the preset test model or the technical planning of using the preset test model and the preset initial test case. Different preset test units correspond to different test scenarios to achieve different test purposes. All preset test units constitute the test unit system of an embodiment of the present invention, such as Figure 2 The test unit system shown includes a preset test unit 1, a preset test unit 2, ..., a preset test unit n. For example, please continue to refer to Figure 2, when the test case system of the network management includes functional testing, commercial stress playback testing, performance limit testing and reliability testing, and the above-mentioned four-tuple is used to describe the preset test model, the preset test unit corresponding to the functional test can be (1,1,1,*), wherein the number of users U is set to 1, the number of operation objects O is set to 1, the operation frequency F is set to 1, the operation parameter is represented by *, and is related to the specific business. The specific value of the operation parameter is defined by the specific business, so that the preset test unit (1,1,1,*) is used to perform functional testing on the network management, mainly to test whether the functions of the network management can operate normally, that is, to test whether the processing flow corresponding to the functions of the network management can be smoothly executed.

[0029] The user's operation log is a log that records the user's operation behavior in the communication system to which the network manager belongs. The network manager has a module for recording user operation events. Based on this module, the user's operation log can be collected and obtained. Therefore, the embodiment of the present invention is applicable to communication systems including but not limited to those with complete user operation records. It should be noted that the embodiment of the present invention is applicable to the collection of operation logs and the desensitization of logs in compliance with the requirements of the corresponding information security protection laws of various countries or regions such as the EU's GDPR (General Data Protection Regulation) or China's "Personal Information Protection Law".

[0030] Based on the above settings, obtain the preset test model and preset test unit, and obtain the user's operation log. For example, please continue to refer to Figure 2 In the data lake, the operation logs of different existing network users can be stored. The test unit system contains different preset test units. The user's operation log is obtained from the data lake, and the preset test model and preset test unit are obtained, and then gathered in the log analysis system.

[0031] S12. Process the operation log according to the preset test model and the preset test unit to obtain a test data file.

[0032] Explanatory, according to the preset test model, the operation log is first preprocessed to comb the operation log into preprocessed data corresponding to the data format of the preset test model. Exemplarily, when the preset test model is the above-mentioned four-tuple, the operation log is first combed into a data format corresponding to the user, the operation object, the operation frequency, and the operation parameter according to the number of users, the number of operation objects, the operation frequency, and the operation parameter contained in the four-tuple.

[0033] Then, based on the preset test unit and the preprocessed data, the numerical values ​​required by the preset test unit are counted. The numerical values ​​required by the test unit are defined based on a preset test model corresponding to, but not limited to, a quadruple, so as to process the operation log, and obtain data that conforms to the preset test model and is required by the preset test unit from the operation log, that is, the test data, which can also be called a test value. The test data represents the specific numerical values ​​required for network management testing, and the test data is combined with the preset test unit to generate a test data file. The test data file contains a test blueprint and specific test values, that is, the test data file contains specific test data that drives the execution of preset initial test cases and scheduling strategy information for different preset initial test cases. Therefore, the test data file is data determined based on the actual situation of existing network users.

[0034] For example, please refer to Figure 2 ,like Figure 2 As shown, when the above-mentioned preset test model is described by a quaternary group, and the preset test unit corresponding to the functional test is (1,1,1,*), that is, when the preset test unit defines the required test data based on the quaternary group, the operation log is processed to obtain the specific value of each * that conforms to the quaternary group from the operation log, that is, the * of different services corresponds to different values, thereby obtaining the (1,1,1,*) corresponding to different services, and generating a test data file according to the corresponding (1,1,1,*), specifically (1,1,1,*) is included in the test data file, and at the same time, the test data file also includes the execution process or processing process of the network management test using the specific value corresponding to (1,1,1,*), so that the preset test model clarifies that the operation logs stored in the data lake are preprocessed by the quaternary group and its corresponding parameters, that is, the preset test model clarifies how to perform preliminary analysis and processing on the operation logs stored in the data lake, and uses the preset test model and the preset test unit to process the operation logs to generate specific test data files and specific test cases.

[0035] S13: Associating the test data file with a preset initial test case to test the network management.

[0036] Explanatory speaking, the system test of network management products is to simulate the operation behavior of different users at the same time and verify whether each user gets the expected results. Based on this, when the preset test model is established based on user behavior, that is, the preset test model is used to describe the user's operation behavior, the test of network management products is to test according to the preset test model while paying attention to the execution results of the test and the impact of the execution on the network management. Therefore, the test cases for network management testing can describe the input according to the preset test model, and add corresponding input verification, so as to construct test cases for network management testing.

[0037] Based on the above concept, according to the purpose of testing the network management and the corresponding functional operations that are of key concern, an initial test case is pre-constructed, that is, a preset initial test case, which is a test case that does not contain a specific test value, and the preset initial test case is used to define how the functional operation and the preset test model cooperate to perform the test, the test data contained in the test data file is the input of the preset initial test case, and the output of the preset initial test case is the test result of testing the network management. The preset initial test case can be reused because it does not contain a specific test value, and several preset initial test cases can form a test case library, so that multiple sets of test data can be combined with a set of test cases to complete network management tests of multiple test dimensions or test purposes, that is, the same preset initial test case can be combined with different test data to achieve network management tests of corresponding test dimensions or test purposes. Exemplarily, in the case where the above-mentioned preset test model is described by a four-tuple, the input of the preset initial test case is the above-mentioned four-tuple parameter description, and the value of the four-tuple parameter is the specific test data contained in the test data file.

[0038] When testing the network manager, the functional operation items (i.e., test blueprint) with arranged execution order contained in the test data file are associated and driven to correspond to the preset initial test cases in the test case library, and the corresponding test data is transmitted. The test data is a specific numerical value that conforms to the preset test model (including but not limited to the preset N-tuple), so that the network manager can be tested and the test results can be output, so that the test data file can be associated with the preset initial test case to test the network manager.

[0039] The embodiment of the present invention is a technical concept for testing a network manager based on decoupling a preset initial test case from a test data file. By obtaining a preset test model and a preset test unit, and obtaining a user's operation log, the operation log is processed according to the preset test model and the preset test unit to obtain a test data file. The test data file contains test data, so as to obtain test data according to the actual situation of the existing network users, and the test data file is associated with the preset initial test case to test the network manager. Since the network manager is tested according to the actual situation of the existing network users, and the preset initial test case is decoupled from the test data file, the test case of the network manager can be efficiently constructed and updated, which is beneficial to the management and expansion of the test case, and can make the network manager that passes the test more compatible with the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0040] In one embodiment, see Figure 7 , Figure 7 The first sub-flow diagram of the network management test method provided by the embodiment of the present invention. Figure 7As shown, in this embodiment, the operation log is processed according to the preset test model and the preset test unit to obtain a test data file, including:

[0041] S71, preprocessing the operation log according to the preset N-tuple to obtain preprocessed data whose data format corresponds to the preset N-tuple;

[0042] S72. According to the preprocessed data, obtain the numerical value required by the preset test unit, and assign the numerical value to the preset test unit to obtain a test data file, wherein the preset test unit is constructed based on the preset N-tuple.

[0043] Explanatory, the operation log is preprocessed according to the preset N-tuple, and the operation log is combed into preprocessed data corresponding to the data format of the preset test model. Exemplarily, when the preset test model uses parameters containing the following four-tuple to describe user operations {number of users, number of operation objects, operation frequency, operation parameters}, the operation log is first combed into a data format corresponding to the user, operation object, operation frequency, and operation parameter according to the number of users, number of operation objects, operation frequency, and operation parameter contained in the four-tuple, thereby obtaining preprocessed data corresponding to the four-tuple.

[0044] Then, according to the preset test unit and the preprocessed data, the numerical value required by the preset test unit is counted. The numerical value required by the test unit is defined based on the preset test model corresponding to the quaternary group, so that the data based on the preset test model and required by the preset test unit, i.e., the test data, is counted from the operation log. The test data represents the specific numerical value required for the network management test, and the test data is combined with the preset test unit to generate a test data file. Thus, the test data file is data determined according to the actual situation of the existing network users. Exemplarily, in the case where the preset test model is the above-mentioned quaternary group, the numerical values ​​corresponding to the number of users, the number of operation objects, the operation frequency, and the operation parameters contained in the operation log are counted to obtain the quaternary parameter value. The quaternary parameter value represents the specific numerical values ​​corresponding to the parameters "number of users, number of operation objects, operation frequency, and operation parameters" contained in the quaternary group. Then, since the preset test unit represents the execution process or calling process of the preset initial test case based on the quadruple, after obtaining the specific numerical value corresponding to the quadruple parameter, that is, the quadruple parameter value, the quadruple parameter value is substituted into the preset test unit to obtain a test data file containing the specific test numerical value, thereby combining the quadruple parameter value with the preset test unit to obtain a test data file.

[0045] In an embodiment of the present invention, an operation log is preprocessed according to a preset N-tuple to obtain preprocessed data having a data format corresponding to the preset N-tuple, a numerical value required by a preset test unit is obtained according to the preprocessed data, and the numerical value is assigned to the preset test unit to obtain a test data file, thereby realizing the technical concept of decoupling the preset initial test case from the test data file to obtain a test data file independent of the preset initial test case, and the test data file is obtained based on the operation log, so that the test data file is obtained according to the actual situation of the users of the existing network, which can make the network management that passes the test more matched with the actual situation of the existing network, thereby improving the effectiveness of the network management.

[0046] In one embodiment, a plurality of preset test units are included, and the preset test units include at least one of the following: a functional test unit, a commercial pressure playback test unit, a performance limit test unit, and a reliability test unit.

[0047] Explanatory, see Figure 4 , Figure 4 A schematic diagram of a network management test dimension framework of a network management test method provided by an embodiment of the present invention, such as Figure 4 As shown, multiple preset test units can be used in network management testing, that is, the network management test contains several preset test units, different preset test units realize corresponding test purposes and test functions, and form a test framework, and then according to different test dimensions or test purposes, the corresponding preset initial test cases are called from the preset test case library, and the preset initial test cases are combined with the corresponding test data files to obtain different target test cases, and the target test cases are used to test the network management to achieve different test strategies and test purposes, and the preset initial test cases can be reused in different test schemes, that is, the same preset initial test case is called in different test data files, and then the test data file is determined by the preset test unit, and different test data files can call the same preset initial test case to achieve the corresponding test dimensions or test purposes, and realize that multiple sets of test data are combined with a set of test cases to complete network management tests with multiple test dimensions or test purposes. Among them, the preset initial test cases include at least one of the following: a functional test unit, a performance efficiency test model, and a reliability test unit. Furthermore, the performance efficiency test model may include a commercial stress playback test unit and a performance limit test unit (ie, business PT-performance limit test unit), which can improve the adaptability of network management testing to daily operation and maintenance or user operating habits in security scenarios, thereby improving the effectiveness of network management testing.

[0048] Furthermore, several test data files are used to perform network management tests according to the pre-set test logic, thereby achieving network management tests with different test dimensions or test purposes. Figure 5 , Figure 5 A test logic diagram of a network management test method provided by an embodiment of the present invention, such as Figure 5 As shown, the function of the network management system is first tested by a functional testing unit, and then the performance and efficiency testing model (commercial stress playback test and business PI test, respectively) is used to test the performance and efficiency of the network management system. Finally, the reliability testing unit is used to test the reliability of the network management system, so as to achieve a complete set of tests on the network management system that meet the corresponding test strategies and test objectives, which can improve the effectiveness of the network management test.

[0049] In one embodiment, the preset test unit is a functional test unit, and the preset N-tuple is an N-tuple in which all parameters except the operation parameter are 1.

[0050] Explanatory, since the preset test unit determines the test data file, and the test data file determines the test purpose or test function, therefore, when the preset test unit is a functional test unit and the preset test model is a preset N-tuple, all parameters except the operation parameters in the preset N-tuple are set to 1, and the preset N-tuple after the value is 1 is used as the preset test unit corresponding to the functional test unit. Exemplarily, as described above, when the preset test model is a four-tuple {number of users, number of operation objects, operation frequency, operation parameters}, the number of users, the number of operation objects, and the operation frequency are all set to 1, and {1, 1, 1, operation parameters} are obtained, wherein the operation parameters can be described by *, and the operation parameters are related to the specific business, thereby obtaining (1, 1, 1, *), and (1, 1, 1, *) is used as the preset test unit corresponding to the functional test.

[0051] In the embodiment of the present invention, when the preset test unit is a functional test unit, the preset test unit is set to a preset N-tuple obtained by taking the values ​​of 1 for all parameters contained in the preset test model except the operation parameters. This not only configures the functional test based on the preset test model of the user behavior, thereby performing the functional test on the network manager according to the actual situation of the existing network users, but also focuses on testing whether the corresponding functions of the network manager are normal from the functional perspective, so as to make the test of the network manager targeted and single-purpose as much as possible without involving other factors, avoid complicating the test of the network manager, and when there is a problem during the test, timely locate and maintain the problem, thereby improving the efficiency of the network manager test.

[0052] In one embodiment, the preset test unit is the commercial pressure playback test unit, and the preset test model is a preset N-tuple, the preset N-tuple includes at least one of the following: the number of users, the number of operation objects, the operation frequency, and the operation parameters, and the preset test unit is defined as follows corresponding to the preset N-tuple:

[0053] The number of users is the number of users who initiate operations on a single functional operation within a preset time granularity;

[0054] The number of operation objects is the average of the sum of the numbers of objects operated by all users within a preset time granularity for a single function operation;

[0055] The operation frequency is the average of the sum of the operation frequencies of all users within a preset time granularity for a single function operation;

[0056] The operation parameter is the operation content corresponding to the user within a preset time granularity for a single functional operation.

[0057] Explanatory, please continue to see Figure 4 ,like Figure 4 As shown, the preset test unit is a commercial stress playback test unit. The commercial stress playback test unit represents the simulation of the actual commercial business (i.e., the operation log) based on the actual commercial business in the production environment, so as to test the network management in the form of replaying the actual commercial business. The commercial stress playback test unit belongs to the performance efficiency test. In the case where the preset test model is a preset N-tuple, the preset N-tuple may include at least one of the following: the number of users, the number of operation objects, the operation frequency, and the operation parameters. Since the preset test unit is constructed based on the preset test model, based on the above-mentioned preset N-tuple, under the concept that the parameter values ​​contained in the preset N-tuple of each functional operation are all positioned at the average level, the preset test unit corresponds to the preset N-tuple definition as follows:

[0058] 1) The number of users is the number of users who initiate operations on a single functional operation within a preset time granularity. The number of users indicates the number of concurrent users who initiate operations on the same functional operation within the preset time granularity. A large number of users indicates a high concurrency, and a small number of users indicates a low concurrency. Exemplarily, the corresponding number of users is determined for a single functional operation. In the case of a preset time granularity including but not limited to 15 minutes, for a single functional operation A, the number of users who initiate operations on functional operation A within every 15 minutes is counted. The number of users indicates the number of concurrent users who initiate operations on functional operation A within every 15 minutes.

[0059] 2) The number of objects operated is the average of the sum of the number of objects operated by all users within the preset time granularity for a single functional operation, that is, for a single functional operation, the ratio of the sum of the number of objects operated by all users within the preset time granularity to the total number of users is calculated. For example, for functional operation A, which includes several objects, within the time granularity of 15 minutes, if the number of objects operated by user A is 20, the number of objects operated by user B is 30, and the number of objects operated by user C is 10, then the number of users = (20 + 30 + 10) / 3 = 20.

[0060] 3) Operation frequency is the average of the sum of the operation frequencies of all users within the preset time granularity for a single function operation, that is, for a single function operation, the ratio of the sum of the operation frequencies of all users within the preset time granularity to the total number of users is calculated, where the operation frequency represents the frequency and number of times the user performs the operation within the preset time granularity. For example, for function operation A, within the time granularity of 15 minutes, if the operation frequency of user A is 15, the operation frequency of user B is 30, and the operation frequency of user C is 15, then the operation frequency = (15 + 30 + 15) / 3 = 20.

[0061] 4) Operation parameters are the operation content corresponding to the user within the preset time granularity for a single functional operation. The operation content can also be called operation details. For example, when the functional operation is an information interaction operation, the operation content can be specific communication methods such as video, voice, and text. The operation parameters will be extracted from the operation log according to the specific business corresponding to the functional operation.

[0062] The embodiments of the present invention define corresponding preset test units according to preset N-tuples corresponding to preset test models, thereby realizing construction of corresponding preset test units based on user behaviors, and then performing commercial stress playback tests on network managers according to the actual conditions of users in the existing network. This can make the network managers that pass the test more closely match the actual conditions of the existing network, thereby improving the effectiveness of the network manager test.

[0063] Furthermore, the preset test unit includes at least one of the following: a daily playback test unit, a weekly playback test unit, and a nighttime playback test unit; wherein the daily playback test unit is a test strategy for performing commercial stress playback tests on network managers based on the preset test unit and taking each day as the time unit; the weekly playback test unit is a test strategy for performing commercial stress playback tests on network managers based on the preset test unit and taking each week as the time unit; the nighttime playback test unit is a test strategy for performing commercial stress playback tests on network managers based on the preset test unit and taking the preset time period at night every day as the time unit.

[0064] Explanatory, the embodiment of the present invention designs different test time dimensions, and designs corresponding playback test units from the three dimensions of day, week, and night, namely, a day-level playback test unit, a weekly playback test unit, and a night-time playback test unit, so as to test the network manager from different dimensions, and different dimensions can be combined to test the network manager, namely, the preset test unit includes at least one of the following: a day-level playback test unit, a weekly playback test unit, and a night-time playback test unit; wherein the day-level playback test unit is a test strategy that performs a commercial stress playback test on the network manager based on a preset test unit with each day as the time unit; the weekly playback test unit is a test strategy that performs a commercial stress playback test on the network manager based on a preset test unit with each week as the time unit. Due to the weekly time characteristics of the network manager, the weekly playback test unit is used to test the network manager, which can make the network manager that passes the test more matched with the actual situation of the existing network, thereby improving the effectiveness of the network manager test; the night-time playback test unit is a test strategy that performs a commercial stress playback test on the network manager based on a preset test unit with a preset time period at night every day as the time unit.

[0065] Exemplarily, according to the definition of the above-mentioned preset test unit corresponding to the preset N-tuple, the daily playback test unit, the weekly playback test unit, and the night playback test unit are respectively described as follows:

[0066] 1) For the daily playback test unit, first design a commercial stress playback test unit, then extract 7 busy hours per day according to the on-site operation and maintenance frequency, and determine the 7 busy hours as 9:00-12:00 and 14:00-18:00 on each working day. Then, for the playback data of each busy hour, the 15-minute time granularity with the largest number of users in the busy hour can be selected for data equivalence, that is, the four-tuple parameter value of each functional operation in each busy hour is the value corresponding to the 15-minute time granularity with the largest number of users of the corresponding functional operation in this busy hour. Finally, the four-tuple parameter value of the 7 busy hours is selected for the daily playback. For any busy hour, the busy hour with the largest number of function operation items among the busy hours on different days is selected as the busy hour data for day-level playback. For example, for day-level playback of the busy hour 9:00-10:00, the entire log time range is 3 days. There are 15 function operation items from 9:00-10:00 on the first day, 14 function operation items from 9:00-10:00 on the second day, and 17 function operation items from 9:00-10:00 on the third day. Then, the four-tuple parameter value of the busy hour 9:00-10:00 for day-level playback is the four-tuple parameter value of 9:00-10:00 on the third day.

[0067] 2) For the weekly playback test unit, the goal is to playback the operation behavior of the live network for a week. The corresponding playback model is based on the commercial stress playback test unit corresponding to the daily playback test unit, and is extended to playback the live network data from Monday to Friday. From Monday to Friday, the playback can also be performed according to 7 busy hours, but the playback data for each day will be selected within the same day of the week within the log range. For example, the playback data for Monday 9:00-10:00 is selected based on the four-tuple parameter data of 9:00-10:00 with the largest number of function operation items from 9:00-10:00 on Monday within the log range. The same applies from Tuesday to Friday. In this way, the weekly playback data is generated. Due to the weekly time characteristics of the network management, the weekly playback test unit is used to test the network management, which can make the network management that passes the test more consistent with the actual situation of the live network, thereby improving the effectiveness of the network management test.

[0068] 3) For the night playback test unit, since there are major operations such as upgrades and expansions at night, the six busy hours of 21:00-24:00 and 00:00-3:00 can be selected. The value strategy for each busy hour can refer to the value corresponding to the daily playback test unit.

[0069] The embodiment of the present invention designs different test time dimensions, designs corresponding playback test units from three dimensions of day, week and night, and can combine different dimensions to test the network manager. The network manager can be tested according to the actual situation of the existing network users at different times, and the network manager that passes the test can be more matched with the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0070] In one embodiment, the preset initial test case is the performance limit test unit, and the preset initial test case includes at least one of the following:

[0071] The preset initial test case includes all functional operations and corresponding preset test models;

[0072] The preset initial test case includes all functional operations corresponding to the functions operated by the user within the preset time granularity and the corresponding preset test models;

[0073] The preset initial test case includes all functional operations that depend on the same preset service and the corresponding preset test models.

[0074] Explanatory, please continue to see Figure 4 ,like Figure 4As shown, the preset initial test case is a performance limit test unit, which represents a model for testing the performance limit of the network management, and the performance limit test unit belongs to the performance efficiency test. For the performance limit test, a hierarchical test can be used, and different levels of tests can be combined, that is, the preset initial test case includes at least one of the following:

[0075] 1) The preset initial test case includes all functional operations and the corresponding preset test models, that is, all functional operations are arranged into test files. Each test file consists of different functional operations and the corresponding preset N-tuple parameter values. Different functional operations can be executed sequentially.

[0076] 2) The preset initial test case includes all functional operations corresponding to the functions performed by users within the preset time granularity and the corresponding preset test models, that is, all functional operations corresponding to the functions performed by users within the time granularity of 15 minutes and the corresponding preset N-tuple parameter values ​​are selected, and different functional operations can be executed concurrently.

[0077] 3) The preset initial test case includes all functional operations that depend on the same preset service and the corresponding preset test models. The test file can be compiled based on the list L of functional operations that depend on a certain service. The content of the test file consists of L and the corresponding preset N-tuple parameter values. Different functional operations can be executed concurrently.

[0078] The embodiment of the present invention tests the network manager by designing the performance limit test as different layered tests and combining the different layered tests. It can perform the performance limit test on the network manager according to the actual conditions of different performance angles of the users of the existing network, and can make the network manager that passes the test more closely match the actual conditions of the existing network, thereby improving the effectiveness of the network manager test.

[0079] For further information, please refer to Figure 5 , the preset test model is a preset N-tuple, the preset N-tuple includes at least one of the following: the number of users, the number of operation objects, the operation frequency, and the operation parameter, and the preset test unit is defined as follows corresponding to the preset N-tuple:

[0080] The number of users is the maximum value of the number of concurrent users in all time periods determined according to a preset time granularity within the scope of the operation log for a single functional operation;

[0081] The number of operation objects is the maximum value of the sum of all objects operated by a single user within the scope of the operation log for a single function operation;

[0082] The operation frequency is 1;

[0083] The operation parameter is the operation content of the user corresponding to the user number.

[0084] Explanatory, in the case where the preset test model is a preset N-tuple, the preset N-tuple includes at least one of the following: number of users, number of operation objects, operation frequency, and operation parameters. Since the preset test unit represents the execution process or specific use process of the preset test model, based on the above preset N-tuple, under the concept of positioning the parameter values ​​contained in the preset N-tuple of each functional operation at the maximum value level, for the performance limit test, the preset test unit corresponds to the preset N-tuple and is defined as follows:

[0085] 1) The number of users is the maximum number of concurrent users in all time periods determined according to the preset time granularity for a single functional operation within the scope of the operation log. For example, in the case of a time granularity of 15 minutes, within the time range specified by the operation log, for a single functional operation A, if the number of users operating functional operation A in one 15-minute period is M, the number of users operating functional operation A in each of the other 15 minutes can be uniformly represented by N, and N<M, then M is determined as the number of users.

[0086] 2) The number of operation objects is the maximum value of the sum of all objects operated by a single user within the scope of the operation log for a single functional operation. Exemplarily, within the data range defined by the operation log, for a single functional operation A, if user B1 operates the number of objects M = object m1 + object m2 + ... + object mm, the number of objects operated by other users B2, ..., Bn can be uniformly represented by N, N = object n1 + object n2 + ... + object nn, and N < M, then M is determined as the number of operation objects.

[0087] 3) The operation frequency is 1. In the performance limit test, the operation frequency is fixed at 1.

[0088] 4) The operation parameter is the operation content of the user corresponding to the user number. The operation content can also be called operation details. For example, when the functional operation is an information interaction operation, the operation content can be a specific communication method such as video, voice, or text. The operation parameter will be extracted from the operation log according to the specific business corresponding to the functional operation.

[0089] The embodiment of the present invention sets corresponding preset test units according to the preset N-tuple corresponding to the preset test model, and positions the parameter values ​​contained in the preset N-tuple of each functional operation at the maximum value level, thereby setting corresponding preset test units based on user behavior, and then performing extreme performance testing on the network manager according to the actual situation of the existing network users. The network manager that passes the test can be more closely matched with the actual situation of the existing network, thereby improving the effectiveness of the network manager test.

[0090] In one embodiment, after the test data file is associated with a preset initial test case to test the network management, the method further includes:

[0091] Get the preset scale mapping table and the target scale;

[0092] According to the preset scale mapping table and the target scale, adjusting the scale-related values ​​contained in the test data file;

[0093] The network management is tested using the adjusted test data file.

[0094] Explanatory, a scale mapping table is pre-constructed, i.e., a preset scale mapping table, which describes an adjustment mapping relationship of enlarging or reducing the scale-related values ​​involved in the test data file according to the comparison between the target scale and the current scale based on the test data file obtained according to the current operation log as the benchmark test file, wherein the benchmark test file represents a file corresponding to a commonly used scale under normal circumstances. Exemplarily, assuming that the current scale is A, the target scale B of the test is first determined, and in the case of A>B, the scale-related values ​​involved in the test data file are reduced, and in the case of A<B, the scale-related values ​​involved in the test data file are expanded, and the preset scale mapping table describes the mapping relationship of enlarging or reducing the scale-related values ​​involved in the test data file under different scale situations of the target scale B, and the mapping relationship can include expansion or reduction, and can also include the amplitude or degree of expansion or reduction.

[0095] Based on the construction of the above-mentioned preset scale mapping table, when the network manager faces larger-scale or other-scale operation and maintenance requirements and needs to retest the network manager, first determine the target scale and obtain the preset scale mapping table, adjust the scale-related values ​​contained in the test data file according to the preset scale mapping table and the target scale, and use the adjusted test data file to test the network manager.

[0096] The embodiment of the present invention obtains a preset scale mapping table and a target scale, adjusts the scale-related values ​​contained in the test data file according to the preset scale mapping table and the target scale, and uses the adjusted test data file to test the network manager. When the scale faced by the network manager changes, test data of any scale can be adaptively acquired, thereby improving the effectiveness and efficiency of the network manager test.

[0097] In one embodiment, after the operation log is processed, the following steps are included:

[0098] Processing the operation log to obtain the user function operation corresponding to the user operation;

[0099] Compare the user function operation with the preset initial test cases contained in the preset test case library;

[0100] In the case where there is no preset initial test case corresponding to the user function operation, a prompt indicating that the preset initial test case is missing is issued.

[0101] Explanatory, please continue to see Figure 2 and Figure 6 , process the operation log. This step can be the same step as the step of processing the operation log for obtaining the test data file, and obtain the user function operation corresponding to the user operation. The user function operation is the output of the log analysis system. When the preset test model is a preset N-tuple, the user function operation is the preset N-tuple corresponding to the preset test model. Multiple user function operations can be presented in a function operation list and can be included in an atomic operation analysis report, wherein the atomic operation analysis report is a file generated by processing the operation log to analyze the user's operation.

[0102] Pre-build several preset initial test cases and combine all the preset initial test cases into a test case library, namely the preset test case library, see Figure 6 The preset test case library in the preset test case library generally includes multiple preset initial test cases, and then the user function operation is compared with the different preset initial test cases contained in the preset test case library. The operation item identifier corresponding to the user function operation, including but not limited to the operation item name, can be compared with the operation item identifier corresponding to the preset initial test case contained in the preset test case library, including but not limited to the operation item name, so as to include but not limited to judging whether there is a preset initial test case corresponding to the user function operation through the preset test case library, and in the case that the preset initial test case corresponding to the user function operation does not exist, a prompt of missing preset initial test case is issued, so as to prompt relevant personnel to add corresponding preset initial test cases.

[0103] The embodiment of the present invention processes the operation log to obtain the user function operation corresponding to the user's operation, compares the user function operation with different preset initial test cases, and issues a prompt of missing preset initial test case when there is no preset initial test case corresponding to the user function operation. The preset initial test case can be improved according to the actual situation of the users in the existing network, so that the network management that passes the test is more matched with the actual situation of the existing network, thereby improving the effectiveness of the network management test.

[0104] In one embodiment, associating the test data file with a preset initial test case to test the network management includes:

[0105] According to the functional operation items in the test data file and arranged in execution order, calling the preset initial test case from the preset test case library;

[0106] The test data contained in the test data file is input into the preset initial test case to test the network management.

[0107] Explanatoryly, according to the functional operation items with arranged execution order contained in the test data file, the preset initial test case is called from the preset test case library, and the same preset initial test case in the preset test case library can be called by different test data files to achieve different test purposes or test functions, so as to realize the reuse of the same preset initial test case, and then the test data contained in the test data file is input to the preset initial test case, so as to associate the decoupled test data file with the preset initial test case, and pass the corresponding test data to the preset initial test case. When the preset test model is a preset N-tuple, the test data is the value of the preset N-tuple, so as to test the network manager, and after completing the test of the network manager, the corresponding test result is output.

[0108] The embodiment of the present invention tests the network manager by calling a preset initial test case from a preset test case library according to the functional operation items in the test data file with an arranged execution order, and inputting the test data contained in the test data file into the preset initial test case, thereby realizing the technical concept of decoupling the preset initial test case from the test data file to test the network manager, which can realize efficient construction and update of the test cases of the network manager, is beneficial to the management and expansion of the test cases, and can improve the flexibility of the network manager test.

[0109] See also Figure 8 , Figure 8 A schematic block diagram of the structure of a network management device provided by an embodiment of the present invention. Figure 8 As shown, the network management device 300 includes a processor 301 and a memory 302 , and the processor 301 and the memory 302 are connected via a bus 303 , such as an I2C (Inter-integrated Circuit) bus.

[0110] Specifically, the processor 301 is used to provide computing and control capabilities to support the operation of the entire network management device. The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0111] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0112] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present invention, and does not constitute a limitation on the network management device to which the embodiment of the present invention is applied. The specific server may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] The processor is used to run a computer program stored in a memory, and implement any one of the network management testing methods provided by the embodiments of the present invention when executing the computer program.

[0114] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the network management device described above can refer to the corresponding process in the aforementioned network management test method embodiment, and will not be repeated here.

[0115] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any network management testing method provided in the description of the embodiment of the present invention.

[0116] The storage medium may be an internal storage unit of the network management device described in the above embodiment, such as the memory of the network management device. The storage medium may also be an external storage device of the network management device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the network management device.

[0117] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0118] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0119] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A network management testing method, characterized in that: include: Obtain the preset test model and preset test unit, and obtain the user's operation log; Processing the operation log according to the preset test model and the preset test unit to obtain a test data file; The test data file is associated with a preset initial test case to test the network management.

2. The network management testing method according to claim 1, characterized in that: The preset test model includes a preset N-tuple, and the preset N-tuple includes at least one of the following: the number of users, the number of operation objects, the operation frequency, and the operation parameter, wherein N is a natural number.

3. The network management testing method according to claim 2, characterized in that: According to the preset test model and the preset test unit, the operation log is processed to obtain a test data file, including: Preprocessing the operation log according to the preset N-tuple to obtain preprocessed data whose data format corresponds to the preset N-tuple; According to the preprocessed data, a numerical value required by the preset test unit is obtained, and the numerical value is assigned to the preset test unit to obtain a test data file, wherein the preset test unit is constructed based on the preset N-tuple.

4. The network management testing method according to claim 3, characterized in that: It comprises a plurality of preset test units, and the preset test units include at least one of the following: a functional test unit, a commercial pressure playback test unit, a performance limit test unit, and a reliability test unit.

5. The network management testing method according to claim 4, characterized in that: The preset test unit is the functional test unit, and the preset N-tuple is an N-tuple in which all parameters except the operation parameters are 1.

6. The network management testing method according to claim 4, characterized in that: The preset test unit is the commercial pressure playback test unit, and the preset test unit corresponds to the preset N-tuple and is defined as follows: The number of users is the number of users who initiate operations on a single functional operation within a preset time granularity; The number of operation objects is the average of the sum of the numbers of objects operated by all users within a preset time granularity for a single function operation; The operation frequency is the average of the sum of the operation frequencies of all users within a preset time granularity for a single function operation; The operation parameter is the operation content corresponding to the user within a preset time granularity for a single functional operation.

7. The network management testing method according to any one of claims 4 to 6, characterized in that: The preset test unit is the performance limit test unit, and the preset test unit includes at least one of the following: The preset test unit includes all functional operations and corresponding preset test models; The preset test unit includes all functional operations corresponding to the functions operated by the user within the preset time granularity and the corresponding preset test models; The preset test unit includes all functional operations that depend on the same preset service and the corresponding preset test models.

8. The network management testing method according to claim 7, characterized in that: The preset test unit corresponding to the preset N-tuple is defined as follows: The number of users is the maximum value of the number of concurrent users in all time periods determined according to a preset time granularity within the scope of the operation log for a single functional operation; The number of operation objects is the maximum value of the sum of all objects operated by a single user within the scope of the operation log for a single function operation; The operation frequency is 1; The operation parameter is the operation content of the user corresponding to the user number.

9. The network management testing method according to claim 1, characterized in that: After the test data file is associated with a preset initial test case to test the network management, the method further includes: Get the preset scale mapping table and the target scale; According to the preset scale mapping table and the target scale, adjusting the scale-related values ​​contained in the test data file; The network management is tested using the adjusted test data file.

10. The network management testing method according to claim 1, characterized in that: Processing the operation log further includes: Processing the operation log to obtain the user function operation corresponding to the user operation; Compare the user function operation with the preset initial test cases contained in the preset test case library; In the case where there is no preset initial test case corresponding to the user function operation, a prompt indicating that the preset initial test case is missing is issued.

11. The network management testing method according to claim 1, characterized in that: The test data file is associated with a preset initial test case to test the network management, including: According to the functional operation items in the test data file and arranged in execution order, calling the preset initial test case from the preset test case library; The test data contained in the test data file is input into the preset initial test case to test the network management.

12. A network management device, characterized in that: The network management device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of the network management testing method as described in any one of claims 1 to 11 are implemented.

13. A computer-readable storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the network management testing method described in any one of claims 1 to 11.

Images