Test case generation method and system for communication equipment, equipment and medium

Abstract

The invention relates to the technical field of automatic test case generation, and provides a test case generation method and system for communication equipment, equipment and a medium, and the method comprises the steps: obtaining command line and hierarchy information of a test case of the communication equipment, and carrying out feature vectorization to obtain a hierarchy and a command line vector; command lines are classified according to levels, and connecting lines are established between level information and convertible level vectors to form a first graph data structure; and connecting lines among the hierarchical command line vectors to form a second graph data structure. Inputting the data into a graph neural network for coding, splicing and outputting a feature vector, and obtaining a command line vector embedded with hierarchical information; a plurality of such vectors are input into a pointer network, and an optimal command line is output as a test case. The method can overcome the limitation of the existing heuristic algorithm, and improve the test case quality and generation efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic test case generation, and in particular, relates to a test case generation method, system, device and medium for communication equipment. Background Art

[0002] In the modern software development process, continuous integration (CI) has become a key practice to improve development efficiency and software quality. The core concept of continuous integration is to frequently integrate code into the trunk and verify the correctness of each integration through automated testing. This practice can quickly discover and solve integration problems, thereby avoiding a large number of conflicts and errors caused by integration delays.

[0003] Automated testing plays a vital role in continuous integration. By writing and running automated test cases, the development team is able to detect problems in the code immediately after each code submission. This not only speeds up the process of discovering and fixing problems, but also improves the reliability and stability of the code. Automated testing can cover multiple aspects of the code, including unit testing, integration testing, and system testing, thereby ensuring the quality of the software at different levels.

[0004] Traditionally, the generation of automated test cases for communication equipment relies on technicians' deep understanding of equipment protocols and specific test scenarios, and manual writing and supplementation of commands under different views based on existing scenarios. Technicians need to design and write test cases based on protocol documents, equipment functions, and expected behaviors in specific scenarios. Although this method can generate high-quality test cases, it has obvious limitations: with the continuous updating of protocols and equipment, the number and complexity of test cases are also growing exponentially, and traditional manual methods are difficult to meet the needs of continuous integration environments.

[0005] Currently, common test case generation methods are mainly based on heuristic algorithms, which arrange and combine existing test case sets and optimize them according to device feedback in each iteration. These methods usually consider the impact of the order of different test cases on the final generation results.

[0006] For example, assuming that the existing test case generation method is based on simple permutations and combinations, for the test of a communication device, it may generate the following test case sequence:

[0007] Command line 1 -> Command line 2 -> Command line 3 -> Command line 4

[0008] However, in actual communication devices, these command lines may have a hierarchical relationship, for example, command line 1 and command line 2 belong to one level, command line 3 and command line 4 belong to another level, and there is no strict dependency relationship between the order of the two levels.

[0009] However, since the existing methods do not take this hierarchical structure into consideration and simply perform permutations and combinations, this may result in situations that do not conform to the actual operation logic of the device when executing the test cases. For example, command line 3 may be executed first, and then command line 1. In actual devices, this order may be unreasonable, resulting in inaccurate test results and failure to effectively discover problems in the device.

[0010] In short, this method is not completely suitable for automatic test case generation of communication equipment. The test cases of communication equipment are often at different levels, and there is no significant correlation between the order of test cases at different levels, which makes the existing test case generation method perform poorly in this scenario. Summary of the invention

[0011] To solve the above problems, the present invention provides a test case generation method, system, device and medium for communication equipment, which uses a multi-level graph pointer network technology based on a reinforcement learning framework to overcome the limitations of existing heuristic algorithms and improve the test case quality and generation efficiency.

[0012] The present invention is achieved through the following technical solutions:

[0013] Obtain command lines of different test cases of software running on the communication device, and level information corresponding to the command lines; vectorize the command lines and level information to obtain level vectors and command line vectors;

[0014] Classify the command lines according to the command line levels to obtain command line sets at different levels;

[0015] For the hierarchical information, if the two hierarchical levels can be converted, a connection is created between the hierarchical vectors of the two hierarchical levels to obtain a first graph data structure consisting of a hierarchical set; in the command line set corresponding to each hierarchical level, a connection is created between the command line vectors to obtain a second graph data structure consisting of the command line set;

[0016] The first graph data structure and the corresponding level vector, the second graph data structure and the corresponding command line vector are respectively input into two different graph neural networks for multi-layer encoding, and then the feature vectors output from the two different graph neural networks are concatenated to obtain a command line vector embedded with level information;

[0017] A plurality of command line vectors with embedded hierarchical information are input into the pointer network, and the pointer network outputs the best execution order of the command lines. The command lines are spliced ​​according to the best execution order of the command lines to obtain the test cases of the communication equipment.

[0018] Further,

[0019] The command lines for obtaining different test cases of the software running on the communication device include:

[0020] Extract the original command lines of different test cases from the source code of the software run by the communication device based on regular expressions, and obtain the hierarchical information of the original command lines according to the extraction position during extraction;

[0021] Based on the extraction results, the useless original command lines are cleaned up to obtain a command line set;

[0022] For the original command lines with parameter prompts in the command line set, according to the parameter condition matching algorithm, the original command lines with parameter prompts are spliced ​​into command lines for different test cases that can be executed.

[0023] Further,

[0024] If the parameter part of the original command line is marked with brackets;

[0025] The algorithm for matching the parameter conditions is used to combine the command lines with the parameter prompts into an executable command line; including:

[0026] Identify all command line segments in the current sequence that begin with the previous bracket and end with the next bracket.

[0027] Process parameter logic. If the parameters are separated by |, keep one parameter. If the parameter is a range, replace the parameter with one of the range values.

[0028] Recursively process the command line until there are no more argument flags.

[0029] Further,

[0030] The command lines are classified according to the command line levels, including:

[0031] A random forest classification model is trained based on the hierarchical information of the command line set and the corresponding information of the command line set, and the random forest classification model is used to classify the command lines without labeled hierarchies.

[0032] Further,

[0033] The plurality of command line vectors embedded with hierarchical information are input into the pointer network, the pointer network outputs the best execution order of the command lines, and the command lines are spliced ​​according to the best execution order of the command lines;

[0034] include:

[0035] Set all command line vectors h that embed level information c Input into the encoder of the pointer network, at each time t the encoder generates a corresponding hidden state e t , after time step j, enters the decoder;

[0036] The decoder is based on the hidden state d of the decoder at the current moment i , and the encoder hidden state e of the previous j time steps j , j∈(1,2,3…,n) performs additive attention mechanism to obtain the output at the current moment and an Attention score for each time period input

[0037] Attention score Calculate a distribution probability p. The command line corresponding to the largest distribution probability p is the command line output by the decoder at the current moment. The output order of the command lines is the optimal execution order of the command lines.

[0038] in:

[0039]

[0040] p(C i |C1,C2,C3…C i-1 ,P)=softmax(u i )

[0041] In the formula, v T , W1, W2 are learnable parameters that are continuously optimized during the iteration of the model.

[0042] Further,

[0043] The method further includes: optimizing the network parameters of the graph neural network and the pointer network, which includes:

[0044] Send the test case of the communication device output by the pointer network to the communication device, and calculate the reward function based on the return state of the communication device;

[0045] The test cases of the communication devices output by the pointer network are input into a critic network, and the critic network outputs a baseline score

[0046] Calculate the gradient of model optimization based on the baseline score and reward function, thereby updating and optimizing the network parameters of the graph neural network and pointer network;

[0047] The reward function R t for:

[0048]

[0049] Among them, R success For positive rewards, TS fail is the set of failed cases, rank(TS fail ) is the order of failed test cases, TS is the set of all test cases;

[0050] The gradient of the computational model optimization is:

[0051]

[0052] in, is the gradient of the policy parameters, π θ is the policy function, a t is the action at time t.

[0053] Further,

[0054] The critic network includes an encoder-decoder structure consisting of two LSTM networks.

[0055] A test case generation system for communication equipment, characterized by comprising:

[0056] The vector generation module is used to obtain command lines of different test cases of the software running on the communication device, and the level information corresponding to the command lines; the command lines and the level information are feature vectorized to obtain the level vector and the command line vector;

[0057] The classification module is used to classify command lines according to the command line level to obtain command line sets at different levels;

[0058] A graph structure building module is used for creating a connection between the level vectors of the two levels for the level information, if the two levels can be converted, to obtain a first graph data structure consisting of the level set; in the command line set corresponding to each level, creating a connection between the command line vectors, to obtain a second graph data structure consisting of the command line set;

[0059] An information embedding module, used to input the first graph data structure and the corresponding hierarchical vector, the second graph data structure and the corresponding command line vector into two different graph neural networks for multi-layer encoding, and then concatenate the feature vectors output from the two different graph neural networks to obtain a command line vector embedded with hierarchical information;

[0060] The test case generation module is used to input multiple command line vectors embedded with hierarchical information into the pointer network, the pointer network outputs the best execution order of the command lines, and the command lines are spliced ​​according to the best execution order of the command lines to obtain the test case of the communication device.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] The present invention classifies command lines according to levels to obtain command line sets at different levels, and simultaneously creates connections between level vectors to construct a first graph data structure, and creates connections between command line vectors at the same level to construct a second graph data structure. This graph structure construction method can clearly reflect the relationship between levels and the connection between command lines at the same level, and is suitable for test scenarios in which communication equipment has a clear hierarchical structure.

[0063] Then, the first graph data structure, the second graph data structure, and the corresponding hierarchical vectors and command line vectors are respectively input into two different graph neural networks for multi-layer encoding. The graph neural network can effectively process graph structure data, capture the relationships and features between nodes, and embed hierarchical information into the command line vector. The resulting command line vector with embedded hierarchical information contains both the features of the command line itself and the information of the hierarchy, making subsequent processing more accurate and targeted.

[0064] Finally, multiple command line vectors with embedded hierarchical information are input into the pointer network, and the optimal command line is output as a test case. The pointer network can be screened according to the embedded hierarchical information and command line characteristics, does not rely on traditional sequential relationships, and is more in line with the testing needs of communication equipment, thereby overcoming the limitations of existing heuristic algorithms and improving the quality and generation efficiency of test cases.

[0065] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or be understood by implementing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 A schematic diagram of the method of the present invention is shown;

[0068] Figure 2The command line pre-processing flow chart of the present invention is shown;

[0069] Figure 3 The detailed logic flow chart of command line pre-processing of the present invention is shown;

[0070] Figure 4 A multi-level graph pointer network framework diagram of the present invention is shown;

[0071] Figure 5 The reinforcement learning framework diagram of the present invention is shown;

[0072] Figure 6 A schematic diagram of a disordered command line set according to an embodiment of the present invention is shown;

[0073] Figure 7 The reinforcement learning framework diagram of the present invention is shown;

[0074] Figure 8 The reinforcement learning framework diagram of the present invention is shown;

[0075] Fig. 9 The reinforcement learning framework diagram of the present invention is shown. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0077] Figure 1 A schematic diagram of the method according to the present invention is shown, and the specific implementation details of the present invention include:

[0078] 1. Step 1: Data acquisition, obtain vectorized hierarchical data and command line data:

[0079] Starting from the source code of the communication device (referring to the software source code of the operating system running on the communication device), the original command lines of different test cases in the source code are roughly extracted based on regular expressions; (When extracting, the hierarchical information of the original command line is obtained according to the extraction position, and the hierarchical information is generally related to the source code file name); (At this time, the extracted test case command line does not contain hierarchical information)

[0080] Hierarchical information: In the source code of the communication device, there are some original command lines such as "config vlan 1,2" which belongs to the config view level, "router-id 10.120.120.21" which belongs to the router view level, and "ip address10.120.120.11 / 24" which belongs to the vlan view level.

[0081] Among them, "config", "router" and "vlan" are different levels of information, and each command line has its corresponding level.

[0082] Based on the extraction results, clean up the useless original command lines;

[0083] For the extracted command line set with parameter prompts, according to the parameter condition matching algorithm, the executable command line is spliced ​​as the test case set to be generated for the model; (such as Figure 2 is a diagram of the annotation level) (e.g. Figure 3 Detailed logic flow chart for command line pre-processing)

[0084] The hierarchical data and command line data are input into the fine-tuned pre-trained model, and the use case set is feature vectorized to obtain the vectorized hierarchical data and command line data.

[0085] Specific operation example of data acquisition in step 1:

[0086] (1) According to the source code address, based on the regular expression matching rules, extract the content of the command line in the XML tag. Each command line needs to be marked with the view level it comes from according to the view value in the tag;

[0087] (2) Perform preliminary cleaning based on the extracted command line set to remove interference items such as duplicates and null values;

[0088] (3) Based on the preliminarily cleaned command line set, recursively process the command line content based on the bracket matching algorithm (assuming that the command line parameters are marked with brackets). (This step mainly replaces the parameters with specific IP values, port ports, and corresponding values ​​related to the specific scenario);

[0089] The specific operation of the bracket matching algorithm is:

[0090] ① Identify all command line segments in the current sequence that start with ( and end with ).

[0091] ② Process the parameter logic. If the parameters are separated by |, you only need to keep one parameter. If the parameter is a range (1-4), you only need to use one of the range values ​​to replace the parameter.

[0092] ③ Recursively process the remaining parts in steps 1-3 until there are no more parameter flags.

[0093] (4) After parameter replacement, the command line and hierarchical information are input into the pre-trained model that has been fine-tuned for the command line test task to obtain a preliminary feature embedding vector, where the hierarchical vector set is represented by V = {V1, V2, V3…V n} indicates that the command line set at the specific level is used Represents, where j∈{1,2,3…n}.

[0094] 2. Step 2: Classify the command line data according to the command line hierarchy, and obtain the hierarchy information and command line sets at different hierarchies; abstract the hierarchy information and command line information into a graph structure, and input the corresponding graph structure, hierarchy vector and command line vector into the graph neural network for encoding, and finally obtain the command line vector embedded with the hierarchy information:

[0095] (1) Command line level classification:

[0096] A random forest classification model is trained based on the hierarchical information of the command line set and the corresponding information of the command line set. This model is only used to process command lines without labeled hierarchies.

[0097] Then, the corresponding command line sets are classified according to the hierarchical information to obtain the hierarchical information and command line sets at different hierarchical levels.

[0098] Training of Random Forest Classification Model:

[0099] 1. Prepare data:

[0100] Assume that there is a set of command line data with layer information annotated as a training set. For example: Command line 1: "configureport1speed1000", the annotated layer is the port configuration layer. Command line 2: "createvlan10", the annotated layer is the VLAN setting layer. Command line 3: "setrouter-id192.168.1.1", the annotated layer is the routing configuration layer.

[0101] 2. Feature Extraction

[0102] Extract features from the command line, which may include keywords, syntax structures, operation objects, etc. For example, for "configureport1speed1000", features such as "configure", "port", and "speed" can be extracted; for "createvlan10", features such as "create", and "vlan" can be extracted.

[0103] 3. Training the Random Forest Model

[0104] The training data with labeled hierarchical information is randomly divided into multiple subsets for training each tree.

[0105] Constructing decision trees: For each tree, randomly select features and samples from the training subset to split and construct a decision tree.

[0106] Repeatedly build multiple decision trees to form a random forest.

[0107] After the model is trained, for new command lines without labeled levels, such as "modifyinterfacesettings", its features are extracted and input into the random forest model. The model will predict the level category to which the command line belongs based on the relationship between the learned features and levels.

[0108] (2) Multi-layer graph neural network encoding:

[0109] The hierarchical information and command line information are abstracted into a graph structure, and the corresponding graph structure, hierarchical vector and command line vector are respectively input into the graph neural network for encoding, and finally the command line vector embedded with hierarchical information is obtained (the relationship between command line vectors at the same level is embedded, and view messages are embedded for the command line).

[0110] like Figure 6 is a simple set of random command lines. Based on the information in the source code, we can capture their related view information, represented by V1, V2, and V3 respectively (such as Figure 7 They correspond to the config view, router view, and vlan view respectively. By constructing our view graph structure and command line graph structure respectively (such as Figure 8 , where our graph structure is expressed through a two-dimensional array. 0 means no directed edge exists, 1 means there is a directed edge, and the existence of a directed edge means that you can use the relevant command to transfer from one view to another view or execute two command lines in sequence. Since the command lines under each view are fully connected graphs, the order of sequence is not required, so they are omitted and only the graph structure of the view is given.

[0111] The two graph structures we constructed (the hierarchical graph structure and the command line graph structure) are combined with the hierarchical vector and the command line vector for GNN encoding respectively. Finally, the vector input to the pointer network is the command line vector concatenated with the hierarchical vector.

[0112] Specific operation example of step 2:

[0113] According to the result obtained in step 1, the level information RV is set as the label, and a random forest model that can be classified according to the specific level is trained (this model is only used to obtain the corresponding level information according to the command line that does not have view information in the source code).

[0114] Construct the graph data structure of the hierarchy and command line: Generate the relationship graph between views according to the device logic (if the two hierarchies can be converted, an edge is created between the two view nodes). The graph data structure composed of the hierarchy set V is represented as G v =(V,E v ), where E v It is the set of edges between levels, represented by an adjacency matrix composed of 0s and 1s, and V represents the level vector node.

[0115] For each level V j , and its corresponding command line set C j The graph data structure formed is a fully connected graph. The graph data structure formed by the command line set V is expressed as in is the fully connected adjacency matrix between command lines.

[0116] Multi-layer graph neural network encoding: The hierarchical graph information, command line graph information and corresponding vectors are input into two different graph neural networks for multi-layer encoding. The overall algorithm is shown in the following formula.

[0117] Taking the hierarchical graph encoder as an example, the hierarchical encoding of each layer is based on all the layers adjacent to the current node W and b are learnable parameters in the graph neural network. The hierarchical information and command line information are input into the two graph encoders to obtain the corresponding view feature vectors. and the command line feature vector

[0118]

[0119] Concatenate the feature vectors obtained from the two graph encoders to form the input vector of the input pointer network:

[0120] 3. Step 3: Generate test cases using pointer network:

[0121] The command line vector embedded with hierarchical information is input into the pointer network encoder (Bi-LSTM). The bidirectional LSTM encoder encodes the relationship between command lines at different levels, calculates the Attention score based on the encoded hidden state and the encoder's hidden state at each moment, and outputs the command line with the highest score. That is, the output of the pointer network at a certain moment is a certain position of the input, that is, the corresponding command line.

[0122] Specific operation example of step 3:

[0123] Set all command line vectors h that embed level information c Input to the pointer network encoder (Bi-lstm), at each moment t the encoder will generate a corresponding hidden state e t , after time step j, to the decoder part.

[0124] Each output of the decoder will be based on the hidden state d of the decoder at the current moment i , and the encoder hidden state e of the previous j time steps j , j∈(1,2,3…,n) performs additive attention mechanism to obtain the output at the current moment and an Attention score for each time period input

[0125] Then calculate a distribution probability for the score, and the largest distribution probability will be the target output by the decoder at the current moment. T , W1, W2 are all learnable parameters, and the parameters will be continuously optimized in the subsequent iterations of the model.

[0126]

[0127] p(C i |C1,C2,C3…C i-1 ,P)=softmax(u i )

[0128] The overall model strategy of step 2 and step 3 constitutes a multi-level graph pointer network model. The encoder of the model consists of hierarchical GNN, command line GNN and BiLSTM. The GNN encoder is used to capture the graph-level dependencies between use cases (the relationship between the hierarchy and the use case), and the bidirectional LSTM is used to capture the relationship between use cases at different levels. By combining encoders, the model's ability to understand use cases can be effectively improved, thereby achieving better output effects. (The multi-level graph pointer network framework used in steps 2 and 3 is shown in the figure below) Figure 4 (shown)

[0129] Step 4: Reinforcement learning structure optimization model strategy (such as Figure 5 shown):

[0130] According to the output test command line, it is sent to the communication device for verification, and the reward function of the corresponding behavior of the actor network (the model designed in steps 2 and 3) is calculated through the return status of the device (combined with the number of basic test cases passed and the position information of the execution sorting of error cases).

[0131] At this point, a critic network is used to predict a baseline score based on the output of the actor network. The gradient of model optimization is calculated based on the baseline score and the reward function to update the network parameters. Repeat steps 2 to 4 until the model converges.

[0132] Specific operation example of step 4:

[0133] The reinforcement learning part uses the Actor-Critic framework, in which the Actor network is responsible for strategy selection (steps 2 and 3), and the Critic network is responsible for judging based on the selected strategy and generating a baseline score to reduce the variance of the overall model.

[0134] The actor network encodes the input vector sequence H according to the existing command line j , and then generate an output sequence O = MultiGraphPtrNetwork (H j ), used to predict the execution order of command lines. The critic network (Seq2Seq implementation) is used to evaluate the performance of the current strategy and generate a baseline value Used to reduce the variance of the policy gradient, the specific process is as follows:

[0135] (1) Action selection: At each time step t, the actor network selects an action based on the current strategy.

[0136] a t ~π θ (a|s t );

[0137] (2) Calculate the reward function: Reward function R t Defines the execution action a t Rewards received after

[0138] (3) Baseline value generated by critic network: The critic network is an encoder-decoder structure composed of two simple LSTM networks;

[0139] (4) Policy update: Use the policy gradient method to update the policy parameters θ. Specifically, the Actor-Critic method is used to update the policy parameters:

[0140] For the reward function part of reinforcement learning, the position information is introduced into the reward function, which not only takes into account the number of execution results but also the failure position distribution information.

[0141] Combined with two new reinforcement learning reward functions APHF and HFC, the APHF function takes into account the historical failure distribution information of the test cases, and the HFC function calculates the total number of historical failures of the test cases and gives more rewards to the test cases with more failures. Inspired by the idea of ​​APHF function, a new reward function is proposed according to the characteristics of this task. The formula is shown below.

[0142]

[0143] Where R t Represents positive reward. Its indicator is the result we get from sending the generated test case set. It can be the result of the test case passing or the coverage indicator of the test case. The larger the better. The second item is a reward function based on the distribution information of failed test cases. The more failed test cases are at the front, the smaller the reward will be. The last item is an adjustment item used to balance the reward value.

[0144] The new reward function combines positive incentives (the number of successful executions) and negative penalties (the distribution of error execution locations), which can effectively guide model learning and improve the quality of command line sequence generation. At the same time, by placing high penalties on errors at the beginning of the sequence, the model will pay more attention to the accuracy of the first few steps, ensuring that the command line sequence can be executed smoothly from the beginning.

[0145] The following are embodiments of the method of the present invention:

[0146] Suppose you want to test a new type of switch, which has multiple functional modules, such as port configuration, VLAN settings, routing configuration, etc., and there is a hierarchical relationship between these functional modules, and each functional module has its own command line.

[0147] When using the patented method, the automated test cases and their corresponding hierarchical information are first abstracted into a graph structure. For example, the port configuration is considered as one level, the VLAN setting is considered as another level, and the routing configuration is considered as another level. The dependencies between them are represented by the edges of the graph.

[0148] Then, different graph neural networks are used to encode these two parts respectively. For example: for the command lines "configureport1speed1000" (configure the speed of port 1 to 1000) and "createvlan10" (create VLAN10). Through the encoding of the graph neural network, for the command line "configureport1speed1000", the graph neural network will encode it according to its hierarchical information and its relationship with other command lines, and obtain a vector representation that contains the command line features and hierarchical information. Similarly, for the command line "createvlan10", similar encoding processing will be performed. These encoded vector representations can better mine the hierarchical features between different test cases, so that the model can more accurately understand the relationship between command lines and their role in communication equipment testing.

[0149] Afterwards, the command line vector embedded with hierarchical information is input into the pointer network. The bidirectional LSTM encoder in the pointer network will encode the relationship between the two command lines at different levels to obtain their hidden states.

[0150] Next, the pointer network will calculate the Attention score based on the hidden state of the encoder at each moment. For example, based on the logic and test requirements of the device, it may be considered more reasonable to execute "configureport1speed1000" first and then "createvlan10" in the current scenario. In this case, the pointer network will calculate the corresponding Attention score so that "configureport1speed1000" gets a higher score at a certain moment.

[0151] Finally, the pointer network will output the command line with the highest score, that is, according to the calculation results, it may output "configureport1speed1000" first, and then output "createvlan10". In this way, the pointer network generates the corresponding test case execution order based on the input command line and layer information.

[0152] Through the role of pointer network, the execution order of command lines can be better arranged according to the characteristics of communication equipment and test requirements, thereby improving the effectiveness and accuracy of test cases.

[0153] (The pointer network needs to be trained. The trained pointing network can sort the command line vectors embedded with hierarchical information to obtain the best execution order)

[0154] In summary, the technology of the present invention has the following characteristics:

[0155] 1. This solution can adapt to the CI / CD rapid iteration development process. In the rapid iteration CI / CD process, the system can automatically optimize in continuous interaction with the device to quickly adapt to changes in the device and protocol, and update and generate new test cases in real time.

[0156] 2. This solution can greatly improve the generalization ability of the model for communication equipment. For newly added test cases or new equipment, it can well capture the relationship between test cases, greatly improving the test coverage and the quality of test cases.

[0157] 3. This solution is simple and efficient, and has strong pertinence for automated test cases of communication equipment. It does not require excessive processing of test cases, and has strong reliability, portability and scalability.

[0158] Based on the method of the present invention, the embodiment of the present disclosure also provides a system corresponding to the above method, which includes:

[0159] The vector generation module is used to obtain the command lines of different test cases of the communication device and the hierarchical information corresponding to the command lines; the command lines and the hierarchical information are feature vectorized to obtain the hierarchical vector and the command line vector.

[0160] The classification module is used to classify command lines according to command line levels to obtain command line sets at different levels.

[0161] A graph structure building module is used to create a connection between the level vectors of the two levels for hierarchical information, if the two levels can be converted, to obtain a first graph data structure consisting of a level set; in the command line set corresponding to each level, a connection is created between the command line vectors to obtain a second graph data structure consisting of a command line set.

[0162] The information embedding module is used to input the first graph data structure, the second graph data structure and the corresponding hierarchical vector and command line vector into two different graph neural networks for multi-layer encoding, and then splice the feature vectors output from the two different graph neural networks to obtain a command line vector embedded with hierarchical information.

[0163] The test case generation module is used to input multiple command line vectors embedded with hierarchical information into the pointer network and output the optimal command line as a test case.

[0164] Based on the same inventive concept as the above disclosed content, an embodiment of the present disclosure also provides a device corresponding to the above method, which includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above method.

[0165] It should be noted that the electrical connection between the above-mentioned units does not necessarily mean the connection between lines. An indirect connection method can be applied to the embodiments of the present disclosure as long as the purpose of the present disclosure is achieved.

[0166] Based on the same inventive concept, the present disclosure also provides a computer storage medium on which executable instructions are stored. When the instructions are executed by a processor, the processor performs the above method.

[0167] Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for generating test cases for a communication device, characterized in that: include: Obtain command lines of different test cases of the software running on the communication device, as well as the level information corresponding to the command lines; Characterize the command line and level information into vectors to obtain level vectors and command line vectors; Classify the command lines according to the command line levels to obtain command line sets at different levels; For the hierarchical information, if the two hierarchical levels can be converted, a connection is created between the hierarchical vectors of the two hierarchical levels to obtain a first graph data structure consisting of a hierarchical set; in the command line set corresponding to each hierarchical level, a connection is created between the command line vectors to obtain a second graph data structure consisting of the command line set; The first graph data structure and the corresponding level vector, the second graph data structure and the corresponding command line vector are respectively input into two different graph neural networks for multi-layer encoding, and then the feature vectors output from the two different graph neural networks are concatenated to obtain a command line vector embedded with level information; A plurality of command line vectors with embedded hierarchical information are input into the pointer network, and the pointer network outputs the best execution order of the command lines. The command lines are spliced ​​according to the best execution order of the command lines to obtain the test cases of the communication equipment.

2. A method for generating test cases for a communication device according to claim 1, characterized in that: The command lines for obtaining different test cases of the software running on the communication device include: Extract the original command lines of different test cases from the source code of the software run by the communication device based on regular expressions, and obtain the hierarchical information of the original command lines according to the extraction position during extraction; Based on the extraction results, the useless original command lines are cleaned up to obtain a command line set; For the original command lines with parameter prompts in the command line set, according to the parameter condition matching algorithm, the original command lines with parameter prompts are spliced ​​into command lines for different test cases that can be executed.

3. A method for generating test cases for a communication device according to claim 2, characterized in that: If the parameter part of the original command line is marked with brackets; The algorithm for matching the parameter conditions is used to combine the command lines with the parameter prompts into an executable command line; including: Identify all command line segments in the current sequence that begin with the previous bracket and end with the next bracket. Process parameter logic. If the parameters are separated by |, keep one parameter. If the parameter is a range, replace the parameter with one of the range values. Recursively process the command line until there are no more argument flags.

4. A method for generating test cases for a communication device according to claim 1, characterized in that: The command lines are classified according to the command line levels, including: A random forest classification model is trained based on the hierarchical information of the command line set and the corresponding information of the command line set, and the random forest classification model is used to classify the command lines without labeled hierarchies.

5. The test case generation method for a communication device according to claim 1, characterized in that: The plurality of command line vectors embedded with hierarchical information are input into the pointer network, the pointer network outputs the best execution order of the command lines, and the command lines are spliced ​​according to the best execution order of the command lines; include: Set all command line vectors h that embed level information c Input into the encoder of the pointer network, at each time t the encoder generates a corresponding hidden state e t , after time step j, enters the decoder; The decoder is based on the hidden state d of the decoder at the current moment i , and the encoder hidden state e of the previous j time steps j , j∈(1,2,3…,n) performs additive attention mechanism to obtain the output at the current moment and an Attention score for each time period input Attention score Calculate a distribution probability p. The command line corresponding to the largest distribution probability p is the command line output by the decoder at the current moment. The output order of the command lines is the optimal execution order of the command lines. in: p(C i |C1,C2,C3…C i-1 ,P)=softmax(u i ) In the formula, v T , W1, W2 are learnable parameters that are continuously optimized during the iteration of the model.

6. A method for generating test cases for a communication device according to claim 5, characterized in that: Also includes: Optimizing the network parameters of the graph neural network and the pointer network includes: Send the test case of the communication device output by the pointer network to the communication device, and calculate the reward function based on the return state of the communication device; The test cases of the communication devices output by the pointer network are input into a critic network, and the critic network outputs a baseline score Calculate the gradient of model optimization based on the baseline score and reward function, thereby updating and optimizing the network parameters of the graph neural network and pointer network; The reward function R t for: Among them, R success For positive rewards, TS fail is the set of failed cases, rank(TS fail ) is the order of failed test cases, TS is the set of all test cases; The gradient of the computational model optimization is: in, is the gradient of the policy parameters, π θ is the policy function, a t is the action at time t.

7. A method for generating test cases for a communication device according to claim 6, characterized in that: The critic network includes an encoder-decoder structure consisting of two LSTM networks.

8. A test case generation system for communication equipment, characterized in that: include: A vector generation module is used to obtain command lines of different test cases of software running on the communication device, and the level information corresponding to the command lines; Characterize the command line and level information into vectors to obtain level vectors and command line vectors; The classification module is used to classify command lines according to the command line level to obtain command line sets at different levels; A graph structure building module is used for creating a connection between the level vectors of the two levels for the level information, if the two levels can be converted, to obtain a first graph data structure consisting of the level set; in the command line set corresponding to each level, creating a connection between the command line vectors, to obtain a second graph data structure consisting of the command line set; An information embedding module, used to input the first graph data structure and the corresponding hierarchical vector, the second graph data structure and the corresponding command line vector into two different graph neural networks for multi-layer encoding, and then concatenate the feature vectors output from the two different graph neural networks to obtain a command line vector embedded with hierarchical information; The test case generation module is used to input multiple command line vectors embedded with hierarchical information into the pointer network, the pointer network outputs the best execution order of the command lines, and the command lines are spliced ​​according to the best execution order of the command lines to obtain the test case of the communication device.

9. A test case generating device for a communication device, comprising: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the test case generation method for a communication device as described in any one of claims 1 to 7.

10. A computer storage medium having executable instructions stored thereon, wherein when the instructions are executed by a processor, the processor is enabled to implement the method according to any one of claims 1 to 7.

Images