Test case generation method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to a test case generation method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: extracting a plurality of behavior events from a target user log, and generating behavior event vectors according to behavior keywords in each behavior event; inputting the behavior event vector into an entity labeling model to obtain an entity keyword matched with a preset entity category and a non-entity keyword not matched with the preset entity category in each behavior keyword; determining the attention weight of the non-entity keyword and the entity keyword for the behavior event vector, and taking the target user log as a test case generation log under the condition that the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword; and generating a target test case according to each entity keyword in the plurality of test case generation logs. By adopting the method, the test case can be closer to the actual operation of the user.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a test case generation method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] Key Management Service (KMS) is a cloud security service whose functions include generating, importing, enabling, disabling, delayed deletion, and automatic rotation of keys, and supports data encryption and decryption and digital signature verification through API interfaces. In order to ensure the high security, compliance, reliability, and performance of KMS products, and to meet compliance and interoperability requirements, designing high-quality test cases is the key to ensuring effective testing.

[0003] Currently, test case generation mostly relies on manual design by testers. However, in actual use, users may have some operational behaviors that are difficult for testers to cover during design, resulting in insufficient test coverage. Summary of the invention

[0004] Based on this, it is necessary to provide a test case generation method, apparatus, computer equipment, storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a test case generation method. The method comprises:

[0006] Extracting multiple behavior events from the target user log, and generating a behavior event vector according to the behavior keywords in each of the behavior events;

[0007] Inputting the behavior event vector into an entity annotation model to obtain entity keywords that match a preset entity category and non-entity keywords that do not match the preset entity category among the behavior keywords;

[0008] Determine the attention weights of the non-entity keyword and the entity keyword for the behavior event vector, and when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, generate a log using the target user log as a test case;

[0009] Generate a target test case according to each entity keyword in the multiple test case generation logs.

[0010] In one embodiment, generating a behavior event vector according to the behavior keywords in each of the behavior events includes:

[0011] Extracting behavior keywords from each of the behavior events, and generating word embedding vectors for each of the behavior keywords;

[0012] Sequentially concatenating the word embedding vectors into a word embedding vector sequence, and inputting the word embedding vector sequence into a temporal feature extraction model to obtain a keyword vector for each of the behavior keywords;

[0013] A behavior event vector is determined according to each of the keyword vectors.

[0014] In one embodiment, determining a behavior event vector according to each of the keyword vectors includes:

[0015] Determine the weight of each keyword vector respectively, and sum each keyword vector according to the weight to obtain a key sentence vector;

[0016] The key sentence vector and each of the keyword vectors are sequentially concatenated to obtain a behavior event vector.

[0017] In one embodiment, determining the attention weights of the non-entity keyword and the entity keyword in the behavior event vector includes:

[0018] Performing fusion processing on the keyword vectors of each entity keyword to obtain a fused entity keyword vector;

[0019] Inputting the behavior event vector, the fused entity keyword vector and the keyword vector of each non-entity keyword into an attention weight extraction model; wherein the attention weight extraction model includes an attention layer;

[0020] Using the behavior event vector as the target output of the attention weight extraction model, using the fused entity keyword vector and the keyword vectors of each of the non-entity keywords as inputs of the attention weight extraction model, and iteratively training the attention weight extraction model until the output of the attention weight extraction model is consistent with the behavior event vector;

[0021] From the attention layer in the attention weight extraction model, an attention weight matrix is ​​obtained, and the attention weights corresponding to the fused entity keyword vectors in the attention weight matrix are used as the attention weights of each of the entity keywords for the behavior event vectors, and the attention weights corresponding to the keyword vectors of each of the non-entity keywords in the attention weight matrix are used as the attention weights of each of the non-entity keywords for the behavior event vectors.

[0022] In one embodiment, generating a target test case according to each entity keyword in the plurality of test case generation logs includes:

[0023] Obtain multiple test case generation logs, and for any first test case generation log, determine the similarity score of the first test case generation log according to the similarity between each entity keyword in the first test case generation log and each entity keyword in each second test case generation log; wherein the first test case generation log is any test case generation log, and the second test case generation log is a log in each of the test case generation logs except the first test case generation log;

[0024] According to the similarity scores of the test case generation logs, a target test case generation log is determined from the test case generation logs, and a target test case is generated according to the target test case generation log.

[0025] In one embodiment, generating a target test case according to the target test case generation log includes:

[0026] Classify each entity keyword in the target test case generation log to obtain the behavior category to which each entity keyword belongs;

[0027] For any of the behavior categories, generating a target behavior process corresponding to the behavior category according to each of the entity keywords corresponding to the behavior category;

[0028] Determine target test cases based on each of the target behavior processes.

[0029] In a second aspect, the present application also provides a test case generation device. The device comprises:

[0030] An extraction module, used to extract multiple behavior events from the target user log, and generate a behavior event vector according to the behavior keywords in each of the behavior events;

[0031] An input module, used for inputting the behavior event vector into an entity annotation model to obtain entity keywords that match the preset entity category and non-entity keywords that do not match the preset entity category among the behavior keywords;

[0032] A determination module, configured to determine the attention weights of the non-entity keyword and the entity keyword for the behavior event vector, and to generate a log using the target user log as a test case when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword;

[0033] A generation module is used to generate target test cases based on the entity keywords in the log generated by the plurality of test cases.

[0034] In one embodiment, the extraction module is further used to:

[0035] Extracting behavior keywords from each of the behavior events, and generating word embedding vectors for each of the behavior keywords;

[0036] Sequentially concatenating the word embedding vectors into a word embedding vector sequence, and inputting the word embedding vector sequence into a temporal feature extraction model to obtain a keyword vector for each of the behavior keywords;

[0037] A behavior event vector is determined according to each of the keyword vectors.

[0038] In one embodiment, the extraction module is further used to:

[0039] Determine the weight of each keyword vector respectively, and sum each keyword vector according to the weight to obtain a key sentence vector;

[0040] The key sentence vector and each of the keyword vectors are sequentially concatenated to obtain a behavior event vector.

[0041] In one embodiment, the determining module is further used to:

[0042] Performing fusion processing on the keyword vectors of each entity keyword to obtain a fused entity keyword vector;

[0043] Inputting the behavior event vector, the fused entity keyword vector and the keyword vector of each non-entity keyword into an attention weight extraction model; wherein the attention weight extraction model includes an attention layer;

[0044] Using the behavior event vector as the target output of the attention weight extraction model, using the fused entity keyword vector and the keyword vectors of each of the non-entity keywords as inputs of the attention weight extraction model, and iteratively training the attention weight extraction model until the output of the attention weight extraction model is consistent with the behavior event vector;

[0045] From the attention layer in the attention weight extraction model, an attention weight matrix is ​​obtained, and the attention weights corresponding to the fused entity keyword vectors in the attention weight matrix are used as the attention weights of each of the entity keywords for the behavior event vectors, and the attention weights corresponding to the keyword vectors of each of the non-entity keywords in the attention weight matrix are used as the attention weights of each of the non-entity keywords for the behavior event vectors.

[0046] In one embodiment, the generating module is further used for:

[0047] Obtain multiple test case generation logs, and for any first test case generation log, determine the similarity score of the first test case generation log according to the similarity between each entity keyword in the first test case generation log and each entity keyword in each second test case generation log; wherein the first test case generation log is any test case generation log, and the second test case generation log is a log in each of the test case generation logs except the first test case generation log;

[0048] According to the similarity scores of the test case generation logs, a target test case generation log is determined from the test case generation logs, and a target test case is generated according to the target test case generation log.

[0049] In one embodiment, the generating module is further used for:

[0050] Classify each entity keyword in the target test case generation log to obtain the behavior category to which each entity keyword belongs;

[0051] For any of the behavior categories, generating a target behavior process corresponding to the behavior category according to each of the entity keywords corresponding to the behavior category;

[0052] Determine target test cases based on each of the target behavior processes.

[0053] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, any one of the above methods is implemented.

[0054] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, any of the above methods is implemented.

[0055] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0056] The above-mentioned test case generation method, device, computer equipment, storage medium and computer program product obtain the user log actually generated in the production environment, and extract the behavior keywords in the user log, determine whether each behavior keyword is an entity keyword and the attention weight of each entity keyword for the behavior event vector. Since the attention weight can reflect the importance of the entity keyword in the behavior event vector, when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, it can be determined that the user log contains more entity information, and this user log can be used as a test case generation log to generate a test case. The embodiment of the present application uses a log containing more entity information in the production environment to generate a test case, which can not only make the test case cover the user's actual operation behavior, but also improve the quality of the test case. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a flow chart of a test case generation method in an embodiment;

[0058] Figure 2 is a flow chart of step 102 in one embodiment;

[0059] Figure 3 is a flow chart of step 206 in one embodiment;

[0060] Figure 4 is a flow chart of step 106 in one embodiment;

[0061] Figure 5 is a flow chart of step 108 in one embodiment;

[0062] Figure 6 is a flow chart of step 504 in one embodiment;

[0063] Figure 7 A schematic diagram of a flow chart of a test case generation method in an embodiment;

[0064] Figure 8 A structural block diagram of a test case generating device in an embodiment;

[0065] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] In one embodiment, Figure 1As shown, a test case generation method is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0068] Step 102: extract multiple behavior events from the target user log, and generate a behavior event vector according to the behavior keywords in each behavior event.

[0069] In an embodiment of the present application, the server may periodically read user logs from the KMS service, and the target user logs may be partial logs filtered out from the read user logs, such as logs with a length greater than a preset length, logs with no missing values, etc.

[0070] User logs include multiple behavior events. A behavior event is used to record an operation performed by a user in the KMS service, such as key generation operations, key activation operations, etc. Behavior keywords refer to text information related to user behavior in behavior events. For example, in a behavior event, the timestamp of a user's execution of a certain behavior, the IP address of the user's terminal (since the user needs to access the KMS system through the terminal and initiate an execution request for the behavior to the KMS system, the IP address of the terminal may be recorded in the log), and the behavior performed by the user are recorded at the same time. The behavior keyword is the text information corresponding to the behavior performed by the user.

[0071] Behavior keywords can be extracted through the trained text classification model. When training the text classification model, sample user logs can be obtained, and each part of the sample user logs can be labeled as a behavior keyword, and then the text classification model can be trained through the labeled sample user logs. In the application, the target user log can be input into the text classification model to obtain the classification of each word in the target user log by the text classification model, and then the words classified as keywords can be used as behavior keywords.

[0072] Based on the behavior keywords in each behavior event, a behavior event vector for representing each behavior event in the user log can be generated. For example, the behavior event vector can be obtained by converting the behavior keywords in each behavior event into a vector and then summing, concatenating, averaging, etc. the vectors.

[0073] In one embodiment, the behavior event vector may carry the timing information of each behavior keyword. Figure 2 As shown:

[0074] Step 202, extracting behavior keywords from each behavior event, and generating word embedding vectors for each behavior keyword;

[0075] Step 204, sequentially concatenate the word embedding vectors into a word embedding vector sequence, input the word embedding vector sequence into a temporal feature extraction model, and obtain a keyword vector for each behavior keyword;

[0076] Step 206: Determine a behavior event vector based on each keyword vector.

[0077] In the embodiment of the present application, after obtaining the behavior keywords, considering that the temporal characteristics of each word generally need to be considered in the entity labeling task, the vector with the temporal characteristics can be used as the vector corresponding to the behavior keyword.

[0078] First, any word embedding algorithm (such as Word2Vec algorithm, GloVe algorithm) can be used to generate word embedding vectors corresponding to each behavior keyword. After the word embedding vectors are sequentially spliced ​​into a word embedding vector sequence according to the order of arrangement of each behavior keyword in the behavior event, any temporal feature extraction model that can extract bidirectional temporal features (such as BiLSTM (Bidirectional Long Short-Term Memory)) can be used to extract the keyword vector of each behavior keyword. In the BiLSTM model, the model will process the word embedding vector sequence in forward order through the forward LSTM branch and process the word embedding vector sequence in reverse order through the backward LSTM branch. For any word embedding vector, when the model processes the word embedding vector, the output of the forward LSTM hidden layer can be regarded as the previous context vector of the word embedding vector, and the output of the backward LSTM hidden layer can be regarded as the following context vector of the word embedding vector. Combining these two vectors can obtain the keyword vector of the behavior keyword with temporal features, as shown in formula (1):

[0079] Formula (I)

[0080] in, is the keyword vector of the t-th behavior keyword, is the hidden layer output of the forward LSTM for the t-th behavior keyword, It is the hidden layer output of the backward LSTM for the t-th behavior keyword.

[0081] After obtaining each keyword vector, you can use methods such as summing up, averaging, and sequential concatenation to fuse each keyword vector to obtain a behavior event vector. Or, considering that each keyword vector has different levels of meaning (the meaning of the word itself, and the meaning of the sentence after concatenation), you can also use a hierarchical vector method to construct a behavior event vector based on each keyword vector. Figure 3 As shown:

[0082] Step 302, respectively determine the weight of each keyword vector, and sum up each keyword vector according to the weight to obtain a key sentence vector;

[0083] Step 304: sequentially concatenate the key sentence vector and each keyword vector to obtain a behavior event vector.

[0084] In the embodiment of the present application, the weight of the keyword vector is used to characterize the importance of the behavior keyword in the entire behavior event. Since the extracted behavior keywords are all words related to the user behavior, and the words related to the user behavior in the log are usually a sentence, this sentence may contain the behavior verb itself, the description of the behavior object, and other elements that make up the sentence. Therefore, the importance of each behavior keyword to the behavior event is different, and the importance of each behavior keyword to the behavior event can be obtained through natural language processing and other methods.

[0085] The weight of the keyword vector can be obtained through a trained text processing model. Exemplarily, each sample keyword vector of a sample user log can be obtained, and the target output corresponding to the sample keyword vector can be pre-built. Since the purpose of the embodiment of the present application is to extract more important vectors from the keyword vectors, and a certain keyword vector is more important, which means that the keyword vector has a greater contribution to the meaning of the behavior event, the target output corresponding to the sample keyword vector can be set to a summary of the behavior event.

[0086] Input each sample keyword vector into a text processing model with an attention layer in the order of each sample keyword in the sample user log, and make the text processing model output a sample summary generated according to each sample keyword, and the attention weight assigned to each sample keyword vector by the attention layer. The model is trained based on the difference between the sample summary and the target summary, and the attention weight is adjusted during the training process. As shown in formula (II):

[0087] Formula (II)

[0088] in, is the attention weight, is the keyword vector, It is a parameter that needs to be adjusted during training.

[0089] After the text processing model is trained, each keyword vector can be input into the text processing model in the application to obtain the attention weight corresponding to each keyword vector output by the text processing model. After normalizing the weight with the softmax function, the weight of the keyword vector can be obtained. For example, formula (3):

[0090] Formula (III)

[0091] in, is the weight of the kth keyword vector, and n is the total number of keyword vectors.

[0092] By weighting and summing up each keyword vector according to its weight, we can get the key sentence vector, as shown in formula (IV):

[0093] Formula (IV)

[0094] Among them, B is the key sentence vector.

[0095] By sequentially concatenating the key sentence vector and each keyword vector, the behavior event vector can be obtained.

[0096] Step 104 , input the behavior event vector into the entity annotation model to obtain entity keywords that match the preset entity category and non-entity keywords that do not match the preset entity category in each behavior keyword.

[0097] In the embodiments of the present application, an entity refers to a noun with a specific meaning, and a preset entity category is an entity category that is pre-set by a technician and is more important to the test case, such as the user's operation object (key, certificate, key alias, etc.), operation type (login, create, modify, obtain, etc.), data field filled in during operation, etc. Non-entity refers to a word that does not belong to an entity.

[0098] The entity labeling model can be a model trained based on any model that can perform entity labeling tasks. When training the entity labeling model, sample user logs can be obtained and each entity keyword in the sample user logs can be labeled. The previously obtained behavior event vector is used as the input of the model, and whether each word in the behavior keyword is an entity keyword or a non-entity keyword is used as the output of the model. The model is trained and adjusted based on the annotation of the behavior keywords and the output of the model for the behavior keywords. In the application, the trained entity labeling model can be used to identify whether the behavior keyword is an entity or a non-entity.

[0099] For example, the entity labeling model can be constructed by referring to the principle of the LSTM model. The LSTM model contains a forget gate, an input gate, and an output gate. The forget gate is responsible for determining the state to be discarded from the current cell state, the input gate is used to determine which information in the current cell state will be added to the new cell state, and the output gate is used to determine which part of the new cell state will be output. The forget gate and the input gate are used here to construct the entity labeling model. The formula is shown in formula (V):

[0100] Formula (V)

[0101] in, It is a temporary state after being processed by the forget gate, used to represent the cell state at the last moment The retention ratio of each element in is the sigmoid activation function, is the weight of the forget gate, is the hidden layer output of the model at the previous moment, is the input at this moment, is the bias of the forget gate. [⋅,⋅] represents the concatenation of vectors.

[0102] is the candidate cell state after the new information is introduced through the input gate, is the input gate weight vector, which is used to regulate the contribution of these new information to the cell state. and is the weight of the input gate, and is the bias of the input gate. is the hyperbolic tangent function.

[0103] The old cell state after the weighting of the forget gate , the new candidate cell state after input gate weighting (⊙ represents element-by-element multiplication) After adding element-by-element, the latest cell state can be obtained, which is the vector used for entity keyword and non-entity keyword recognition. (See formula (VI)):

[0104] Formula (VI)

[0105] By labeling each behavior keyword according to the vector, entity keywords and non-entity keywords can be obtained.

[0106] Step 106, determining the attention weights of the non-entity keywords and the entity keywords for the behavior event vector, and when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, taking the behavior event vector as the target behavior event vector.

[0107] In an embodiment of the present application, the attention weight of each non-entity keyword and each entity keyword for the behavior event vector can be determined, which is equivalent to the importance of each non-entity keyword and each entity keyword in generating the behavior event vector. If the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, it indicates that there are more entity keywords in the behavior event, and the target user log is more important for the generation of test cases. If the attention weight of the non-entity keyword is greater than or equal to the attention weight of the entity keyword, it indicates that there are fewer entity keywords in the behavior event, and the target user log is less important for the generation of test cases.

[0108] The attention weight extraction model can be trained to obtain the attention weight. Specifically, an attention layer can be set in the attention weight extraction model, and the input of the attention weight extraction model is the keyword vector of each entity keyword and the keyword vector of each non-entity keyword, so that the output of the attention weight extraction model is a behavior event vector. The attention weight extraction model is trained so that the output of the attention weight extraction model can accurately be the original behavior event vector, so that the attention weight of each entity keyword and each non-entity keyword can be obtained by obtaining the attention weight matrix in the attention layer.

[0109] Furthermore, since the purpose of this step is to determine whether there are enough entity keywords in the behavior event, it is not important whether a single entity keyword has enough contribution to the behavior event vector. Therefore, the keyword vectors of each entity keyword can be fused, and the fused entity keyword vector obtained after the fusion process is used to calculate the attention weight corresponding to the entity keyword. Specifically, see Figure 4 As shown:

[0110] Step 402: fusing the keyword vectors of the entity keywords to obtain a fused entity keyword vector.

[0111] Step 404, input the behavior event vector, the fused entity keyword vector and the keyword vector of each non-entity keyword into the attention weight extraction model; wherein the attention weight extraction model includes an attention layer.

[0112] Step 406, taking the behavior event vector as the target output of the attention weight extraction model, taking the fused entity keyword vector and the keyword vector of each non-entity keyword as the input of the attention weight extraction model, and iteratively training the attention weight extraction model until the output of the attention weight extraction model is consistent with the behavior event vector;

[0113] Step 408, obtain the attention weight matrix from the attention layer in the attention weight extraction model, use the attention weights corresponding to the fused entity keyword vectors in the attention weight matrix as the attention weights of each entity keyword for the behavior event vector, and use the attention weights corresponding to the keyword vectors of each non-entity keyword in the attention weight matrix as the attention weights of each non-entity keyword for the behavior event vector.

[0114] In the embodiment of the present application, the fusion processing method can be summation, averaging, sequential splicing, etc. The embodiment of the present application does not specifically limit how to fuse the various keyword vectors.

[0115] The attention weight extraction model includes an attention layer, and the attention weight extraction model is used to generate a behavior event vector based on the fusion entity keyword vector and the keyword vector of each non-entity keyword. The attention weight extraction model can only be preliminarily trained before application, because the ultimate purpose of using the attention weight extraction model is not to obtain the precise behavior event vector output by it, but to obtain the output of its attention layer.

[0116] When applied, the behavior event vector, the fused entity keyword vector, and the keyword vector of each non-entity keyword are input into the attention weight extraction model. Then, according to the behavior event vector, the attention weight extraction model is iteratively trained until the output of the attention weight extraction model is consistent with the behavior event vector. At this time, the attention weight matrix in the attention layer can be obtained, which represents the importance of the fused entity keyword vector and the keyword vector of the non-entity keyword to the behavior event vector when generating the behavior event vector.

[0117] Since the purpose of this step is to obtain the attention weight matrix rather than optimize the model, the model does not need to be saved after the iterative training of the model is completed, so as to increase the speed of completing the iterative training of the model when processing other target user logs next time.

[0118] The weight of the corresponding fused entity keyword vector in the attention weight matrix is ​​used as the attention weight of the entity keyword. The weight of the keyword vector corresponding to the non-entity keyword in the attention weight matrix is ​​used as the attention weight of each non-entity keyword. If the attention weight of the entity keyword is greater than the attention weights of all non-entity keywords, the target user log is used as a test case to generate a log. If the attention weight of the entity keyword is less than or equal to the attention weight of any non-entity keyword, the target user log is discarded.

[0119] Step 108, generating a target test case based on the entity keywords in the multiple test case generation logs.

[0120] In the embodiment of the present application, after collecting enough test case generation logs, target test cases can be generated according to entity keywords in the test case generation logs. For example, an automatic operation process for the KMS system can be generated according to entity keywords in the test case generation log, and the automatic operation process can be used as the target test case.

[0121] In one embodiment, the target test case generation log can be filtered out from each test case generation log according to the similarity between each test case generation log, such as Figure 5 As shown, including:

[0122] Step 502, obtaining multiple test case generation logs, for any first test case generation log, determining a similarity score of the first test case generation log according to the similarity between each entity keyword in the first test case generation log and each entity keyword in each second test case generation log; wherein the first test case generation log is any test case generation log, and the second test case generation log is a log in each test case generation log except the first test case generation log;

[0123] Step 504 , determining a target test case generation log from each test case generation log according to the similarity score of each test case generation log, and generating a target test case according to the target test case generation log.

[0124] In the embodiment of the present application, the fused entity keyword vector corresponding to the first test case generation log can be obtained, and the fused entity keyword vector corresponding to the second test case generation log can be obtained, and the similarity between the two fused entity keyword vectors is calculated as the similarity between the first test case generation log and the second test case generation log. Since the fused entity keyword vector contains all entity keyword information in the log, and since the keyword vectors of these entity keywords contain time series features, the fused entity keyword vector can not only reflect all entity keywords in the log, but also reflect the connection relationship between these entity keywords, and can better represent the information of the entire log.

[0125] According to the similarity between the first test case generation log and each second test case generation log, a similarity score of the first test case generation log can be determined. The similarity score is used to characterize the overall similarity between the first test case generation log and other second test case generation logs. For example, the similarity score can be obtained by summing and averaging the respective similarities.

[0126] According to the similarity scores of the logs generated by each test case, the target test case generation log can be determined. The embodiment of the present application does not limit which test case generation logs are used as the target test case generation logs, and can be selected by those skilled in the art according to needs. For example, when it is necessary to generate a test case that can represent the user's routine operations, a test case generation log with a higher similarity score can be selected as the target test case generation log. When it is necessary to generate a test case that can represent an edge case or an abnormal situation, a test case generation log with a lower similarity score can be selected as the target test case generation log.

[0127] In one embodiment, Figure 6 As shown, the target test case is generated according to the target test case generation log, including:

[0128] Step 602, classify each entity keyword in the target test case generation log to obtain the behavior category to which each entity keyword belongs;

[0129] Step 604, for any behavior category, according to each entity keyword corresponding to the behavior category, generate a target behavior process corresponding to the behavior category;

[0130] Step 606, determining target test cases according to each target behavior process.

[0131] In the embodiment of the present application, the trainable classification model can classify each entity keyword to determine the behavior type contained in the target test case generation log. Exemplarily, the behavior type can be divided into five categories: key creation, key management, key version management, cryptographic operations, and alias management. During classification, the vector input to the classification model can be a keyword vector, and the output is one or more behavior types that the keyword vector may belong to.

[0132] A multi-layer perceptron can be used to classify keyword vectors. The cross entropy loss function is used as the loss function for training the multi-layer perceptron, as shown in formula (VII):

[0133] Formula (VII)

[0134] Among them, Loss is the loss, n is the total number of samples used for training, T is the total number of behavior types, Set to 1 when the model's classification c of sample i matches the true classification of sample i, otherwise set to 0; is the probability that sample i belongs to category c.

[0135] After obtaining each behavior category in the target test case generation log, the target behavior process can be generated according to each entity keyword belonging to the behavior category. The natural language generation model can be used to map the keyword vector of each entity keyword back to natural language to obtain the target behavior process. Since the keyword vector carries time sequence information, this method can retain the order information of each entity keyword in the original log, thereby improving the authenticity of the test case simulating user operations.

[0136] Each target behavior process may be used as a target test case, or the target behavior processes may be spliced ​​together to obtain a target test case for simulating multiple operations performed continuously by a user.

[0137] The test case generation method provided by the embodiment of the present application obtains the user log actually generated in the production environment, and extracts the behavior keywords in the user log, determines whether each behavior keyword is an entity keyword and the attention weight of each entity keyword for the behavior event vector. Since the attention weight can reflect the importance of the entity keyword in the behavior event vector, when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, it can be determined that the user log contains more entity information, and this user log can be used as a test case generation log to generate a test case. The embodiment of the present application uses a log containing more entity information in the production environment to generate a test case, which can not only make the test case cover the user's actual operating behavior, but also improve the quality of the test case.

[0138] In one embodiment, Figure 7 As shown, a test case generation method is provided, comprising:

[0139] S1, preprocess the target user logs, extract behavior keywords, use word embedding algorithm to represent the behavior keywords, and obtain word embedding vectors.

[0140] Data preprocessing includes but is not limited to deleting missing data, data with incorrect format and content, etc. The word embedding algorithm uses the GloVe algorithm. In this step, the behavior keywords can be initially divided into entities and non-entities through classification, etc., so as to improve the speed of entity identification in the future.

[0141] S2, use bidirectional LSTM to build a behavior keyword-sentence vector generation module, extract the context representation of each behavior keyword in the behavior sentence, and obtain the keyword vector of the behavior keyword.

[0142] S3, according to the importance of each behavior keyword in the behavior sentence, assign different weights to the behavior keywords, extract the information of each behavior keyword according to the weight and merge them as the key sentence vector.

[0143] Through the above method, the temporal information in the user behavior can be retained, and by assigning different attention weights to each behavior keyword, the key information in the user behavior can also be mined.

[0144] S4: Based on the keyword vectors and key sentence vectors processed in S3, an entity annotation model is constructed. The entity annotation model is used to calculate the keyword vectors of entity information and the keyword vectors of non-entity information.

[0145] S5, fuses the entity information, and inputs the fused entity information and the original non-entity information into the entity discriminator for entity discrimination again, thereby reducing the degree of confusion of the model for entity keywords and non-entity keywords, and extracting a more accurate vector representation of user entity behavior.

[0146] Among them, the entity discriminator can be implemented by using the aforementioned attention weight extraction model. Here, the non-entity information can also be fused in the same way as the entity information. The entity discriminator outputs the entity keyword when the attention score of the entity keyword is higher, and outputs the non-entity keyword when the attention score of the non-entity keyword is higher.

[0147] S6, input the entity information into the multi-layer perceptron, use the cross entropy loss function to optimize the model, and realize the generation of key management service test cases.

[0148] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0149] Based on the same inventive concept, the embodiment of the present application also provides a test case generation device for implementing the test case generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more test case generation device embodiments provided below can refer to the limitations of the test case generation method above, and will not be repeated here.

[0150] In one embodiment, Figure 8 As shown, a test case generating device 800 is provided, comprising: an extraction module 802, an input module 804, a determination module 806, and a generation module 808, wherein:

[0151] An extraction module 802 is used to extract multiple behavior events from the target user log and generate a behavior event vector according to the behavior keywords in each of the behavior events;

[0152] An input module 804 is used to input the behavior event vector into an entity annotation model to obtain entity keywords that match a preset entity category and non-entity keywords that do not match the preset entity category in each of the behavior keywords;

[0153] A determination module 806, configured to determine the attention weights of the non-entity keyword and the entity keyword for the behavior event vector, and when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, generate a log using the target user log as a test case;

[0154] The generating module 808 is used to generate a target test case according to each entity keyword in the log of the plurality of test cases.

[0155] The test case generation device provided by the embodiment of the present application obtains the user log actually generated in the production environment, and extracts the behavior keywords in the user log, determines whether each behavior keyword is an entity keyword and the attention weight of each entity keyword for the behavior event vector. Since the attention weight can reflect the importance of the entity keyword in the behavior event vector, when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, it can be determined that the user log contains more entity information, and this user log can be used as a test case generation log to generate a test case. The embodiment of the present application uses a log containing more entity information in the production environment to generate a test case, which can not only make the test case cover the user's actual operation behavior, but also improve the quality of the test case.

[0156] In one embodiment, the extraction module 802 is further used to:

[0157] Extracting behavior keywords from each of the behavior events, and generating word embedding vectors for each of the behavior keywords;

[0158] Sequentially concatenating the word embedding vectors into a word embedding vector sequence, and inputting the word embedding vector sequence into a temporal feature extraction model to obtain a keyword vector for each of the behavior keywords;

[0159] A behavior event vector is determined according to each of the keyword vectors.

[0160] In one embodiment, the extraction module 802 is further used to:

[0161] Determine the weight of each keyword vector respectively, and sum each keyword vector according to the weight to obtain a key sentence vector;

[0162] The key sentence vector and each of the keyword vectors are sequentially concatenated to obtain a behavior event vector.

[0163] In one embodiment, the determining module 806 is further configured to:

[0164] Performing fusion processing on the keyword vectors of each entity keyword to obtain a fused entity keyword vector;

[0165] Inputting the behavior event vector, the fused entity keyword vector and the keyword vector of each non-entity keyword into an attention weight extraction model; wherein the attention weight extraction model includes an attention layer;

[0166] Using the behavior event vector as the target output of the attention weight extraction model, using the fused entity keyword vector and the keyword vectors of each of the non-entity keywords as inputs of the attention weight extraction model, and iteratively training the attention weight extraction model until the output of the attention weight extraction model is consistent with the behavior event vector;

[0167] From the attention layer in the attention weight extraction model, an attention weight matrix is ​​obtained, and the attention weights corresponding to the fused entity keyword vectors in the attention weight matrix are used as the attention weights of each of the entity keywords for the behavior event vectors, and the attention weights corresponding to the keyword vectors of each of the non-entity keywords in the attention weight matrix are used as the attention weights of each of the non-entity keywords for the behavior event vectors.

[0168] In one embodiment, the generating module 808 is further configured to:

[0169] Obtain multiple test case generation logs, and for any first test case generation log, determine the similarity score of the first test case generation log according to the similarity between each entity keyword in the first test case generation log and each entity keyword in each second test case generation log; wherein the first test case generation log is any test case generation log, and the second test case generation log is a log in each of the test case generation logs except the first test case generation log;

[0170] According to the similarity scores of the test case generation logs, a target test case generation log is determined from the test case generation logs, and a target test case is generated according to the target test case generation log.

[0171] In one embodiment, the generating module 808 is further configured to:

[0172] Classify each entity keyword in the target test case generation log to obtain the behavior category to which each entity keyword belongs;

[0173] For any of the behavior categories, generating a target behavior process corresponding to the behavior category according to each of the entity keywords corresponding to the behavior category;

[0174] Determine target test cases based on each of the target behavior processes.

[0175] Each module in the above device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0176] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a test case generation method is implemented.

[0177] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0178] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0179] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0180] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0181] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0182] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0183] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0184] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A test case generation method, characterized in that: The method comprises: Extracting multiple behavior events from the target user log, and generating a behavior event vector according to the behavior keywords in each of the behavior events; Inputting the behavior event vector into an entity annotation model to obtain entity keywords that match a preset entity category and non-entity keywords that do not match the preset entity category among the behavior keywords; Determine the attention weights of the non-entity keyword and the entity keyword for the behavior event vector, and when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword, generate a log using the target user log as a test case; Generate a target test case according to each entity keyword in the multiple test case generation logs.

2. The method according to claim 1, characterized in that The step of generating a behavior event vector according to the behavior keywords in each of the behavior events includes: Extracting behavior keywords from each of the behavior events, and generating word embedding vectors for each of the behavior keywords; Sequentially concatenating the word embedding vectors into a word embedding vector sequence, and inputting the word embedding vector sequence into a temporal feature extraction model to obtain a keyword vector for each of the behavior keywords; A behavior event vector is determined according to each of the keyword vectors.

3. The method according to claim 2, characterized in that The determining of the behavior event vector according to each of the keyword vectors includes: Determine the weight of each keyword vector respectively, and sum each keyword vector according to the weight to obtain a key sentence vector; The key sentence vector and each of the keyword vectors are sequentially concatenated to obtain a behavior event vector.

4. The method according to claim 2 or 3, characterized in that: The determining of the attention weights of the non-entity keyword and the entity keyword in the behavior event vector includes: Performing fusion processing on the keyword vectors of each entity keyword to obtain a fused entity keyword vector; Inputting the behavior event vector, the fused entity keyword vector and the keyword vector of each non-entity keyword into an attention weight extraction model; wherein the attention weight extraction model includes an attention layer; Using the behavior event vector as the target output of the attention weight extraction model, using the fused entity keyword vector and the keyword vectors of each of the non-entity keywords as inputs of the attention weight extraction model, and iteratively training the attention weight extraction model until the output of the attention weight extraction model is consistent with the behavior event vector; From the attention layer in the attention weight extraction model, an attention weight matrix is ​​obtained, and the attention weights corresponding to the fused entity keyword vectors in the attention weight matrix are used as the attention weights of each of the entity keywords for the behavior event vectors, and the attention weights corresponding to the keyword vectors of each of the non-entity keywords in the attention weight matrix are used as the attention weights of each of the non-entity keywords for the behavior event vectors.

5. The method according to claim 1, characterized in that The step of generating a target test case according to each entity keyword in the multiple test case generation logs includes: Obtain multiple test case generation logs, and for any first test case generation log, determine the similarity score of the first test case generation log according to the similarity between each entity keyword in the first test case generation log and each entity keyword in each second test case generation log; wherein the first test case generation log is any test case generation log, and the second test case generation log is a log in each of the test case generation logs except the first test case generation log; According to the similarity scores of the test case generation logs, a target test case generation log is determined from the test case generation logs, and a target test case is generated according to the target test case generation log.

6. The method according to claim 5, characterized in that The step of generating a target test case according to the target test case generation log comprises: Classify each entity keyword in the target test case generation log to obtain the behavior category to which each entity keyword belongs; For any of the behavior categories, generating a target behavior process corresponding to the behavior category according to each of the entity keywords corresponding to the behavior category; Determine target test cases based on each of the target behavior processes.

7. A test case generating device, characterized in that: The device comprises: An extraction module, used to extract multiple behavior events from the target user log, and generate a behavior event vector according to the behavior keywords in each of the behavior events; An input module, used for inputting the behavior event vector into an entity annotation model to obtain entity keywords that match the preset entity category and non-entity keywords that do not match the preset entity category among the behavior keywords; A determination module, configured to determine the attention weights of the non-entity keyword and the entity keyword for the behavior event vector, and to generate a log using the target user log as a test case when the attention weight of the entity keyword is greater than the attention weight of the non-entity keyword; A generation module is used to generate target test cases based on the entity keywords in the log generated by the plurality of test cases.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images