A micro-service system test case dynamic sequencing method based on semantic similarity

Abstract

This invention discloses a dynamic sorting method for microservice test cases based on semantic similarity. It utilizes the testing strategy characteristics of microservice systems to statically prioritize test cases according to their importance and the impact of test case failures on the microservice system. Furthermore, it performs data preprocessing operations such as word segmentation on the test cases, and uses the Word2Vec model to convert test cases into embedded vectors to obtain the similarity between test cases. Based on the static priority and semantic similarity information between test cases, the priority of test cases is dynamically adjusted during execution.

Description

Technical Field

[0001] This invention patent belongs to the field of test case sorting technology, and relates to a dynamic sorting method for test cases of microservice systems based on semantic similarity. Background Technology

[0002] Test Case Prioritization (TCP) is one of the most commonly used techniques to speed up testing. TCP prioritizes test cases according to predetermined testing objectives without reducing the number of test cases, allowing higher-priority test cases to be executed first. The goal is to detect faults as early as possible, accelerate fault localization, and thus reduce testing costs.

[0003] The main TCP methods currently available are:

[0004] Search-based TCP technology was first proposed by Google engineer James A. Whittaker in 2006. This technology, based on genetic algorithms and metaheuristics, is used for automating the prioritization of test cases. Search-based test case prioritization uses optimization algorithms to determine the execution order of test cases, thereby discovering defects more quickly and improving testing efficiency. Its main advantages include:

[0005] 1. Maximizes testing efficiency: Search-based test case ranking technology can search for the optimal test sequence based on a specific objective function for different software systems, thereby effectively improving testing efficiency.

[0006] 2. Capable of solving nonlinear multi-objective problems: This technique can search for multiple objective functions and can handle complex nonlinear multi-objective optimization problems.

[0007] 3. It can improve test coverage: After test cases are sorted, test cases that can cover more program paths or modules will be executed first, thereby gradually improving test coverage.

[0008] However, search-based test case ranking techniques also have some drawbacks, such as:

[0009] 1. High computational cost: Since the search algorithm requires a lot of computational resources, it may take a long time to obtain the optimal solution when the program is large or requires high coverage.

[0010] 2. Results are affected by heuristic functions: The results of search-based test case ranking techniques are often affected by the design of heuristic functions. If the design is poor, it may lead to the search for suboptimal or even incorrect sequences.

[0011] 3. Dependence on objective function: Sorting techniques depend on a specific, computable objective function, but sometimes the objective function may not reflect the actual test requirements, so the optimal sorting result may not be obtained.

[0012] Coverage-based TCP technology was first proposed by Professor Mary Jean Harrold of the University of Texas at Austin in 1994. This technique, based on the concepts of code coverage and program slicing, is used to determine the execution order of test cases, thereby discovering defects in the program more quickly. This method aims to prioritize test cases with higher coverage, ensuring complete system coverage. However, this method cannot comprehensively cover all situations and ignores the relationships between test cases, potentially generating a large number of redundant test cases. Furthermore, this method relies heavily on source code analysis and often fails to achieve good results when the project code is unavailable.

[0013] Similarity-based TCP technology was first proposed by Gil and Yehudai in 1993. This technology uses a similarity metric to categorize test cases and then prioritizes test cases within each category to more effectively discover defects and reduce redundant test cases. Software testing often uses a reward-diversity approach, meaning higher priority can be assigned to test cases with the greatest differences. These highly differentiated test cases target significantly different functionalities, enabling faster detection of a wider range of errors. Furthermore, it only requires knowledge of the test cases themselves, without overly relying on the source code, making it widely applicable.

[0014] Most existing TCP technologies rely on test case coverage information, execution history information, and source code information of the system under test. These methods have poor generalization ability and are applicable to only a limited range of scenarios.

[0015] Similarity-based TCP technology only needs to measure the similarity between test cases, making it suitable for most testing scenarios. Currently, most similarity-based TCP methods are general-purpose test case ranking algorithms. However, for the specific application scenario of test case ranking in microservice systems, adjustments need to be made when applying similarity-based TCP methods to achieve dynamic ranking. Summary of the Invention

[0016] The purpose of this invention is to propose a dynamic sorting method for test cases in microservice systems based on semantic similarity.

[0017] The technical solution to achieve the purpose of this invention is: a dynamic sorting method for microservice test cases based on semantic similarity, comprising the following steps:

[0018] Step 1) Analyze the characteristics of microservice functional testing from unit testing, contract testing, integration testing, component testing and end-to-end testing; analyze the characteristics of microservice non-functional testing from coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing and cohesion testing; and statically prioritize test cases according to their importance and the impact of test case failures on the system.

[0019] Step 2) Preprocess the test cases to generate a numerical representation of the test case sequence for similarity assessment, and generate the embedding vector corresponding to the test case sequence through semantic analysis.

[0020] Step 3) Based on the priority of static test cases, select high-priority test cases to execute. Based on the feedback of the execution results, adjust the priority of test cases that are not similar to the test cases that are found to have errors.

[0021] Further, in step 1, the characteristics of microservice functional testing are analyzed from the perspectives of unit testing, contract testing, integration testing, component testing, and end-to-end testing. The characteristics of microservice non-functional testing are analyzed from the perspectives of coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing, and cohesion testing. Test cases are statically prioritized based on their importance and the impact of test case failures on the system, with functional tests having higher priority than non-functional tests. Specifically:

[0022] a) The priorities in functional testing, from highest to lowest, are as follows:

[0023] Unit tests to verify external communication and interaction with external services; unit tests to verify data access and interaction with external databases; consumer-side contract tests that define requests and expected responses between consumers and providers through streaming APIs; provider-side contract tests that simulate consumers sending requests to providers based on contract files; integration tests for interaction scenarios between multiple microservices; component tests to verify the correctness of a specific component or module when combined with other components or modules; and end-to-end tests for the entire microservice application to verify whether it meets user needs and achieves overall business goals.

[0024] b) The priorities for non-functional testing, from highest to lowest, are as follows:

[0025] Availability testing verifies the number of critical service instances; fault tolerance testing measures system capabilities by determining whether service circuit breaking, rate limiting, degradation, and load balancing mechanisms are implemented; performance testing measures microservice response time, throughput, capacity, and resource utilization; testing the strength of dependencies between services; and cohesion testing measures whether microservices implement a complete domain model business within a single service.

[0026] Further, in step 2, the test cases are preprocessed to generate a numerical representation of the test case sequence for similarity assessment. Semantic analysis methods are used to generate embedding vectors corresponding to the test case sequence, reflecting the similarity between test cases. The specific method is as follows:

[0027] a) Analyze the characteristics of the test cases, perform data preprocessing, and generate a numerical representation of the test case sequence for similarity assessment:

[0028] For test cases in text format, a third-party library is called to remove stop words, punctuation marks, and tags from the original test case dataset to generate clean test cases. For test cases in code format, the ASTParser parser is called to parse the test case code, transforming it into an abstract syntax tree (AST). Then, the program token is obtained by traversing the nodes in the AST to generate clean test cases.

[0029] For the two types of clean test cases that have been processed, perform word segmentation to obtain a list of words corresponding to the test cases. Each word is used as a key, and the number of existing keys is used as a value. This ensures that each word is assigned a unique identifier, i.e., an index. Then, a mapping relationship between words and indexes is established, generating a numerical representation of the test case sequence for subsequent similarity evaluation.

[0030] b) Generate embedding vectors corresponding to the test case sequence using semantic analysis methods:

[0031] The Word2Vec model is used to convert the test case sequence into an embedding vector. For each test case sequence, a center word C is randomly selected, and all context words within an N-word window related to the center word C are found. For each context word, the softmax function is used to calculate the probability of belonging to each word, and the word vector and context vector are updated through backpropagation and gradient descent.

[0032] The embedding vector is normalized using the L2 norm.

[0033] Furthermore, in step 3, based on the priority of static test cases, select high-priority test cases for execution. Based on the feedback from the execution results, adjust the priority of test cases that are dissimilar to those that detected errors. The specific method is as follows:

[0034] a) The test case ranking problem is described as a total recall problem, the goal of which is to examine and label as few candidate test cases as possible while achieving the highest recall rate;

[0035] b) Set the test case set E, the test case set to be executed L, the test case set LR for test failures, the batch size N1, and the query strategy threshold N2;

[0036] c) Select high-priority test cases from the test case set for execution, and store the successful and failed test cases into L and LR respectively;

[0037] d) Select the sampling method based on the relationship between LR and N2, and adjust the test case set for subsequent execution. If LR is greater than N2, select the deterministic sampling method, train the SVM model based on the embedding vectors corresponding to the test cases in LR, and divide the unexecuted test cases into two parts: a positive score set and a negative score set. The test case set that is predicted to fail is the negative score set, and the other is the positive score set. Then, select the part of the positive score set that is not similar to the current failed test case set as the test case set to be executed later. Otherwise, select the uncertain sampling method, and directly use all the remaining unexecuted test case sets as the test case set to be executed later.

[0038] e) Iterate through steps c) and d), selecting high-priority test cases from the updated test case set for execution, until all test cases have been executed.

[0039] A dynamic sorting system for microservice test cases based on semantic similarity, implementing the aforementioned dynamic sorting method for microservice test cases to achieve dynamic sorting of microservice test cases, includes:

[0040] The microservice testing strategy analysis module analyzes the characteristics of microservice functional testing from the perspectives of unit testing, contract testing, integration testing, component testing, and end-to-end testing; it analyzes the characteristics of microservice non-functional testing from the perspectives of coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing, and cohesion testing; and it statically prioritizes test cases based on their importance and the impact of test case failures on the system.

[0041] The test case feature analysis module preprocesses the test cases, generates a sequence of test cases for similarity assessment, and generates the embedding vectors corresponding to the test case sequence through semantic analysis methods.

[0042] The priority adjustment module selects high-priority test cases for execution based on the priority of static test cases. Based on the feedback of the execution results, it adjusts the priority of test cases that are not similar to the test cases that detect errors.

[0043] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned dynamic sorting method for microservice test cases based on semantic similarity to achieve dynamic sorting of microservice test cases.

[0044] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the aforementioned dynamic sorting method for microservice test cases based on semantic similarity is implemented to achieve dynamic sorting of microservice test cases.

[0045] Compared with existing technologies, the significant advantages of this invention are: (1) It can statically divide the priority of test cases for microservice systems. Existing technologies can only statically divide the priority of test cases for traditional application systems and cannot be directly applied to microservice systems. (2) It can evaluate the semantic similarity between test cases. Existing corpora are not specific; for example, Wikipedia is a general corpus and is not suitable for evaluating the semantic similarity of test cases. In this invention, by establishing a corpus corresponding to the test cases, the test cases are preprocessed, and the Word2Vec model is used to convert the test cases into embedded vector representations, which can effectively evaluate the semantic similarity between test cases. (3) Based on the results obtained from the above steps, an SVM model is trained, and the priority of test cases can be dynamically adjusted during the testing process. Using the static division results of test cases, the algorithm iteration no longer randomly selects test cases, but selects a portion of high-priority test cases to execute first, which can detect errors that have a greater impact on the system more quickly. During the execution process, based on the feedback of the execution results and the similarity between test cases, the priority of test cases that are not similar to the failed test cases is dynamically adjusted, expanding the range of error detection types and accelerating the fault detection rate. Attached Figure Description

[0046] Figure 1 These are key technical diagrams included in this invention;

[0047] Figure 2 This is a schematic diagram of the semantic similarity evaluation process for test cases in this invention;

[0048] Figure 3 This is a schematic diagram of the dynamic sorting process of test cases in this invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] like Figure 1As shown, a method for dynamically ranking test cases in a microservice system based on semantic similarity includes the following steps:

[0051] Step 1. First, analyze the characteristics of microservice functional testing from the perspectives of unit testing, contract testing, integration testing, component testing, and end-to-end testing. Then, analyze the characteristics of microservice non-functional testing from the perspectives of coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing, and cohesion testing. Finally, statically prioritize test cases based on their importance and the impact of test case failures on the system.

[0052] a) Functional testing

[0053] This analysis examines the characteristics of microservice functional testing through unit testing, contract testing, integration testing, component testing, and end-to-end testing.

[0054] (1) Unit Testing: In a microservice architecture, unit testing is performed on each individual service or functional module. Since each microservice is independent, unit testing mainly focuses on the correctness of a single service or functional module. Through unit testing, the functionality and performance of each microservice can be verified when running independently. In addition, since microservices usually follow the design principle of "high cohesion and low coupling", unit testing has high isolation and can detect and locate problems more quickly.

[0055] (2) Contract Testing: In a microservice architecture, contract testing is a testing technique used to verify whether the interactions between microservices meet expectations. Contract testing is typically performed at the service interface or API level, ensuring the correctness and stability of services by checking whether requests and responses meet expectations. Contract testing can reduce the complexity of interactions between microservices and improve the efficiency and accuracy of testing.

[0056] (3) Integration Testing: In a microservice architecture, integration testing tests the interactions between multiple microservices. By simulating interaction scenarios between multiple microservices in a real-world operating environment, it verifies whether the collaboration and data transmission between different microservices are normal. Integration testing can discover problems with inter-service interfaces, ensuring the correctness and stability of services. In addition, integration testing can also detect whether the dependencies between different services are correct and whether there are data consistency issues.

[0057] (4) Component Testing: In a microservice architecture, component testing involves testing a specific component or module. Component testing typically follows unit testing and verifies the correctness of a component or module when combined with other components or modules. Component testing can uncover issues at the interfaces between modules, ensuring proper collaboration between components. Furthermore, component testing can also assess the performance and stability of components.

[0058] (5) End-to-end testing: In a microservice architecture, end-to-end testing is performed on the entire microservice application. End-to-end testing typically follows integration and component testing to verify the functionality and performance of the entire application. End-to-end testing can uncover problems throughout the application, ensuring its correctness and stability. Furthermore, end-to-end testing can also assess the application's performance and security.

[0059] b) Non-functional testing

[0060] Non-functional testing is an important aspect of microservice testing. It focuses on the non-functional attributes of the system, such as coexistence, availability, fault tolerance, performance, coupling, and cohesion. Based on this, we can analyze the characteristics of non-functional testing in microservice testing.

[0061] (1) Coexistence testing: Microservice architecture may have conflicts such as multiple services, multiple resources, and circular dependencies. Coexistence testing mainly focuses on the system's ability to cooperate when multiple services and resources coexist. By simulating the joint operation of multiple services and resources, the correctness of the system in handling various complex situations is ensured.

[0062] (2) Availability testing: Thanks to high availability technology and container orchestration technology, there is redundancy in the number of service instances under the microservice architecture. Availability testing mainly focuses on the impact of the number of critical service instances on the system.

[0063] (3) Fault tolerance testing: Each service in a microservice architecture is independent, so fault tolerance testing mainly focuses on the recovery capability of a single service when faced with abnormal situations. By simulating various failure scenarios, such as network failures and system crashes, the behavior of services under abnormal conditions can be tested to ensure that the services can recover correctly and continue to provide services.

[0064] (4) Performance Testing: Each service in a microservice architecture has its specific functions and performance requirements. Performance testing mainly focuses on the performance of a single service under different loads. By simulating requests under different load conditions, metrics such as service response time and throughput can be tested to ensure that the service can maintain good performance under different loads.

[0065] (5) Coupling Testing: In a microservice architecture, different services should be kept as independent as possible. Coupling testing mainly focuses on the degree of independence between different services. By simulating requests between different services, the strength of the interrelationship between different services is tested.

[0066] (6) Cohesion Testing: Each service in a microservice architecture should ensure the completion of business functions within a unique domain. Cohesion testing focuses on the completeness of the business functions within a service. By establishing complete processes for different business functions, it tests whether each business function is fully implemented within a single service.

[0067] In summary, the characteristics of microservice testing are mainly reflected in the following aspects: each microservice can be independently tested for unit and component testing; contract testing is used to verify the interaction between microservices; integration testing and end-to-end testing are used to verify the functionality and performance of the entire microservice application; and non-functional testing focuses on the non-functional attributes of the system. These characteristics together constitute a complete testing system under the microservice architecture, providing strong assurance for the quality and stability of microservice applications.

[0068] Compared to traditional software testing, microservice testing adds component testing and contract testing. This means microservice testing requires testing individual services as well as the communication between them. Due to the unique nature of microservice architectures, their components are often deployed in cloud virtual machines or containers. Thanks to the portability of virtualization and container technologies, the portability and its sub-characteristics of microservice architectures can be largely ignored. At the same time, microservice architectures have additional requirements regarding performance efficiency, compatibility, reliability, and maintainability.

[0069] c) Static priority allocation of microservice test cases

[0070] Test cases are statically prioritized based on their importance and the impact of test case failures on the system.

[0071] The characteristics of microservice testing obtained above reflect the microservice testing process and show some priority relationships between different levels of testing, such as unit tests being performed before other types of tests.

[0072] Based on the microservice testing strategy characteristics obtained from the above analysis, the static priority of microservice test cases (including functional tests and non-functional tests) is divided, with functional tests having a higher priority than non-functional tests. The two test categories can be further subdivided in terms of priority from high to low as follows:

[0073] (1) Functional testing

[0074] P1-0: Unit tests for verifying external communication and external service interaction modules are classified into P1-0 level according to their functional importance.

[0075] P1-1: Unit tests for the verification data access and external database interaction modules are classified into P1-1 level according to their functional importance.

[0076] P1-2: Consumer-side contract testing, which defines the requests and expected responses between consumers and providers through a streaming API, is classified as P1-2 level.

[0077] P1-3: Contract testing on the provider side of the module that simulates consumers sending requests to providers based on the contract document is classified as level P1-3.

[0078] P1-4: Integration tests that target interaction scenarios between multiple microservices are classified into P1-4 levels.

[0079] P1-5: Component tests that verify the correctness of a specific component or module when combined with other components or modules are classified into P1-5 levels.

[0080] P1-6: End-to-end tests performed on the entire microservice application to verify whether user needs are met and overall business objectives are achieved are classified as P1-6 levels.

[0081] (2) Non-functional testing

[0082] P2-0: The coexistence test of potential service conflicts, resource conflicts, and circular dependency issues between various services is classified as P2-0 level.

[0083] P2-1: Availability tests that verify the number of critical service instances are classified as P2-1 level.

[0084] P2-2: Fault tolerance testing that measures system capabilities by judging whether mechanisms such as circuit breaking, rate limiting, degradation, and load balancing are implemented is classified as P2-2 level.

[0085] P2-3: Performance tests that measure microservice response time, throughput, capacity, and resource utilization are classified as P2-3 level.

[0086] P2-4: Tests on the strong or weak coupling between testing services are classified into P2-4 levels.

[0087] P2-5: Cohesion testing, which tests whether a microservice implements a complete domain model business within a single service, is classified as level P2-5.

[0088] Based on the above priority classification principles, when applied to actual microservice testing, the test cases are divided into the above levels to obtain the static priority of microservice test cases.

[0089] Step 2. First, perform test case data preprocessing to generate a numerical representation of the test case sequence for similarity assessment; then, generate the embedding vectors corresponding to the test case sequence through semantic analysis methods.

[0090] a) Analyze the characteristics of the test cases, perform data preprocessing, and generate a numerical representation of the test case sequence for similarity assessment:

[0091] Test cases are divided into two types: text-based test cases and code script test cases. For text-based test cases, text semantic analysis is used for data preprocessing; for code-based test cases, the code is parsed into a syntax tree (AST) to analyze its syntax and semantic information, thus generating clean test cases. Preprocessing the test case data facilitates subsequent similarity evaluation. For text-based test cases, a third-party library is used to remove stop words, punctuation marks, tags, etc., from the original test case dataset to generate clean test cases; for code-based test cases, the ASTParser is used to parse the test case code, transforming it into an Abstract Syntax Tree (AST). Then, the program token is obtained by traversing the nodes in the AST, generating clean test cases.

[0092] Next, word segmentation is performed on the two types of clean test cases that have been processed to obtain a list of words corresponding to the test cases. Each word is used as a key, and the number of existing keys is used as a value. This ensures that each word is assigned a unique identifier, i.e., an index, and establishes a mapping relationship between words and indices. This generates a numerical representation of the test case sequence for subsequent similarity evaluation.

[0093] The specific algorithm is as follows:

[0094] Input: Original dataset input_data, test script code_file

[0095] Output: cleaned data (cleaned_data) and word segmentation dictionary (word_dict)

[0096] step:

[0097] #Remove stop words, special characters, punctuation marks, etc. from text-based test cases to generate clean test cases.

[0098]

[0099] #For code-based test cases, obtain the program's abstract syntax tree and extract the program token to generate clean test cases.

[0100]

[0101]

[0102] #Construct a dictionary from the two types of clean test cases that have already been processed, establish key-value pairs in the word segmentation dictionary, i.e., the mapping relationship from words to indices, and generate a numerical representation of the test case sequence for subsequent similarity evaluation.

[0103]

[0104] #word_dict is a word list containing a mapping between words and their indices.

[0105] b) Generate embedding vectors corresponding to the test case sequence through semantic analysis methods;

[0106] After data preprocessing, the Word2Vec model is used to convert the test case sequence into embedding vectors, and then L2 norm is used for normalization. The embedding vectors reflect the similarity between test cases.

[0107] 1) Train test cases using Word2Vec and convert them into embedding vectors. The steps are as follows:

[0108] 1. Initialization:

[0109] For each word w, a vector representation v_w is randomly generated.

[0110] Define the headword C and the context window size N.

[0111] 2. Training process:

[0112] For each test case sequence training sample (e.g., sentences or paragraphs in a text corpus), a center word C is randomly selected (note: not all words in the same sentence are necessarily selected as the center word). All context words (w_c1, w_c2, ..., w_cN) within N word windows related to the center word C are found. If w_c is both the center word and a context word in the current window, it will also be included in the vector learned as a context word.

[0113] For each context word, the following steps are performed: The probability of belonging to each word is calculated using the softmax function (i.e., logQ(Wc, Wj) in the formal formula), where Wj represents a candidate word. Relevant parameters are updated, namely, the word vectors and context vectors (used to capture the relationships between words in the context). These parameters are updated using backpropagation and gradient descent to minimize the loss function.

[0114] 3. Model Output:

[0115] The vector representation of each word obtained through learning.

[0116] 2) Regularize the obtained embedding vector.

[0117] Function L2Norm()

[0118] #Calculate the L2 norm of each vector

[0119] norms=np.linalg.norm(input_data,ord=2,axis=1,keepdims=True)

[0120] # Perform L2 normalization on each vector

[0121] normalized_data=input_data / norms

[0122] Step 3. Based on the priority of static test cases, select high-priority test cases to execute. Based on the feedback of the execution results, for test cases that detect errors, adjust the priority of test cases that are not similar to them (i.e. vectors that are far away from the classification hyperplane) to expand the scope of fault detection.

[0123] Based on the static priority of test cases, some test cases are executed first. Then, based on the feedback of the execution results and the similarity between test cases, the priority of test cases that are not similar to the failed test cases is dynamically adjusted.

[0124] a) Describe the test case ranking problem as a total recall problem: given a candidate set E, which contains a small set of positive scores. For each x∈E, it can be labeled as positive (x∈R) or negative. From the label Initially, the task is to examine and label as few candidate use cases as possible (min|L|) while achieving a very high recall rate (max|L∩R| / |R|).

[0125] b) Define method inputs: test case set E, test case set R that will fail, batch size N1, query strategy threshold N2; Define method outputs: test case set L to be executed, test case set LR that failed.

[0126] c) Initialization When LR is empty, select high-priority test cases from the partitioning results obtained in step 1) and put the passing and failing test cases into L and LR. When LR is not empty, train the SVM model based on the embedding vectors obtained in step 2) (that is, take the embedding vector set in LR as input, select a linear kernel function, call the third-party library function of Python to train the SVM model, and output the classification hyperplane CL and support vectors, which is essentially partitioning the set according to similarity). The obtained hyperplane CL is used to dynamically adjust the priority of subsequent test cases.

[0127] d) After obtaining the hyperplane model, it is necessary to select the test case set X to be executed in the next iteration, that is, to dynamically adjust its priority. The sampling method is selected according to the relationship between LR and N2. When LR > N2, there are enough samples, and the deterministic sampling method is selected to maximize the probability of prediction failure. Otherwise, the uncertain sampling method is selected to expand the search range and improve the global search capability. (The classification hyperplane CL divides the unexecuted test case set in c above into two parts: a positive score set and a negative score set. That is, the test case set that is predicted to fail is the negative score set, and the other is the positive score set. Deterministic sampling selects the part that is not similar to the current failed test case set from the positive score set. Uncertain sampling directly uses all the unexecuted test case sets in c above as the test case set to be executed in the next iteration to improve the global search capability.)

[0128] e) After obtaining X, continue executing this part of the test cases, and add the passed and failed test cases to L and LR. The method loops until L = E, iteratively adjusting the priority of the test cases.

[0129] This invention also proposes a dynamic sorting system for microservice test cases based on semantic similarity, implementing the aforementioned dynamic sorting method for microservice test cases to achieve dynamic sorting of microservice test cases, including:

[0130] The microservice testing strategy analysis module analyzes the characteristics of microservice functional testing from the perspectives of unit testing, contract testing, integration testing, component testing, and end-to-end testing; it analyzes the characteristics of microservice non-functional testing from the perspectives of coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing, and cohesion testing; and it statically prioritizes test cases based on their importance and the impact of test case failures on the system.

[0131] The test case feature analysis module preprocesses the test cases, generates a sequence of test cases for similarity assessment, and generates the embedding vectors corresponding to the test case sequence through semantic analysis methods.

[0132] The priority adjustment module selects high-priority test cases for execution based on the priority of static test cases. Based on the feedback of the execution results, it adjusts the priority of test cases that are not similar to the test cases that detect errors.

[0133] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned dynamic sorting method for microservice test cases based on semantic similarity to achieve dynamic sorting of microservice test cases.

[0134] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the aforementioned dynamic sorting method for microservice test cases based on semantic similarity is implemented to achieve dynamic sorting of microservice test cases.

[0135] In summary, this invention provides a test case sorting method that dynamically adjusts the priority of test cases based on the static priority of test cases in a microservice system and the semantic similarity between test cases. This solves the problems of poor generalization ability and limited applicability of existing TCP technologies.

[0136] Example

[0137] To verify the effectiveness of the present invention, the following experiment was conducted.

[0138] It is known that four test case sets A, B, C, and D have been generated, which are integration test, contract test, usability test, and performance test, respectively.

[0139] Implement the steps mentioned above in sequence:

[0140] Step 1: Divide the test cases into priority groups T = {a1,a2…,an,b1,b2…,bn,c1,c2…,c3,d1,d2…,d3} using the static test case partitioning method.

[0141] Step 2: Perform data cleaning and other preprocessing on each test case, then construct a word segmentation dictionary, and finally train the embedding vectors and apply L2 regularization.

[0142] a) Data cleaning:

[0143] • Text test case set T1: cleanedT1 is obtained by removing stop words, special symbols, and punctuation marks;

[0144] • Code test case set T2: Remove comments, etc., and then perform lexical and syntactic analysis to obtain a token sequence. Add the token sequence to the given input data list to finally obtain cleanedT2;

[0145] b) Constructing a word segmentation dictionary: The cleaned data is segmented using jieba, and then the text dictionary word_dict is constructed using the gensim.corpora.Dictionary function.

[0146] c) Train the embedding vectors and perform L2 regularization: Train the word_dict above using Word2Vec to obtain the embedding vector set Vector = {v1, v2, ..., vn}. For each vector v1 = {x1, x2, ..., xm}, perform regularization using the following formula:

[0147]

[0148] Where v' is the L2-regularized vector, and ||v|| represents the L2 norm of vector v, calculated using the following formula:

[0149]

[0150] It measures the length of vector v. Dividing each element of the original vector v by its L2 norm yields a new vector v' with the same direction but uniform length, ultimately resulting in Vector'.

[0151] Step 3: Based on the static priority T of the test cases, execute a subset of test cases first. Then, based on the feedback from the execution results and the similarity vector between test cases, dynamically adjust the priority of test cases that are dissimilar to those that have already failed.

[0152] a) Initialization:

[0153] Executed test case set

[0154] Test case set that failed

[0155] Test case set E = {a1,a2…,an,b1,b2…,bn,c1,c2…,c3,d1,d2…,d3}

[0156] Batch size N1 = 10

[0157] Query threshold N2 = 20

[0158] b) Select a subset of test cases from E for execution:

[0159] After execution, we get L = {a1, a2, a3…} and LR = {a2, a4…}

[0160] c) Train an SVM based on logistic regression to obtain the hyperplane CL;

[0161] d) If LR > N2, use deterministic sampling in combination with CL to select |L| test cases and adjust their priorities; if LR < N2, use non-deterministic sampling to select |L| test cases from E and continue execution;

[0162] e) Repeat c) and d) until |L| = |E|.

[0163] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0164] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for dynamically ranking microservice test cases based on semantic similarity, characterized in that, Includes the following steps: Step 1: Analyze the characteristics of microservice functional testing from the perspectives of unit testing, contract testing, integration testing, component testing, and end-to-end testing; analyze the characteristics of microservice non-functional testing from the perspectives of coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing, and cohesion testing; and statically prioritize test cases based on their importance and the impact of test case failures on the system. Step 2: Preprocess the test cases to generate a numerical representation of the test case sequence for similarity assessment, and generate the embedding vector corresponding to the test case sequence through semantic analysis methods. Step 3: Based on the priority of static test cases, select high-priority test cases for execution. Based on the feedback from the execution results, adjust the priority of test cases that are dissimilar to those that detected errors. The specific method is as follows: a) The test case ranking problem is described as a total recall problem, the goal of which is to examine and label as few candidate test cases as possible while achieving the highest recall rate; b) Set the test case set E, the test case set to be executed L, the test case set LR for test failures, the batch size N1, and the query strategy threshold N2; c) Select high-priority test cases from the test case set for execution, and store the successful and failed test cases into L and LR respectively; d) Select the sampling method based on the relationship between LR and N2, and adjust the test case set for subsequent execution. If LR is greater than N2, select the deterministic sampling method, train the SVM model based on the embedding vectors corresponding to the test cases in LR, and divide the unexecuted test cases into two parts: a positive score set and a negative score set. The test case set that is predicted to fail is the negative score set, and the other is the positive score set. Then, select the part of the positive score set that is not similar to the current failed test case set as the test case set to be executed later. Otherwise, select the uncertain sampling method, and directly use all the remaining unexecuted test case sets as the test case set to be executed later. e) Iteration steps c) and d) select high-priority test cases from the updated test case set for execution until all test cases have been executed.

2. The method for dynamically ranking microservice test cases based on semantic similarity according to claim 1, characterized in that, Step 1: Analyze the characteristics of microservice functional testing from the perspectives of unit testing, contract testing, integration testing, component testing, and end-to-end testing. Analyze the characteristics of microservice non-functional testing from the perspectives of coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing, and cohesion testing. Based on the importance of the tests and the impact of test case failures on the system, statically prioritize test cases, with functional tests having higher priority than non-functional tests. Specifically: a) The priorities in functional testing, from highest to lowest, are as follows: Unit tests to verify external communication and interaction with external services; unit tests to verify data access and interaction with external databases; consumer-side contract tests that define requests and expected responses between consumers and providers through streaming APIs; provider-side contract tests that simulate consumers sending requests to providers based on contract files; integration tests for interaction scenarios between multiple microservices; component tests to verify the correctness of a specific component or module when combined with other components or modules; and end-to-end tests for the entire microservice application to verify whether it meets user needs and achieves overall business goals. b) In non-functional testing, the priorities from highest to lowest are as follows: Availability testing verifies the number of critical service instances; fault tolerance testing measures system capabilities by determining whether service circuit breaking, rate limiting, degradation, and load balancing mechanisms are implemented; performance testing measures microservice response time, throughput, capacity, and resource utilization; testing the strength of dependencies between services; and cohesion testing measures whether microservices implement a complete domain model business within a single service.

3. The method for dynamically ranking microservice test cases based on semantic similarity according to claim 1, characterized in that, Step 2 involves preprocessing the test cases to generate a numerical representation of the test case sequence for similarity assessment. Semantic analysis is used to generate embedding vectors corresponding to the test case sequence, reflecting the similarity between test cases. The specific method is as follows: a) Analyze the characteristics of the test cases, perform data preprocessing, and generate a numerical representation of the test case sequence for similarity assessment: For test cases in text format, a third-party library is called to remove stop words, punctuation marks, and tags from the original test case dataset, generating clean test cases. For test cases in code form, the ASTParser is called to parse the test case code, transforming it into an Abstract Syntax Tree (AST). Then, the program token is obtained by traversing the nodes in the AST, generating clean test cases. For the two types of clean test cases that have been processed, perform word segmentation to obtain a list of words corresponding to the test cases. Each word is used as a key, and the number of existing keys is used as a value. This ensures that each word is assigned a unique identifier, i.e., an index. Then, a mapping relationship between words and indexes is established, generating a numerical representation of the test case sequence for subsequent similarity evaluation. b) Generate embedding vectors corresponding to the test case sequence using semantic analysis methods: The Word2Vec model is used to convert the test case sequence into an embedding vector. For each test case sequence, a center word C is randomly selected, and all context words of the center word C within N word windows are found. For each context word, the probability of belonging to each word is calculated using the softmax function, and the word vector and context vector are updated using the backpropagation algorithm and gradient descent method. The embedding vector is normalized using the L2 norm.

4. A dynamic sorting system for microservice test cases based on semantic similarity, characterized in that, Implementing the dynamic sorting method for microservice test cases based on semantic similarity as described in any one of claims 1-3, to achieve dynamic sorting of microservice test cases, includes: The microservice testing strategy analysis module analyzes the characteristics of microservice functional testing from the perspectives of unit testing, contract testing, integration testing, component testing, and end-to-end testing; it analyzes the characteristics of microservice non-functional testing from the perspectives of coexistence testing, availability testing, fault tolerance testing, performance testing, coupling testing, and cohesion testing; and it statically prioritizes test cases based on their importance and the impact of test case failures on the system. The test case feature analysis module preprocesses the test cases, generates a sequence of test cases for similarity assessment, and generates the embedding vectors corresponding to the test case sequence through semantic analysis methods. The priority adjustment module selects high-priority test cases for execution based on the priority of static test cases. Based on the feedback of the execution results, it adjusts the priority of test cases that are not similar to the test cases that detect errors.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the dynamic sorting method for microservice test cases based on semantic similarity as described in any one of claims 1-3, thereby realizing dynamic sorting of microservice test cases.

6. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the dynamic sorting method for microservice test cases based on semantic similarity as described in any one of claims 1-3, thereby realizing dynamic sorting of microservice test cases.

Images