Test case management method and device, equipment and storage medium

Abstract

The invention discloses a test case management method and device, equipment and a storage medium, and belongs to the technical field of automated testing, and the method comprises the following steps: establishing a standardized structure of a test case, and compiling a plurality of test case configuration files and user-defined resource configuration files; binding labels to the test cases corresponding to the test case configuration files, and performing classified management on all the self-defined resource configuration files; analyzing a custom resource configuration file which contains a to-be-executed test case configuration file in quotation to obtain test information and label information of all test cases quoted by the custom resource configuration file; and screening out a test case to be executed from all the referenced test cases by utilizing the label information, and executing a test operation according to the test information of the test case to obtain a test result. According to the method and the device, the test process is configured by utilizing the YAML file, so that the writing and maintenance process of the test case can be simplified, and multiple test scenes are easily accessed and expanded.

Description

Technical Field

[0001] The present invention relates to the field of automated testing technology, and in particular to a test case management method, device, equipment and storage medium. Background Art

[0002] In the modern software development process, automated testing frameworks play a vital role in ensuring software quality and improving development efficiency. An ideal automated testing framework not only needs to be sustainable and scalable, but also needs to ensure that test cases are easy to expand and maintain, and can be easily called and integrated.

[0003] However, many automated testing frameworks on the market have exposed some shortcomings in actual applications. For example, some frameworks are rigidly designed and difficult to adapt to the rapid changes in project requirements, which makes the maintenance and expansion of the framework particularly difficult in the later stages of the software life cycle. Another example is that the writing, organization and maintenance of test cases in some frameworks are cumbersome, lacking intuitive interfaces and powerful management tool support, which increases the burden on testers. Another example is that there are obstacles to the integration of some frameworks with existing development environments and tools, especially in complex distributed systems or microservice architectures, making it difficult to achieve cross-platform and cross-language test automation. In addition, as the scale of testing expands, some frameworks show performance bottlenecks in concurrent testing, big data processing, etc., which affects test efficiency and result accuracy.

[0004] Most importantly, with the rise and popularity of cloud-native technologies such as Kubernetes, it has become crucial to effectively test applications running on the Kubernetes platform, but existing frameworks are unable to provide comprehensive testing support for the Kubernetes environment. Summary of the invention

[0005] The purpose of the present invention is to provide a test case management method to solve the problem that the test framework in the prior art cannot provide comprehensive test support for the Kubernetes environment.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] A test case management method of the present application comprises the following steps:

[0008] Establishing a standardized structure of test cases, and writing a number of test case configuration files and custom resource configuration files according to the standardized structure, each custom resource configuration file references at least one of the test case configuration files;

[0009] Bind tags to the test cases corresponding to each test case configuration file, and classify and manage all custom resource configuration files;

[0010] Select the test case to be executed, and parse the custom resource configuration file that contains the test case configuration file to be executed to obtain the test information and label information of all the test cases referenced by it;

[0011] The tag information is used to filter out the test case to be executed from all the referenced test cases, and a test operation is performed according to the test information to obtain a test result.

[0012] Preferably, the establishment of a standardized structure of test cases includes:

[0013] A standardized structure of a test case is constructed by defining a custom resource definition, wherein the standardized structure includes test steps and expected results of the test case, wherein the test steps include an object filter and a path field for referencing at least one test case configuration file.

[0014] Preferably, the steps of respectively binding tags to the test cases corresponding to the test case configuration files include:

[0015] When the test case does not have any label, binding a label to it according to the label information preset in the object filter;

[0016] When the label of the test case does not match the label information preset in the object filter, the label is changed.

[0017] Preferably, the classified management of all custom resource configuration files includes:

[0018] Build a folder structure based on functional modules or test scenarios. Each folder represents a functional module or test scenario, and place each custom resource configuration file in the corresponding folder.

[0019] Preferably, the step of parsing the referenced custom resource configuration file containing the test case configuration file to be executed to obtain the test information and label information of all the test cases referenced by the custom resource configuration file includes:

[0020] Parse and reference the custom resource configuration file that contains the test case configuration file to be executed, and obtain the absolute or relative path of all the test cases referenced by it;

[0021] The configuration files of all referenced test cases are read and parsed according to the absolute or relative path to obtain the test information and label information of each referenced test case.

[0022] Preferably, the step of using the tag information to filter out the test case to be executed from all referenced test cases includes:

[0023] The tag information of each referenced test case is matched with the specified tag information respectively, wherein the tag information includes the key, value and operator of the tag. If the match is successful, the test case to be executed is obtained.

[0024] Preferably, performing a test operation according to the test information to obtain a test result includes:

[0025] Execute the test operation according to the test steps included in the test information, and obtain the expected result of the test case to be executed and its real-time status during the test, wherein the expected result includes a timeout time and an expected condition;

[0026] Determine whether its real-time status within the timeout period meets the expected condition. If not, the test fails, otherwise the test succeeds.

[0027] A test case management device, comprising:

[0028] A construction module is used to establish a standardized structure of test cases, and to write a number of test case configuration files and custom resource configuration files according to the standardized structure, each custom resource configuration file references at least one of the test case configuration files;

[0029] The management module is used to bind tags to the test cases corresponding to each test case configuration file, and to classify and manage all custom resource configuration files;

[0030] A parsing module is used to select a test case to be executed, and parse the custom resource configuration file that contains the test case configuration file to be executed to obtain the test information and label information of all the test cases referenced by it;

[0031] The test module is used to filter out the test cases to be executed from all the referenced test cases by using the tag information, and perform test operations according to the test information to obtain test results.

[0032] An electronic device comprises a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a test case management method as described in any one of the above.

[0033] A computer-readable storage medium storing a computer program, wherein the computer program enables a computer to implement a test case management method as described in any one of the above when executed.

[0034] The present invention has the following beneficial effects:

[0035] The present invention configures the test process by utilizing YAML files, which not only decouples the test cases from the underlying test framework code, thus simplifying the test case writing and maintenance process, but also supports rapid access to newly developed APIs and new function tests, while also facilitating access to and expansion of multiple test scenarios, thereby improving the reliability and traceability of the test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a test case management method provided by an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of a test case management device provided by an embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of an electronic device for implementing a test case management method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] To make the technical solution of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The terms "first", "second", etc. in the claims and specification of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is only to describe the distinction method adopted when describing objects with the same attributes in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that a process, method, system, product or device containing a series of units is not necessarily limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] like Figure 1 As shown, this embodiment provides a test case management method, comprising the following steps:

[0041] S110, establishing a standardized structure of test cases, and compiling a number of test case configuration files and custom resource configuration files according to the standardized structure, each custom resource configuration file references at least one test case configuration file;

[0042] S120, binding tags to the test cases corresponding to the test case configuration files respectively, and classifying and managing all the custom resource configuration files;

[0043] S130, selecting a test case to be executed, parsing the custom resource configuration file that references the test case configuration file to be executed, and obtaining test information and label information of all the test cases referenced by it;

[0044] S140: Filter out the test cases to be executed from all the referenced test cases using the tag information, and perform test operations according to the test information to obtain test results.

[0045] In this embodiment, a standardized structure of test cases is first established by defining a custom resource definition (Kubernetes CRD). In essence, this operation is completed in the CRD configuration file, namely the YAML file of the CRD, which defines the API version of the custom resource, namely apiVersion, to ensure that the Kubernetes cluster can correctly identify and process the custom resource. The type of the custom resource, namely kind, is also defined to ensure that the Kubernetes cluster can process the custom resource according to the field. The standardized structure also includes basic information of the test case, namely metadata, which can determine the uniqueness and manageability of the resource in the Kubernetes cluster. The metadata contains some necessary fields, such as the name of the test case, namely name, which can ensure that each test case is unique, the namespace where the resource is located, namely namespace, which is used for resource isolation, and spec for describing the test case information in detail, wherein spec includes specific definitions of test steps and expected results, and the corresponding necessary fields are:

[0046] steps: The definition of the test steps, which is an array and also includes the following fields:

[0047] name: the name of the test step, which is convenient for identifying each step. For example: name:create-cluster;

[0048] ops: operation type, defines the type of operation, such as apply, create, update, patch, delete, example: ops:apply;

[0049] paths: operation-related file paths, an array containing the YAML files of test cases, example: paths:-.. / .. / .. / resources / wesql / test_create_3replica.yaml;

[0050] extraObjectFilter: Object filter, contains the following fields:

[0051] matchExpressions: label expression, which is an array that defines the filtering conditions;

[0052] key: the key of the label, example: key:app.kubernetes.io / instance;

[0053] values: the value of the label, which is an array. For example: values:-wesql-cluster-lifecycle;

[0054] operator: operator, defines the logical operation of label filtering, example: operator:in;

[0055] expects: expected result, which is an array that defines the expected status and conditions;

[0056] kind: resource type, example: kind:Cluster;

[0057] conditions: expected conditions, an array in jsonpath format, example: conditions:-'.status.phase==Running';

[0058] skipped: whether to skip this step, example: skipped:0;

[0059] timeout: timeout in seconds, example: timeout:60;

[0060] waittime: waiting time in seconds. The default value is 0. Example: waittime:0.

[0061] This information is necessary to build a standardized structure for test cases to ensure that each test case can be correctly identified, executed, and verified.

[0062] After the standardized structure of the test case is established, the test case YAML file and the custom resource YAML file are written according to the standardized structure, and the standardization of all test case YAML files and custom resource YAML files is ensured. Each custom resource YAML file contains content related to the test input and expected results. In addition, it also contains a path field paths. The paths field is used to reference at least one test case YAML file. The absolute or relative path of the referenced test case YAML file can be obtained through the paths field, and then the test case YAML file can be read.

[0063] Furthermore, when the test case does not have any label, a label is bound to it according to the label information preset in the object filter;

[0064] Change the label of a test case if it does not match the label information preset in the object filter.

[0065] The extraObjectFilter is defined in the standardized structure step of the test case. Based on this, the corresponding test case should be labeled in the test case YAML file. If the test case does not have any labels, the kubectl patch command is used to directly bind fixed labels to the test case according to the label information contained in the extraObjectFilter. The label information specifically includes the label expression used to define the filtering conditions, the label key, the label value, and the operator used to define the label filtering logic operation. The fixed label here refers to the label that meets the filtering conditions defined in the standardized structure to ensure that the subsequent steps can filter the correct resources. If it has a label but does not meet the filtering conditions defined in the standardized structure, its label is manually modified.

[0066] Furthermore, a folder structure is constructed according to the functional modules or test scenarios, each folder represents a functional module or test scenario, and each custom resource configuration file is placed in the corresponding folder.

[0067] Build a folder structure according to functional modules or test scenarios. For example, the suites directory is divided into chaos test chaosmesh, cluster test cluster and operation request test opsrequest according to functional modules. A first main folder containing three subfolders is created, where the first main folder is named suites directory, chaosmesh, cluster and opsrequest are the names of the three subfolders under the first main folder respectively; the resources directory resource files create a folder structure according to different test scenario classifications. The directory contains multiple subdirectories, each subdirectory represents a database type, such as redis and wesql, then a second main folder containing two subfolders is created, where the second main folder is named resources directory resource files, redis and wesql are the names of the two subfolders under the second main file. After the folder structure is created, each custom resource YAML file is placed in the corresponding subfolder, thereby realizing the classified management of custom resource YAML files.

[0068] Further, the custom resource configuration file containing the test case configuration file to be executed is parsed and referenced to obtain the absolute or relative paths of all the test cases referenced by it;

[0069] Read and parse the configuration files of all referenced test cases according to the absolute or relative path to obtain the test information and label information of each referenced test case.

[0070] Select a test case to be executed from all test cases bound to tags, parse the corresponding custom resource YAML file, the paths field of the custom resource YAML file at least references the test case YAML file to be executed, and obtain the absolute or relative paths of all test case YAML files referenced in the custom resource YAML file. Then read and parse each referenced test case YAML file according to these absolute or relative paths to obtain the test information and tag information of each referenced test case. Each test information includes test steps, screening conditions and expected results.

[0071] Parsing the custom resource YAML file will result in at least one result. Therefore, when there is more than one, it is necessary to filter out the test cases to be executed. At this time, label information is needed to match the obtained label information with the specified label information. For example, the specified label information is a resource with the label app.kubernetes.io / instance and the value is wesql-cluster-lifecycle. The referenced test case that meets this condition is the selected test case to be executed. Then, the kubectl command is used to operate the resources according to the test information. In Kubernetes (K8s), operating resources usually involves creating, viewing, updating, and deleting resources. Resources such as Pod, Service, and Deployment can be completed through the kubectl command line tool, and then the test operation is performed according to the test steps.

[0072] Get the real-time status of the test case to be executed during the execution process and compare it with its expected result. The expected result includes the timeout time and expected conditions. If the real-time status meets the expected conditions within the timeout time, the test succeeds, otherwise the test fails. In K8s, defining the expected resource final state, that is, the expected result, is a core concept. It means that users define the state of resources they want in the K8s cluster through declarative configuration files, usually YAML files or JSON formats. For example, the expected Cluster state is defined as Running in expectations. The real-time status of resources is obtained through JSONPath. If the Cluster does not reach the Running state within the specified timeout, it is considered that there is a problem with this test case. Among them, JSONPath conditional verification is used to check specific fields of resource objects. Suppose you want to check whether the status of the Cluster resource named my-cluster is Running, you can use the command kubectl get cluster my-cluster-o jsonpath='{.status.phase}'. kubectl get cluster my-cluster means obtaining detailed information about the Cluster resource named my-cluster. -o jsonpath='{.status.phase}' means using the JSONPath expression {.status.phase} to extract the value of the status.phase field of the Cluster resource.

[0073] Finally, integrating the testing framework into the CI / CD process enables automated testing and continuous delivery.

[0074] This embodiment uses YAML files to configure the test process, which not only decouples the test cases from the underlying test framework code, simplifying the test case writing and maintenance process, but also supports rapid access to newly developed APIs and new function tests. It is also easy to access and expand various test scenarios, thereby improving the reliability and traceability of test results.

[0075] like Figure 2 As shown, this embodiment provides a test case management device, including:

[0076] A construction module 10 is used to establish a standardized structure of test cases, and to write a plurality of test case configuration files and custom resource configuration files according to the standardized structure, wherein each custom resource configuration file references at least one of the test case configuration files;

[0077] The management module 20 is used to bind tags to the test cases corresponding to the test case configuration files, and to classify and manage all the custom resource configuration files obtained by the construction module 10;

[0078] The parsing module 30 is used to select the test case to be executed, and parse the custom resource configuration file containing the test case configuration file to be executed to obtain the test information and label information of all the test cases referenced by it;

[0079] The testing module 40 is used to filter out the test cases to be executed from all the referenced test cases using the tag information obtained by the parsing module 30, and perform a test operation according to the test information to obtain a test result.

[0080] The above-mentioned device can execute the methods provided by all the aforementioned embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above-mentioned methods. For technical details not fully described in this embodiment, please refer to the methods provided by all the aforementioned embodiments of the present invention.

[0081] like Figure 3 As shown, this embodiment provides an electronic device, including a memory 301 and a processor 302, wherein the memory 301 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 302 to implement the above-mentioned test case management method.

[0082] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0083] This embodiment also provides a computer-readable storage medium storing a computer program, and the computer program enables a computer to implement the test case management method as described above when executed.

[0084] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 301 and executed by the processor 302, and the input interface 305 and the output interface 306 complete the I / O interface transmission of data to complete the present invention. The one or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0085] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may include, but is not limited to, a memory 301 and a processor 302. Those skilled in the art may understand that this embodiment is only an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components, or a combination of certain components, or different components. For example, the computer device may also include an input device 307, a network access device, a bus, etc.

[0086] The processor 302 may be a central processing unit (CPU), or other general-purpose processors 302, digital signal processors 302 (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor 302 may be a microprocessor 302, or the processor 302 may be any conventional processor 302, etc.

[0087] The memory 301 may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. The memory 301 may also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 301 may also include both an internal storage unit of the computer device and an external storage device. The memory 301 is used to store computer programs and other programs and data required by the computer device. The memory 301 may also be used to temporarily store in the output device 308, and the aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory ROM303, a random access memory RAM304, a disk or an optical disk.

[0088] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A test case management method, characterized in that: The following steps are involved: Establishing a standardized structure of test cases, and writing a number of test case configuration files and custom resource configuration files according to the standardized structure, each custom resource configuration file references at least one of the test case configuration files; Bind tags to the test cases corresponding to each test case configuration file, and classify and manage all custom resource configuration files; Select the test case to be executed, and parse the custom resource configuration file that contains the test case configuration file to be executed to obtain the test information and label information of all the test cases referenced by it; The tag information is used to filter out the test case to be executed from all the referenced test cases, and a test operation is performed according to the test information to obtain a test result.

2. A test case management method according to claim 1, characterized in that: The standardized structure of establishing test cases includes: A standardized structure of a test case is constructed by defining a custom resource definition, wherein the standardized structure includes test steps and expected results of the test case, wherein the test steps include an object filter and a path field for referencing at least one test case configuration file.

3. A test case management method according to claim 2, characterized in that: The test cases corresponding to the test case configuration files are respectively bound with tags, including: When the test case does not have any label, binding a label to it according to the label information preset in the object filter; When the label of the test case does not match the label information preset in the object filter, the label is changed.

4. A test case management method according to claim 1, characterized in that: The classification management of all custom resource configuration files includes: Build a folder structure based on functional modules or test scenarios. Each folder represents a functional module or test scenario, and place each custom resource configuration file in the corresponding folder.

5. A test case management method according to claim 2, characterized in that: The process of parsing the referenced custom resource configuration file containing the test case configuration file to be executed to obtain the test information and label information of all the test cases referenced by the referenced custom resource configuration file includes: Parse and reference the custom resource configuration file that contains the test case configuration file to be executed, and obtain the absolute or relative path of all the test cases referenced by it; The configuration files of all referenced test cases are read and parsed according to the absolute or relative path to obtain the test information and label information of each referenced test case.

6. A test case management method according to claim 5, characterized in that: The step of using the tag information to filter out the test case to be executed from all the referenced test cases includes: The tag information of each referenced test case is matched with the specified tag information respectively, wherein the tag information includes the key, value and operator of the tag. If the match is successful, the test case to be executed is obtained.

7. A test case management method according to claim 5, characterized in that: The performing of the test operation according to the test information to obtain the test result includes: Execute the test operation according to the test steps included in the test information, and obtain the expected result of the test case to be executed and its real-time status during the test, wherein the expected result includes a timeout time and an expected condition; Determine whether its real-time status within the timeout period meets the expected condition. If not, the test fails, otherwise the test succeeds.

8. A test case management device, characterized in that: include: A construction module is used to establish a standardized structure of test cases, and to write a number of test case configuration files and custom resource configuration files according to the standardized structure, each custom resource configuration file references at least one of the test case configuration files; The management module is used to bind tags to the test cases corresponding to each test case configuration file, and to classify and manage all custom resource configuration files; A parsing module is used to select a test case to be executed, and parse the custom resource configuration file that contains the test case configuration file to be executed to obtain the test information and label information of all the test cases referenced by it; The test module is used to filter out the test cases to be executed from all the referenced test cases by using the tag information, and perform test operations according to the test information to obtain test results.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a test case management method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables a computer to implement a test case management method as claimed in any one of claims 1 to 7 when executed.

Images