A method and apparatus for functional testing of an observability platform

Abstract

The application provides a kind of observability platform function test method and device, the method comprises: S1, custom test case is written and stored;S2, input version to be regressed, according to the version information of version and the test case, the regression test case of version is calculated, and regression test case set is formed;S3, according to the regression test case set, apply computing resource, according to the computing test resource, real-time creation Pod;S4, in the Pod, the regression test case is executed, and the regression test case is tested, test result is obtained and is cached;S5, read the test result and summarize test related information, convert into line protocol format and output to observability platform for secondary processing and analysis.The method of the application integrates the observability data and result generated in the test process with the observability platform, outputs to the observability platform through the standardized data format, and carries out secondary processing and analysis.

Description

Technical Field

[0001] This invention belongs to the field of software testing technology, specifically to a functional testing method and apparatus for an observability platform. Background Technology

[0002] An observability platform is a cloud-native, open platform that provides a range of observability features, including logging, metrics, tracing, and debugging. These features help users quickly locate problems and improve system stability and availability.

[0003] Functional testing of an observability platform refers to black-box testing of various functions of the observability platform through API interfaces to verify whether the observability platform functions normally.

[0004] Traditional API functional testing methods require developers to write test cases that include the order of one or more API calls, parameters, and expected responses. These test cases are then executed through a testing framework to verify whether the observability platform functions correctly.

[0005] Existing functional testing methods for observability platforms have the following limitations:

[0006] 1) Testing frameworks integrated with programming languages ​​often require full collaboration between developers and testers, resulting in high costs for writing test cases and a large workload for writing and maintaining test cases, which is not conducive to rapid iteration of test cases;

[0007] 2) Client-side test orchestration tools with graphical interfaces improve the efficiency of test writing, but because they require installation and execution on the client side, they are not suitable for continuous testing scenarios. For example, when a regression test for a version is triggered, hundreds or thousands of different test cases often need to be executed, and a single client can take tens of hours to complete this task.

[0008] 3) The observable data and results generated during the testing process often circulate within the platform and are difficult to use with external observable platforms. Therefore, they are not entirely applicable to the testing and integration of observable platforms.

[0009] In view of this, the present invention is hereby proposed. Summary of the Invention

[0010] In view of this, the present invention discloses a functional testing method and apparatus for an observability platform, which realizes centralized management and distributed execution of test cases on the server side, thereby improving the execution efficiency of test cases; it integrates the observable data and results generated during the testing process with the observability platform, and outputs them to an external observability platform through a standardized data format for secondary processing and analysis.

[0011] Specifically, the present invention is achieved through the following technical solutions:

[0012] In a first aspect, the present invention discloses a functional testing method for an observability platform, comprising the following steps:

[0013] S1. Write and store custom test cases;

[0014] Specifically, test cases can be written using common client-side graphical tools. JMeter test cases can be written using JMeter's GUI tool, submitted as JMX files, and centrally stored and computed.

[0015] S2. Input the version to be regressed, calculate the regression test cases for the version based on the version information and the test cases, and form a regression test case set;

[0016] S3. Request computing resources based on the set of regression test cases, and create Pods in real time based on the computing test resources;

[0017] S4. Execute the regression test case in the Pod, test the regression test case, obtain the test results and cache them;

[0018] S5. Read the test results and summarize the test-related information, convert them into line protocol format, and output them to the observability platform for secondary processing and analysis.

[0019] Specifically, the data is transmitted using an InfluxDB-compatible row protocol format and output to an external observability platform for secondary processing. During this process, the data acquisition module collects information such as the request address, parameters, status code, response time, and error message for each API call. At the same time, the data acquisition module adds additional tags to the collected data, including the name of the test case, the computing instance to which the execution environment belongs, and custom tags.

[0020] Furthermore, in step S3, the method for requesting computing resources includes dynamically invoking Kubernetes' Virtual Kubelet to request computing resources.

[0021] Furthermore, in step S3, the test resources include the name of the test resources, the type of the test resources, the configuration of the test resources, and the environmental information of the test resources.

[0022] The name of the test resource is its unique identifier;

[0023] The types of test resources include Kubernetes Pods, Deployments, etc.

[0024] The configuration of test resources includes the Pod's CPU, memory, disk, etc.;

[0025] The environment information for the test resources includes the Pod's image, image version, etc.

[0026] Furthermore, in step S1, the definition information of the test cases includes the test device, test case information, version control information, and custom tags.

[0027] The testing device is a collection of multiple test cases;

[0028] Test case information includes the test case name, unique identifier of the test case, API address, parameters, and expected response;

[0029] Version control information includes the version information of the test case itself and the version information of the observability platform to which the test case corresponds;

[0030] Custom tags are user-provided tags used to distinguish different test cases and application scenarios.

[0031] Furthermore, the test-related information includes the definition information of the test cases, the runtime environment information of the test cases, and the execution information of the test cases.

[0032] Furthermore, in step S4, the test results include the API address, parameters, error message, response time, and whether they meet expectations.

[0033] Furthermore, in step S5, the secondary processing and analysis includes verifying whether the observability platform is functioning correctly by calling the API.

[0034] Secondly, the present invention discloses a functional testing apparatus for an observability platform, comprising:

[0035] Test case management module: Stores custom test cases;

[0036] Regression test case calculation module: Input the version to be regressed, calculate the regression test cases for the version based on the version information and the test cases, and form a set of regression test cases;

[0037] Pod scheduling module: Requests computing resources based on the set of regression test cases, and creates Pods in real time based on the computing test resources;

[0038] Test result processing module: Executes the regression test case in the Pod, tests the regression test case, obtains the test results, and caches them;

[0039] Data output module: Reads the test results and summarizes the test-related information, converts it into line protocol format, and outputs it to the observability platform for secondary processing and analysis.

[0040] Thirdly, the present invention discloses a computer-readable storage medium having a computer program thereon that, when executed by a processor, implements the steps of the functional testing method as described in the first aspect.

[0041] Fourthly, the present invention discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the functional testing method as described in the first aspect.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] The functional testing method for observability platforms proposed in this invention enables functional testing of observability platforms. By calling APIs, it verifies whether the observability platform's functions are abnormal. Testers can complete the writing, execution, collection, and analysis of test cases without developer intervention. Test cases can be executed in a distributed manner on the server side, enabling large-scale version regression testing without manual intervention, thus saving manpower. The observability data and results generated during the testing process are automatically output to the observability platform, eliminating the dependency lock-in with the testing platform. Attached Figure Description

[0044] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0045] Figure 1 Flowchart of the functional testing method for the observability platform provided in this embodiment of the invention;

[0046] Figure 2 A schematic diagram of the functional testing device for the observability platform provided in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the resource configuration process provided in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of regression testing interaction provided in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0050] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. However, those skilled in the art will understand that the embodiments described below are some embodiments of the present invention, but not all embodiments, and are only used to illustrate the present invention, and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] To more clearly illustrate the technical solutions in this invention, specific embodiments are described below.

[0052] Example

[0053] Reference Figure 1 As shown, this invention discloses a functional testing method for an observability platform, comprising the following steps:

[0054] S1. Write and store custom test cases;

[0055] S2. Input the version to be regressed, calculate the regression test cases for the version based on the version information and the test cases, and form a regression test case set;

[0056] S3. Request computing resources based on the set of regression test cases, and create Pods in real time based on the computing test resources;

[0057] S4. Execute the regression test case in the Pod, test the regression test case, obtain the test results and cache them;

[0058] S5. Read the test results and summarize the test-related information, convert them into line protocol format, and output them to the observability platform for secondary processing and analysis.

[0059] The actual operation of this invention follows the following process, specifically as follows:

[0060] The user wrote the following test cases using the JMeter client tool, see below. Figure 4 As shown:

[0061] 1. Send a metric data to the ` / write / metrics` interface of the observability platform, where the metric values ​​are: vcpu=1, vmem=10Mb, and the tag is: somekey=somevalue.

[0062] 2. Query indicator data from the observability platform, with the query condition: somekey = somevalue.

[0063] 3. The test case version is generated hash ais8a9sd, the associated observability platform version is v1.0.0, and the union version number is v1.0.0-ais8a9sd.

[0064] Users submit test cases to the testing device, inputting the version v1.0.0 to be regressed. The testing device calculates all eligible regression test cases under v1.0.0 based on the version information and the test cases, forming a regression test case set. It then requests computing resources based on this set, invokes the Kubernetes Virtual Kubelet to allocate independent runtime environments for the test cases, and starts execution. (See [link to documentation]). Figure 3 As shown. After the test cases are executed, the testing device will upload the test results to the cache file. Users can filter the test cases for version v1.0.0 on the observability platform, view the test results, and finally complete the version regression test.

[0065] The test results include the following:

[0066] 1) Test case definition information, including the test case name, test case version, associated observability platform version, joint version number, and custom tags;

[0067] 2) Test case runtime environment information, including the name, type, CPU, memory, disk, and grid of the test case runtime environment;

[0068] 3) Test case execution information, including test case execution results, execution time, execution logs, etc.

[0069] The present invention also provides a functional testing apparatus for an observability platform, such as... Figure 2 As shown, it specifically includes:

[0070] Test case management module: Stores custom test cases;

[0071] Regression test case calculation module: Input the version to be regressed, calculate the regression test cases for the version based on the version information and the test cases, and form a set of regression test cases;

[0072] Pod scheduling module: Requests computing resources based on the set of regression test cases, and creates Pods in real time based on the computing test resources;

[0073] Test result processing module: Executes the regression test case in the Pod, tests the regression test case, obtains the test results, and caches them;

[0074] Data output module: Reads the test results and summarizes the relevant test information, converts it into a line protocol format, and outputs it to the observability platform for further processing and analysis.

[0075] The device is mainly composed of the five modules mentioned above. By building this system, the goal of parallel operation can be achieved by mounting the same file system at the same time.

[0076] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of each unit, please refer to the previous method implementation examples, which will not be repeated here.

[0077] Figure 5 This is a schematic diagram of the structure of a computer device disclosed in this invention. (Reference) Figure 5 As shown, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor via a communication bus 403 and is used to store computer instructions executable by the processor 401. The processor 401 is used to read computer instructions from the memory 402 to implement the steps of the functional testing method described in the above embodiments.

[0078] For the above-described apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0079] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal disks or removable disks), magneto-optical disks, and CD-ROMs and DVD-ROMs. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0080] Finally, it should be noted that although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe the features of specific embodiments of a particular invention. Certain features described in the various embodiments of this specification may also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0081] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0082] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A functional testing method for an observability platform, characterized in that, Includes the following steps: S1. Write and store custom test cases; S2. Input the version to be regressed, calculate the regression test cases for the version based on the version information and the test cases, and form a regression test case set; S3. Request computing resources based on the set of regression test cases, and create Pods in real time based on the computing test resources; S4. Execute the regression test case in the Pod, test the regression test case, obtain the test results and cache them; S5. Read the test results and summarize the test-related information, convert them into line protocol format and output them to the observability platform for secondary processing and analysis; In step S3, the method for requesting computing resources includes dynamically calling Kubernetes VirtualKubelet to request computing resources; In step S3, the test resources include the name of the test resources, the type of the test resources, the configuration of the test resources, and the environmental information of the test resources.

2. The functional testing method according to claim 1, characterized in that, In step S1, the definition information of the test case includes the test device, test case information, version control information, and custom tags.

3. The functional testing method according to claim 2, characterized in that, The test-related information includes the definition information of the test cases, the runtime environment information of the test cases, and the execution information of the test cases.

4. The functional testing method according to claim 1, characterized in that, In step S4, the test results include the API address, parameters, error message, response time, and whether they meet expectations.

5. The functional testing method according to claim 1, characterized in that, In step S5, the secondary processing and analysis includes verifying whether the observability platform is functioning correctly by calling the API.

6. An apparatus for testing the functionality of an observable platform, using the method described in any one of claims 1-5, characterized in that, include: Test case management module: Stores custom test cases; Regression test case calculation module: Input the version to be regressed, calculate the regression test cases for the version based on the version information and the test cases, and form a set of regression test cases; Pod scheduling module: Requests computing resources based on the set of regression test cases, and creates Pods in real time based on the computing test resources; Test result processing module: Executes the regression test case in the Pod, tests the regression test case, obtains the test results, and caches them; Data output module: Reads the test results and summarizes the test-related information, converts it into line protocol format, and outputs it to the observability platform for secondary processing and analysis.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the steps of the functional testing method according to any one of claims 1-5.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the functional testing method as described in any one of claims 1-5.

Images