Jmeter-based assertion health metric method

Abstract

The disclosure provides a Jmeter-based interface verification method, which comprises: converting the obtained response content of the server side and the database query result into first and second result sets respectively, and setting the obtained interface design expected value as a third result set; extracting the result set to be compared in the first result set and comparing it with the second result set or the third result set to determine whether the assertion is successful. The disclosure also provides a Jmeter-based assertion healthiness measurement method, which comprises: configuring a Jmeter test script to add an assertion expected value in a view result control; using the foregoing interface verification method, verifying a plurality of interfaces based on the configured Jmeter test script to generate a plurality of corresponding assertion results; and determining the proportion of empty assertions in the plurality of assertion results according to the corresponding assertion expected values in the plurality of assertion results, and then determining the healthiness of the assertion at least according to the proportion.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a JMeter-based interface verification method and apparatus, computing device, computer-readable storage medium, and computer program product. Furthermore, this disclosure also relates to a JMeter-based assertion health measurement method. Background Technology

[0002] JMeter is a Java-based stress testing tool used for stress testing software. JMeter supports assertions, which determine the correctness of API tests by comparing expected and actual return values. Currently, JMeter offers various assertion methods, such as Beanshell assertions, JSON assertions, and response assertions.

[0003] In related technologies, the server-side response and database query results can be obtained and compared. If the comparison results are inconsistent, the assertion fails; if they are consistent, the assertion succeeds. However, database data is easily contaminated, and frequent regression testing may inadvertently corrupt the data. Furthermore, database query results may not match the expected values ​​of the interface design, negatively impacting the accuracy of the assertion results. Summary of the Invention

[0004] In view of this, this disclosure provides a JMeter-based interface verification method and apparatus, computing device, computer-readable storage medium, and computer program product to alleviate, mitigate, or even eliminate the aforementioned problems.

[0005] According to one aspect of this disclosure, a JMeter-based interface verification method is provided, characterized in that the method includes: obtaining server-side response content, database query results, and interface design expectation values; converting the response content and the database query results into a first result set and a second result set, respectively, and setting the interface design expectation value into a third result set; extracting a result set to be compared from the first result set and comparing it with the second result set or the third result set; asserting success when the comparison result meets any of the following conditions: the result set to be compared is consistent with the third result set; and the result set to be compared is consistent with the second result set, and the second result set is consistent with the third result set.

[0006] According to some embodiments of this disclosure, the result set to be compared is consistent with the third result set when the following condition is met: the meaning of the field values ​​of each record in the result set to be compared is consistent with that in the third result set.

[0007] According to some embodiments of this disclosure, the result set to be compared is consistent with the second result set when the following conditions are met: the total number of records in the result set to be compared is consistent with the total number of records in the second result set; and the meaning of the field values ​​of each record in the result set to be compared is consistent with the meaning of the field values ​​in the second result set.

[0008] According to some embodiments of this disclosure, the second result set is consistent with the third result set when the following condition is met: the meaning of the field values ​​of each record in the second result set and the third result set is consistent.

[0009] According to some embodiments of this disclosure, the first result set, the second result set, and the third result set are all in JSON format.

[0010] According to another aspect of this disclosure, a method for measuring assertion health based on JMeter is provided, characterized in that the method includes: configuring a JMeter test script to add assertion expectation values ​​to a result view control; using the JMeter-based interface verification method provided by the foregoing aspect of this disclosure, verifying multiple interfaces based on the configured JMeter test script to generate corresponding multiple assertion results; determining a first proportion of empty assertions in the multiple assertion results based on the corresponding assertion expectation values ​​in the multiple assertion results; and determining the health of the assertions based at least on the first proportion.

[0011] According to some embodiments of this disclosure, determining the health of an assertion based at least on the first proportion includes: determining a second proportion of non-empty assertions among the plurality of assertion results based on the first proportion, and using the second proportion as the health.

[0012] According to some embodiments of this disclosure, determining the health of an assertion based at least on the first proportion includes: determining the health of an assertion based on the first proportion and the weight of an empty assertion, wherein the weight of the empty assertion is determined at least based on the type of interface test corresponding to the empty assertion.

[0013] According to some embodiments of this disclosure, the weight of the empty assertion is determined based on the type of interface test corresponding to the empty assertion and the importance of the business process.

[0014] According to some embodiments of this disclosure, the types of interface tests corresponding to the empty assertion include single interface tests and business process tests.

[0015] According to another aspect of this disclosure, a JMeter-based interface verification device is provided, characterized in that the device comprises: an information acquisition module configured to acquire server-side response content, database query results, and interface design expectation values; an information conversion module configured to convert the response content and the database query results into a first result set and a second result set, respectively, and to set the interface design expectation values ​​into a third result set; and an information comparison module configured to extract a result set to be compared from the first result set and compare it with the second result set or the third result set, and assert success when the comparison result meets any of the following conditions: the result set to be compared is consistent with the third result set; and the result set to be compared is consistent with the second result set, and the second result set is consistent with the third result set.

[0016] According to another aspect of this disclosure, a JMeter-based assertion health measurement device is provided, characterized in that the device comprises: a script configuration module configured to configure a JMeter test script to add assertion expectation values ​​to a result view control; an interface verification module configured to use the JMeter-based interface verification device provided by the foregoing aspect of this disclosure to verify multiple interfaces based on the configured JMeter test script to generate corresponding multiple assertion results; a proportion determination module configured to determine a first proportion of empty assertions in the multiple assertion results based on the corresponding assertion expectation values ​​in the multiple assertion results; and a health determination module configured to determine the health of the assertions based at least on the first proportion.

[0017] According to another aspect of this disclosure, a computing device is provided, characterized in that the computing device includes: a memory configured to store computer-executable instructions; and a processor configured to perform any of the methods provided according to the foregoing aspects of this disclosure when the computer-executable instructions are executed by the processor.

[0018] According to another aspect of this disclosure, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores computer-executable instructions that, when executed, perform any of the methods provided according to the foregoing aspects of this disclosure.

[0019] According to another aspect of this disclosure, a computer program product is provided, characterized in that the computer program product includes computer-executable instructions that, when executed by a processor, perform any of the methods provided according to the foregoing aspects of this disclosure.

[0020] According to the JMeter-based interface verification method provided in this disclosure, the obtained server-side response content and database query results can be converted into a first result set and a second result set, respectively, and the obtained interface design expectation value can be set as a third result set. Then, the result set to be compared in the first result set is extracted and compared with either the second or third result set. An assertion is successful when the comparison result meets any of the following conditions: the result set to be compared is consistent with the third result set; or the result set to be compared is consistent with the second result set, and the second result set is consistent with the third result set. Through this comparison process, highly efficient automated interface verification can be achieved while ensuring the accuracy of the assertion results. Furthermore, by comparing the interface design expectation value with the server-side response content to determine whether the assertion is successful, database data contamination can be reduced or even avoided.

[0021] These and other aspects of this disclosure will be clear from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0022] In the following description of exemplary embodiments in conjunction with the accompanying drawings, further details, features, and advantages of the present disclosure are disclosed, in which: Figure 1 The illustration shows an example flowchart of a JMeter-based interface verification method according to some embodiments of the present disclosure; Figure 2 Schematic illustration Figure 1 A schematic diagram of an example of a JMeter-based interface verification method; Figure 3 The illustration shows an example flowchart of a JMeter-based assertion health measurement method according to some embodiments of the present disclosure; Figure 4 Schematic illustration Figure 3 A schematic diagram illustrating the JMeter-based assertion health measurement method. Figure 5 The illustration schematically shows an example block diagram of a JMeter-based interface verification apparatus according to some embodiments of the present disclosure; Figure 6 The illustration schematically shows an example block diagram of a Jmeter-based assertion health measurement device according to some embodiments of the present disclosure; Figure 7 An example system is illustrated, which includes an example computing device representing one or more systems and / or devices that can implement the various technologies described herein. Detailed Implementation

[0023] Several embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement the technical solutions of this disclosure. The technical solutions of this disclosure can be embodied in many different forms and for many different purposes, and should not be limited to the embodiments set forth herein. These embodiments are provided to make the technical solutions of this disclosure clear and complete, but the described embodiments do not limit the scope of protection of this disclosure.

[0024] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the relevant field and / or the context of this specification, and will not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0025] Figure 1 An example flowchart of a JMeter-based interface verification method (hereinafter referred to as interface verification method 100 for brevity) according to some embodiments of the present disclosure is illustrated.

[0026] Specifically, in step 110, the server's response content, database query results, and expected interface design values ​​are obtained. For example, an interface verification request can be sent to the server from a client connected to the server (e.g., a web browser, mobile app, etc.), and the server accepts the request and returns a response (content) to the client. Alternatively, data corresponding to the interface verification request can be queried from the relevant database as the database query result. The expected interface design values ​​are the values ​​determined during the interface design phase.

[0027] In step 120, the response content and the database query results are converted into a first result set and a second result set, respectively, and the interface design expectation value is set to a third result set. The purpose of these conversion and setting steps is to format the corresponding data (e.g., unify it into JSON format) to facilitate data comparison.

[0028] In step 130, a result set to be compared is extracted from the first result set and compared with the second result set or the third result set. An assertion is successful when the comparison result satisfies any of the following conditions: the result set to be compared is consistent with the third result set; or the result set to be compared is consistent with the second result set, and the second result set is consistent with the third result set. It should be noted that the above two conditions are independent. That is, the success of the assertion can be determined by comparing only whether the result set to be compared is consistent with the third result set, or by comparing whether the result set to be compared is consistent with the second result set, and also by comparing whether the second result set is consistent with the third result set. Alternatively, the above process can be performed in parallel. That is, a portion of the result sets to be compared in the first result set are compared using the first condition (i.e., comparing the corresponding result set to be compared with the third result set), while another portion of the result sets to be compared are compared using the second condition (i.e., comparing whether the corresponding result set to be compared is consistent with the second result set, and also comparing whether the second result set is consistent with the third result set).

[0029] For example, the first condition is met when the meaning of the field values ​​of each record in the result set to be compared is consistent with the meaning of the field values ​​in the third result set; that is, the result set to be compared is consistent with the third result set. Furthermore, the result set to be compared is consistent with the second result set when, for example, the total number of records in the result set to be compared is consistent with the total number of records in the second result set; or the meaning of the field values ​​of each record in the result set to be compared is consistent with the meaning of the field values ​​in the second result set. The second result set is consistent with the third result set when the meaning of the field values ​​of each record in the second result set is consistent with the meaning of the field values ​​in the third result set.

[0030] Interface verification method 100 enables highly efficient automated interface verification (interface verification method 100 can be implemented as a script and can be run on the corresponding server or client), and ensures the accuracy of assertion results. Specifically, when comparing the expected value of the interface design with the server's response to determine whether an assertion is successful, it can also reduce or even avoid database data corruption. For example, in the first result set, a portion of the result sets to be compared are compared using the first condition (i.e., comparing the corresponding result sets to be compared with the third result set), while another portion of the result sets to be compared are compared using the second condition (i.e., comparing whether the corresponding result sets to be compared are consistent with the second result set, and comparing whether the second result set is consistent with the third result set). Therefore, compared to related technologies that only compare database query results with the server's response, interface verification method 100 can reduce the possibility of database data corruption. The following is a further explanation... Figure 2 To further clarify the principle of interface verification method 100.

[0031] like Figure 2 As shown, the first result set 210, the second result set 220, and the third result set 230 are obtained from the server's response content, database query results, and the expected values ​​of the interface design, respectively, through conversion or setting. The result set to be compared, extracted from the first result set, can then be compared with the second result set 220 or the third result set 230. When the comparison result satisfies the two conditions described in step 130 of the interface verification method 100 above, an assertion of success is made.

[0032] Specifically, when the result set to be compared matches the third result set 230, the assertion succeeds. When the result set to be compared matches the second result set 220, a verification is performed, that is, a further comparison is made between the second result set 220 and the third result set 230 to see if they match. If the second result set 220 matches the third result set 230, the assertion succeeds. Conversely, when the result set to be compared does not match the third result set 230, the assertion fails. Additionally, when the result set to be compared matches the second result set 220, a verification is performed; if the second result set 220 does not match the third result set 230, the assertion fails.

[0033] The inventors discovered that current JMeter test results do not statistically analyze assertion success and failure rates. Even without assertions, the API test result is still considered successful. Therefore, the current API test result "RequestsSummary" cannot accurately measure assertion health. Furthermore, API tests are not of a single type; for example, they can include single-interface and business process tests. In practical applications, different interfaces may have different levels of importance. However, related technologies do not assign weights to assertions in test scripts. Exemplarily, this can be achieved through embodiments described below (e.g., regarding...). Figure 3 and Figure 4 The described embodiments are used to alleviate or even resolve at least one of the above problems.

[0034] Figure 3 An example flowchart illustrating a JMeter-based assertion health measurement method (hereinafter referred to as measurement method 300 for brevity) according to some embodiments of the present disclosure is shown.

[0035] Specifically, in step 310, the JMeter test script is configured to add assertion expectation values ​​to the results view control. The expected value of the assertion is displayed regardless of whether the assertion succeeds or fails (the current FailureMessage in the results view control only displays the assertion expectation value for failed assertions; it should also display the assertion expectation value for successful assertions). For example, the results view control can be configured to add an item to indicate the assertion expectation value, such as naming it "Assertion expectation". This allows for a centralized display of the corresponding interface expectation value, uniquely bound to the interface ID, facilitating interface testers' intuitive viewing of empty assertion cases and convenient centralized verification of assertion correctness. For example, each HTTP request can be labeled with the environment and test number used, for example, labeled as "${env}-interface test case number-'business process Chinese name'". Here, "${env}" describes different environments, such as the test environment. Furthermore, the labels in the assertion results can be matched one-to-one with the assertion expectation values, enabling centralized verification of assertion expectation values.

[0036] In step 320, the interface verification method 100 provided by the foregoing aspects of this disclosure is used to verify multiple interfaces based on the Jmeter test script configured in step 310 above, so as to generate multiple assertion results.

[0037] In step 330, based on the corresponding assertion expectation values ​​among the plurality of assertion results, a first proportion of empty assertions among the plurality of assertion results is determined. For example, when the assertion expectation value is empty, the corresponding assertion can be determined as an empty assertion. Through the corresponding assertion expectation values ​​among the plurality of assertion results, the number of empty assertions can be determined, and thus the proportion of empty assertions among the plurality of assertion results (i.e., the first proportion) can be determined.

[0038] In step 340, the health of the assertion is determined at least based on the first proportion. For example, a second proportion of non-empty assertions among the plurality of assertion results can be determined based on the first proportion, and this second proportion can be used as the health. For example, if N represents the total number of empty assertions, then the first proportion is N / T, where T represents the total number of the plurality of assertion results. In this case, the second proportion equals 1 - N / T = (TN) / T, and (TN) / T can be used as the health. The higher the health value, the smaller the total number of empty assertions, and the more reliable (healthy) the assertion result.

[0039] As another example, the health of an assertion can be determined based on the first proportion and the weight of the empty assertion, wherein the weight of the empty assertion is determined at least according to the type of interface test corresponding to the empty assertion. The type of interface test corresponding to the empty assertion can include single interface tests and business process tests. The weight of the empty assertion can be any number between 0 and 1. For example, w i Let represent the weight of the i-th empty assertion. Then, the health of the assertion can be represented as follows: .

[0040] It should be noted that in this disclosure, the weight of an empty assertion can be determined based on the type of interface test corresponding to the empty assertion, or it can be determined by combining the type of interface test corresponding to the empty assertion and the importance of the business process. For example, a higher weight can be assigned to empty assertions corresponding to business processes with higher importance.

[0041] Metric 300 allows for a more accurate measurement of assertion health by statistically analyzing the proportion of empty assertions. Furthermore, Metric 300 allows for weighting assertions in the test script, further refining the assessment of assertion health. The following section will explain... Figure 4 To further clarify the principle of measurement method 300.

[0042] like Figure 4As shown, in step 410, a test script is run to generate assertion results. For example, a JMeter test script (configured via step 310 above) can be run to verify multiple interfaces, thereby generating multiple corresponding assertion results. In step 420, it is determined whether the corresponding assertion expectation value among the multiple assertion results is empty. It should be noted that the assertion expectation value in step 420 can be obtained through the following pre-configuration: add the assertion expectation value "Assertion expectation" to the configure function of the View Results inTable control in JMeter, and select "label", "responseCode", "responseMessage", "success", "FailureMessage", "Assertionexpectation", and "URL" as the result set. If, in step 420, it is determined that the expected value of the corresponding assertion among the multiple assertion results is empty, then in step 430, the number of empty assertions N can be updated (i.e., increased by 1 from the original value) and an error flag can be issued (this flag can be used to indicate that the current verification result has failed; in the original interface test script request, if no assertion is written, the test will still succeed, which is illogical. We use this error flag to make the current test fail, thereby accurately assessing the health of the script). Then, in step 440, the first proportion of empty assertions among the multiple assertion results can be determined. If, in step 420, it is determined that the expected value of the corresponding assertion among the multiple assertion results is not empty, then it means that the assertion is not empty, and then a comparison can be performed in step 450. Specifically, the corresponding test case number can be found according to the "label", the corresponding interface address can be found according to the "URL", and the values ​​of "responseCode", "responseMessage", "success", "FailureMessage", and "Assertion expectation" can be compared to determine whether the assertion is written correctly, the error message content, and whether the interface test has passed. Finally, in step 460, the health of the assertion is determined at least based on the first percentage (e.g., using the method described above with respect to step 340).

[0043] Figure 5 The illustration shows an example block diagram of a JMeter-based interface verification apparatus (hereinafter referred to as interface verification apparatus 500 for simplicity) according to some embodiments of the present disclosure. Figure 5 As shown, the interface verification device 500 includes an information acquisition module 510, an information conversion module 520, and an information comparison module 530.

[0044] Specifically, the information acquisition module 510 can be configured to: acquire the response content from the server, the database query results, and the interface design expectation value; the information conversion module 520 can be configured to: convert the response content and the database query results into a first result set and a second result set respectively, and set the interface design expectation value into a third result set; the information comparison module 530 can be configured to: extract the result set to be compared from the first result set, and compare it with the second result set or the third result set, and assert success when the comparison result meets any of the following conditions: the result set to be compared is consistent with the third result set; and the result set to be compared is consistent with the second result set, and the second result set is consistent with the third result set.

[0045] Figure 6 An example block diagram of a Jmeter-based assertion health measurement device (hereinafter referred to as assertion health measurement device 600 for brevity) according to some embodiments of the present disclosure is illustrated. Figure 6 As shown, the assertion health measurement device 600 includes a script configuration module 610, an interface verification module 620, a percentage determination module 630, and a health determination module 640.

[0046] Specifically, the script configuration module 610 can be configured to: configure the Jmeter test script to add assertion expectation values ​​to the results view control; the interface verification module 620 can be configured to: use the interface verification device 500 provided by the foregoing aspects of this disclosure to verify multiple interfaces based on the configured Jmeter test script to generate corresponding multiple assertion results; the proportion determination module 630 can be configured to: determine a first proportion of empty assertions in the multiple assertion results according to the corresponding assertion expectation values ​​in the multiple assertion results; and the health determination module 640 can be configured to: determine the health of the assertions at least according to the first proportion.

[0047] It should be understood that both the interface verification device 500 and the assertion health measurement device 600 can be implemented in software, hardware, or a combination of both. Multiple different modules within the interface verification device 500 and the assertion health measurement device 600 can be implemented within the same software or hardware architecture, or a single module can be implemented by multiple different software or hardware architectures.

[0048] Furthermore, the interface verification device 500 and the assertion health measurement device 600 can be used to implement the interface verification method 100 and the measurement method 300 described above, respectively. The relevant details have been described in detail above and will not be repeated here for the sake of brevity. The interface verification device 500 and the assertion health measurement device 600 can each have the same features and advantages as described with respect to the aforementioned interface verification method 100 and measurement method 300.

[0049] Figure 7 An example system is illustrated, including an example computing device 700 representing one or more systems and / or devices that can implement the various technologies described herein. The computing device 700 may be, for example, a server of a service provider, a device associated with a server, a system-on-a-chip, and / or any other suitable computing device or computing system. Any one or more of the interface verification device 500 and the assertion health measurement device 600 described above may take the form of the computing device 700. Alternatively, the interface verification device 500 and the assertion health measurement device 600 may be implemented as a computer program in the form of application 716.

[0050] like Figure 7 The illustrated example computing device 700 includes a processing system 711 communicatively coupled to each other, one or more computer-readable media 712, and one or more I / O interfaces 713. Although not shown, the computing device 700 may also include a system bus or other data and command transfer system that couples the various components to each other. The system bus may include any or a combination of different bus architectures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of the various bus architectures. Various other examples, such as control and data lines, are also conceived.

[0051] Processing system 711 represents the functionality of performing one or more operations using hardware. Therefore, processing system 711 is illustrated as including hardware elements 714 that can be configured as processors, function blocks, etc. This may include application-specific integrated circuits (ASICs) or other logic devices formed using one or more semiconductors in the hardware. Hardware element 714 is not limited by the materials in which it is formed or the processing mechanism employed therein. For example, a processor may consist of semiconductors and / or transistors (e.g., integrated circuits (ICs)). In such a context, processor-executable instructions may be electronically executable instructions.

[0052] Computer-readable medium 712 is illustrated as including memory / storage device 715. Memory / storage device 715 represents a memory / storage capacity associated with one or more computer-readable media. Memory / storage device 715 may include volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disk, magnetic disk, etc.). Memory / storage device 715 may include fixed media (e.g., RAM, ROM, fixed hard disk drive, etc.) and removable media (e.g., flash memory, removable hard disk drive, optical disk, etc.). Computer-readable medium 712 may be configured in various other ways as further described below.

[0053] One or more I / O interfaces 713 represent the functionality to allow users to input commands and information to the computing device 700 using various input devices and optionally also to present information to the user and / or other components or devices using various output devices. Examples of input devices include keyboards, cursor control devices (e.g., mice), microphones (e.g., for voice input), scanners, touch functionality (e.g., capacitive or other sensors configured to detect physical touch), cameras (e.g., capable of detecting non-touch-related movements as gestures using visible or invisible wavelengths (such as infrared frequencies), etc. Examples of output devices include display devices (e.g., monitors or projectors), speakers, printers, network interface cards, haptic-responsive devices, etc. Therefore, the computing device 700 can be configured to support user interaction in various ways as further described below.

[0054] The computing device 700 also includes an application 716. The application 716 may be, for example, a software instance of an interface verification device 500 or an assertion health measurement device 600, and may implement the techniques described herein in combination with other elements in the computing device 700.

[0055] This document describes various technologies within the general context of software and hardware components or program modules. Generally, these modules include routines, programs, elements, components, data structures, etc., that perform specific tasks or implement specific abstract data types. As used herein, the terms "module," "function," and "component" generally refer to software, firmware, hardware, or a combination thereof. The technologies described herein are characterized as platform-independent, meaning that these technologies can be implemented on a variety of computing platforms with various processors.

[0056] Implementations of the described modules and technologies may be stored on or transmitted across some form of computer-readable medium. The computer-readable medium may include a variety of media accessible by the computing device 700. By way of example and not limitation, the computer-readable medium may include "computer-readable storage media" and "computer-readable signal media".

[0057] In contrast to simple signal transmission, carrier waves, or signals themselves, a "computer-readable storage medium" refers to a medium and / or device capable of persistently storing information, and / or a tangible storage device. Therefore, a computer-readable storage medium refers to a non-signal-bearing medium. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices implemented using methods or techniques suitable for storing information (such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data). Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage devices, hard disks, cassette tapes, magnetic tapes, disk storage devices or other magnetic storage devices, or other storage devices, tangible media, or articles of art suitable for storing desired information and accessible by a computer.

[0058] "Computer-readable signal medium" refers to a signal-bearing medium configured to transmit instructions, such as via a network, to computing device 700. A signal medium typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, data signal, or other transmission mechanism. Signal media also include any information transmission medium. The term "modulated data signal" refers to a signal in which one or more of its characteristics are set or altered to encode information. By way of example and not limitation, communication media include wired media such as wired networks or direct connections, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0059] As previously described, hardware element 714 and computer-readable medium 712 represent instructions, modules, programmable device logic, and / or fixed device logic implemented in hardware, which in some embodiments can be used to implement at least some aspects of the techniques described herein. Hardware elements may include components of integrated circuits or systems-on-a-chip, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations or other hardware devices in silicon. In this context, hardware elements can serve as processing devices for executing program tasks defined by instructions, modules, and / or logic embodied by the hardware element, and as hardware devices for storing instructions for execution, such as the previously described computer-readable storage medium.

[0060] The foregoing combinations can also be used to implement the various techniques and modules described herein. Therefore, software, hardware, or program modules and other program modules can be implemented as one or more instructions and / or logic embodied on some form of computer-readable storage medium and / or by one or more hardware elements 714. The computing device 700 can be configured to implement specific instructions and / or functions corresponding to the software and / or hardware modules. Thus, for example, by using the computer-readable storage medium and / or hardware elements 714 of a processing system, modules can be implemented at least partially in hardware as modules executable as software by the computing device 700. Instructions and / or functions can be executable / operable by one or more articles of art (e.g., one or more computing devices 700 and / or processing systems 711) to implement the techniques, modules, and examples described herein.

[0061] In various embodiments, the computing device 700 can be configured in various ways. For example, the computing device 700 can be implemented as a computer-type device, including personal computers, desktop computers, multi-screen computers, laptop computers, netbooks, etc. The computing device 700 can also be implemented as a mobile device, including mobile devices such as mobile phones, portable music players, portable gaming devices, tablet computers, multi-screen computers, etc. The computing device 700 can also be implemented as a television-type device, including devices with or connected to a generally large screen in a leisure viewing environment. These devices include televisions, set-top boxes, game consoles, etc.

[0062] The techniques described herein can be supported by these various configurations of computing device 700, and are not limited to specific examples of the techniques described herein. Functionality can also be implemented, wholly or partially, on the “cloud” 720 using distributed systems, such as through platform 722 as described below.

[0063] Cloud 720 includes and / or represents platform 722 for resource 724. Platform 722 abstracts the underlying functionality of the hardware (e.g., server) and software resources of cloud 720. Resource 724 may include applications and / or data that can be used when performing computer processing on a server located remotely from computing device 700. Resource 724 may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks.

[0064] Platform 722 can abstract resources and functions to connect computing device 700 to other computing devices. Platform 722 can also be used to abstract resource hierarchy to provide a corresponding level of hierarchy for any encountered needs for resource 724 implemented via platform 722. Therefore, in interconnect device embodiments, the implementation of the functions described herein can be distributed throughout system 700. For example, functions can be implemented partly on computing device 700 and partly through platform 722, which abstracts the functions of cloud 720.

[0065] It should be understood that, for clarity, embodiments of this disclosure have been described with reference to different functional units. However, it will be apparent that, without departing from this disclosure, the functionality of each functional unit may be implemented in a single unit, in multiple units, or as part of other functional units. For example, functionality described as being performed by a single unit may be performed by multiple different units. Therefore, references to a particular functional unit are considered merely as references to the appropriate unit used to provide the described functionality, and not as indicating a strict logical or physical structure or organization. Thus, this disclosure may be implemented in a single unit, or may be physically and functionally distributed among different units and circuits.

[0066] It will be understood that although the terms first, second, third, etc., may be used herein to describe various devices, elements, components, or parts, these devices, elements, components, or parts should not be limited by these terms. These terms are used only to distinguish one device, element, component, or part from another device, element, component, or part.

[0067] Although this disclosure has been described in conjunction with some embodiments, it is not intended to be limited to the specific forms set forth herein. Rather, the scope of this disclosure is limited only by the appended claims. Additionally, although individual features may be included in different claims, these may be advantageously combined, and inclusion in different claims does not imply that such a combination of features is not feasible and / or advantageous. The order of features in the claims does not imply that the features must be in any particular order of their operation. Furthermore, in the claims, the word "comprising" does not exclude other elements, and the terms "a" or "an" do not exclude a plurality. Reference numerals in the claims are provided only by way of explicit example and should not be construed as limiting the scope of the claims in any way.

[0068] It should be understood that, for clarity, embodiments of this disclosure have been described with reference to different functional units. However, it will be apparent that, without departing from this disclosure, the functionality of each functional unit may be implemented in a single unit, in multiple units, or as part of other functional units. For example, functionality described as being performed by a single unit may be performed by multiple different units. Therefore, references to a particular functional unit are considered merely as references to the appropriate unit used to provide the described functionality, and not as indicating a strict logical or physical structure or organization. Thus, this disclosure may be implemented in a single unit, or may be physically and functionally distributed among different units and circuits.

[0069] This disclosure provides a computer-readable storage medium storing computer-readable instructions that, when executed, implement the aforementioned Jenkins-based data interface verification method.

[0070] This disclosure provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computing device to perform the Jenkins-based data interface verification method provided in the various optional implementations described above.

[0071] By studying the accompanying drawings, the disclosure, and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments in practicing the claimed subject matter. In the claims, the word "comprising" does not exclude other elements or steps, and "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously.

Claims

1. A method for asserting health metrics based on JMeter, characterized in that, The method includes: Configure the JMeter test script to add assertion expectations to the results view control; The interface verification method based on JMeter is used to verify multiple interfaces based on the configured JMeter test script to generate multiple corresponding assertion results; Based on the corresponding assertion expectation values ​​in the plurality of assertion results, determine the first proportion of empty assertions in the plurality of assertion results; and The health of the assertion is determined at least based on the first percentage. The JMeter-based interface verification method includes: Obtain the server's response content, database query results, and expected values ​​from the interface design; The response content and the database query results are respectively converted into a first result set and a second result set, and the interface design expectation value is set as a third result set; Extract the result set to be compared from the first result set and compare it with the second result set or the third result set. An assertion is successful when the comparison result meets any of the following conditions: The result set to be compared is consistent with the third result set; and The result set to be compared is consistent with the second result set, and the second result set is consistent with the third result set.

2. The method according to claim 1, characterized in that, Determining the health of the assertion based at least on the first proportion includes: Based on the first proportion, a second proportion of non-empty assertions is determined among the plurality of assertion results, and the second proportion is used as the health level.

3. The method according to claim 1, characterized in that, Determining the health of the assertion based at least on the first proportion includes: The health of the assertion is determined based on the first proportion and the weight of the empty assertion, wherein, The weight of the empty assertion is determined at least according to the type of interface test corresponding to the empty assertion.

4. The method according to claim 3, characterized in that, The weight of the empty assertion is determined based on the type of interface test corresponding to the empty assertion and the importance of the business process.

5. The method according to claim 4, characterized in that, The types of interface tests corresponding to the empty assertion include single interface tests and business process tests.

6. The method according to claim 1, characterized in that, The result set to be compared is consistent with the third result set when the following conditions are met: The meaning of the field values ​​of each record in the result set to be compared is consistent with that in the third result set.

7. The method according to claim 1, characterized in that, The result set to be compared is consistent with the second result set when the following conditions are met: The total number of records in the result set to be compared is the same as the total number of records in the second result set; and The meaning of the field values ​​of each record in the result set to be compared is consistent with that in the second result set.

8. The method according to claim 1, characterized in that, The second result set is consistent with the third result set when the following conditions are met: The meanings of the field values ​​for each record in the second result set and the third result set are consistent.

9. The method according to claim 1, characterized in that, The first result set, the second result set, and the third result set are all in JSON format.

10. A Jmeter-based assertion health measurement device, characterized in that, The device includes: The script configuration module is configured to configure JMeter test scripts to add assertion expectations to the results view control; The interface verification module is configured to use a JMeter-based interface verification method to verify multiple interfaces based on a configured JMeter test script, thereby generating multiple assertion results. The proportion determination module is configured to determine a first proportion of empty assertions among the multiple assertion results based on the corresponding assertion expectation values ​​among the multiple assertion results; and A health determination module is configured to determine the asserted health level based at least on the first percentage. The JMeter-based interface verification method includes: Obtain the server's response content, database query results, and expected values ​​from the interface design; The response content and the database query results are respectively converted into a first result set and a second result set, and the interface design expectation value is set as a third result set; Extract the result set to be compared from the first result set and compare it with the second result set or the third result set. An assertion is successful when the comparison result meets any of the following conditions: The result set to be compared is consistent with the third result set; and The result set to be compared is consistent with the second result set, and the second result set is consistent with the third result set.

11. A computing device, characterized in that, The computing device includes: Memory, which is configured to store computer-executable instructions; A processor configured to perform the method according to any one of claims 1 to 9 when the computer-executable instructions are executed by the processor.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed, perform the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, perform the method according to any one of claims 1 to 9.

Images