A test evaluation method and device

Abstract

The present invention discloses a test evaluation method and device, and relates to the field of computer technology. A specific implementation of the method includes monitoring a triggered test, calling a dynamic proxy class, and collecting data during the test; performing scene matching on the data based on a preset scene library and a preset matching model to obtain a covered scene; calling a preset coverage model to calculate the scene coverage of the test. Thus, the implementation of the present invention can solve the problems of high cost, low efficiency, opaque data, and lack of persuasiveness in existing test scene coverage evaluation.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a test evaluation method and device. Background Art

[0002] Test coverage is an important indicator to measure whether the test execution is comprehensive. Common test coverage rates include code coverage, requirement coverage, and defect coverage. Among them, scenario coverage is to count from the user's perspective how many usage scenarios are covered by the test cases, how many usage scenarios there are in total, and calculate the percentage. The statistics of scenario coverage are calculated by the tester according to the executed test cases during the test process. For example, if there are multiple scenario test cases in total, a test case is manually marked as executed after execution. Finally, the percentage is calculated according to the actual executed test cases and the total test cases to get the scenario coverage.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] At present, the statistical method of scenario coverage is a manual calculation process by testers. The disadvantage is that this statistical method is not intelligent enough. Testers may have forgotten how many scenarios they have tested, and they need to re-execute the test case. If the test case is executed by other test colleagues, they need to communicate with other test colleagues to obtain data. Therefore, the existing scenario coverage statistical method is costly, inefficient, and the data is not transparent and lacks persuasiveness. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a test evaluation method and device, which can solve the problems of high cost, low efficiency, opaque data and lack of persuasiveness in existing test scenario coverage evaluation.

[0006] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, a test evaluation method is provided, including monitoring a triggered test, calling a dynamic proxy class, and collecting data during the test process; performing scene matching on the data based on a preset matching model according to a preset scenario library to obtain covered scenes; calling a preset coverage model to calculate the scene coverage of the test.

[0007] Optionally, collect data during the test, including:

[0008] According to the data request in the test process, the business field in the data request is obtained to generate a corresponding business identifier, and then the business identifier set of the test is obtained; wherein all methods on each data request call chain correspond to the same business identifier;

[0009] According to the preset scene library, scene matching is performed on the data based on a preset matching model to obtain covered scenes, including:

[0010] According to the service identifier set, the covered scenarios are matched based on a preset scenario library; wherein the scenario library includes a mapping relationship between the service identifier and the scenario information.

[0011] Optionally, obtaining a business field in the data request to generate a corresponding business identifier includes:

[0012] Obtain the business field in the data request, determine whether the business field is empty, if so, generate a universal unique identification code as a business identifier according to the data request, otherwise generate a corresponding business identifier according to the business field through a preset algorithm model.

[0013] Optionally, the covered scenarios are obtained based on the service identification set and matching based on a preset scenario library, including:

[0014] If there is a single service identifier corresponding to the scene information in the scene library, matching the single service identifier with the set of service identifiers is performed to obtain corresponding scene information, and then marking the scene information with a preset label; or

[0015] If there are multiple business identifiers corresponding to the scene information in the scene library, the multiple business identifiers are matched with the business identifier set respectively. If all the multiple business identifiers are matched successfully, the corresponding scene information is obtained to mark the scene information with a preset label.

[0016] Optionally, calling a preset coverage model to calculate the scenario coverage of the test includes:

[0017] The scene information marked with preset tags is counted and summed to obtain the number of covered scenes, and then the ratio of the number of covered scenes to the total number of scenes in the scene library is calculated to obtain the scene coverage rate of the test.

[0018] Optionally, the covered scenarios are obtained based on the business identification set and matching based on a preset scenario library, further comprising:

[0019] If there is a business identifier in the business identifier set that has not been successfully matched in the scenario library, the code static analysis method is called to generate a call relationship directed graph of the data request corresponding to the business identifier, so as to map the label of each node in the call relationship directed graph to a positive integer of a preset interval through hash coding, and then convert the mapped call relationship directed graph into an adjacency matrix;

[0020] According to the adjacency matrix, a match is performed with the adjacency matrix corresponding to the service identifier in the scenario library to obtain the successfully matched service identifier in the scenario library, and then determine the matched coverage scenario.

[0021] Optionally, after calculating the scenario coverage of the test, the following steps are performed:

[0022] According to the scenario library, unmatched scenario information is obtained, an uncovered scenario set is generated, and an insufficient test warning message is sent.

[0023] In addition, the present invention also provides a test evaluation device, including an acquisition module, which is used to monitor the triggered test, call the dynamic proxy class, and collect data during the test; a processing module, which is used to perform scene matching on the data based on a preset matching model according to a preset scenario library to obtain covered scenes; and call the preset coverage model to calculate the scene coverage of the test.

[0024] An embodiment of the above invention has the following advantages or beneficial effects: the present invention monitors the trigger test, calls the dynamic proxy class, collects the data in the test process; performs scene matching on the data according to the preset scene library to obtain the covered scenes, and then calculates the scene coverage of the test. Thus, the present invention realizes the automatic and sufficient evaluation process of test scene coverage, making up for the shortcomings of manual test statistics.

[0025] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0027] Figure 1 is a schematic diagram of the main process of the test evaluation method according to the first embodiment of the present invention;

[0028] Figure 2 is a schematic diagram of the main process of the test evaluation method according to the second embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of the main process of the test evaluation method according to the third embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of main modules of a test and evaluation device according to an embodiment of the present invention;

[0031] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0032] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0034] Figure 1 is a schematic diagram of the main process of the test evaluation method according to the first embodiment of the present invention. Figure 1 As shown, the test evaluation method includes:

[0035] Step S101, monitoring the trigger test, calling the dynamic proxy class, and collecting data during the test.

[0036] Among them, the dynamic proxy class is Interface InvocationHandler, which is located in the category under the java.lang.reflect package.

[0037] In some embodiments, collecting data during the test process may specifically include: according to the data request during the test process, obtaining the business field in the data request to generate a corresponding business identifier, and then obtaining the business identifier set of the test. Among them, all methods on each data request call chain correspond to the same business identifier. In other words, the information of the execution class is collected through the dynamic proxy class, and each time a data request is made, a unique business identifier BUID is generated by hashing according to the business field, and multiple business fields generate multiple BUIDs.

[0038] In a further embodiment, obtaining the business field in the data request to generate a corresponding business identifier may specifically include: obtaining the business field in the data request, determining whether the business field is empty, and if so, generating a universally unique identifier as a business identifier according to the data request, and if not, generating a corresponding business identifier according to the business field through a preset algorithm model. That is, if the business field is empty, a universally unique identifier UUID (abbreviation for Universally Unique Identifier) ​​is generated.

[0039] Step S102: performing scene matching on the data based on a preset matching model according to a preset scene library to obtain covered scenes.

[0040] In some embodiments, when executing step S102, the covered scenes can be obtained according to the service identifier set and matched based on a preset scene library, wherein the scene library includes a mapping relationship between the service identifier and the scene information.

[0041] In a further embodiment, if there is a single service identifier corresponding to the scene information in the scene library, the single service identifier is matched with the set of service identifiers to obtain the corresponding scene information, and then the scene information is marked with a preset label. Alternatively, if there are multiple service identifiers corresponding to the scene information in the scene library, the multiple service identifiers are matched with the set of service identifiers respectively, and if all the multiple service identifiers are successfully matched, the corresponding scene information is obtained to mark the scene information with a preset label.

[0042] That is to say, the present invention can execute the preset effective equivalence class judgment logic to match the business identification set with the scene library. Specifically, the scene corresponds to a single BUID, and the business identification set is compared with the BUID in the scene library. If they are the same, the match is successful, and the scene information (such as the scene key) is marked with an overlay tag. If the scene corresponds to multiple BUIDs, iterative cumulative matching is performed multiple times (that is, each BUID is matched separately), and under the condition that the threshold N of each scene request test is met, the BUID set of the scene in the scene library is compared. If the sets are equal, the match is successful, and the scene information (such as the scene key) is marked with an overlay tag. Among them, a single successful match has the highest priority.

[0043] Preferably, a request number threshold may be preset for each scenario, that is, each scenario includes at most N requests (that is, the maximum number of request times).

[0044] It is also worth mentioning that if there is a business identifier in the business identifier set that has not been successfully matched in the scenario library, the code static analysis method is called (code static analysis refers to scanning the program code through lexical analysis, syntax analysis, control flow, data flow analysis and other technologies without running the code to verify whether the code meets the standards, security, reliability, maintainability and other indicators. A code analysis technology.) is generated to generate a call relationship directed graph of the data request corresponding to the business identifier, so as to map the label of each node in the call relationship directed graph to a positive integer in a preset interval through hash coding, and then convert the mapped call relationship directed graph into an adjacency matrix.

[0045] Then, according to the adjacency matrix, the adjacency matrix corresponding to the business identifier in the scenario library is matched to determine whether a successfully matched business identifier in the scenario library is obtained. If so, the matching coverage scenario is determined; otherwise, the unmatched business identifier and the adjacency matrix corresponding to the business identifier are stored in the scenario library, and the scenario is configured to update the scenario library.

[0046] It can be seen that the business identification set collected in the test execution process of the present invention is matched with the scenario library. If the match is successful, the scenario coverage is marked, and if the match is unsuccessful, the scenario library is automatically updated. In addition, when the match is unsuccessful, abnormal data and invalid data can be filtered, and then the scenario information can be configured and updated to the scenario library.

[0047] Preferably, the starting node of the directed graph is calculated from the controller layer, and the id is "starting method#parameter type". The id of each node in the directed graph is "method name#parameter type" and the hash code is mapped to a positive integer in a given interval.

[0048] Step S103, calling a preset coverage model to calculate the scenario coverage of the test.

[0049] In some embodiments, when executing step S103, the specific implementation process includes: counting and summing the scene information marked with preset tags to obtain the number of covered scenes, and then calculating the ratio of the number of covered scenes to the total number of scenes in the scene library to obtain the scene coverage of the test. In addition, the present invention can use a coverage calculation tool such as Jacoco.

[0050] Preferably, the coverage model for calculating the scenario coverage of the test may be:

[0051] Scene coverage = (number of covered scenes / total number of scenes) * 100%

[0052] It is also worth noting that after calculating the scenario coverage of the test, it is also possible to obtain the unmatched scenario information according to the scenario library, generate an uncovered scenario set, and send an insufficient test warning message. In other words, the embodiment of the present invention can report an uncovered scenario set and issue an insufficient test warning. Thus, it can be seen that the present invention can not only evaluate the scenario coverage of the tested case, but also report the uncovered scenario and issue an insufficient coverage warning.

[0053] As some other embodiments, after obtaining the scenario coverage of the test, it can be determined whether the scenario coverage of the test is less than a preset coverage threshold, and if so, a code refactoring risk is prompted, otherwise no processing is performed. Thus, it can be seen that the present invention can also perform risk assessment on test cases.

[0054] Figure 2 FIG. 1 is a schematic diagram of the main flow of the test evaluation method according to the second embodiment of the present invention. Figure 2 As shown, the test evaluation method includes:

[0055] Step S201, monitoring a trigger test and calling a dynamic proxy class.

[0056] Step S202, according to the data request in the test process, obtain the service field in the data request to generate a corresponding service identifier, and then obtain the service identifier set of the test.

[0057] Among them, all methods on each data request call chain correspond to the same business identifier.

[0058] In an embodiment, after obtaining the business field in the data request, it can be determined whether the business field is empty, if so, a universal unique identification code is generated according to the data request as a business identifier, otherwise, a corresponding business identifier is generated according to the business field through a preset algorithm model. Preferably, the corresponding unique business identifier is generated through the business field based on a hash algorithm according to the business field.

[0059] Step S203: According to the service identification set, the covered scenes are matched based on a preset scene library.

[0060] The scenario library includes a mapping relationship between a service identifier and scenario information.

[0061] In an embodiment, if there is a single service identifier corresponding to the scene information in the scene library, the single service identifier is matched with the business identifier set to obtain the corresponding scene information, and then the scene information is marked with a preset label. Or if there are multiple service identifiers corresponding to the scene information in the scene library, the multiple service identifiers are matched with the business identifier set respectively, and if all the multiple service identifiers are successfully matched, the corresponding scene information is obtained to mark the scene information with a preset label.

[0062] Step S204, determining whether there is a business identifier in the business identifier set that has not been successfully matched in the scenario library, if yes, proceed to step S205, if not, proceed directly to step S207.

[0063] Step S205, calling the code static analysis method to generate a call relationship directed graph of the data request corresponding to the business identifier, so as to map the label of each node in the call relationship directed graph to a positive integer in a preset interval through hash coding, and then convert the mapped call relationship directed graph into an adjacency matrix.

[0064] Step S206: Match the adjacency matrix with the adjacency matrix corresponding to the service identifier in the scenario library according to the adjacency matrix to obtain the successfully matched service identifier in the scenario library, and then determine the matched coverage scenario.

[0065] Step S207, count the scenario information marked with the preset label and sum them to obtain the number of covered scenarios, and then calculate the ratio of the number of covered scenarios to the total number of scenarios in the scenario library to obtain the scenario coverage rate of the test.

[0066] Step S208, according to the scenario library, obtain the unmatched scenario information, generate an uncovered scenario set, and send a test insufficiency warning message.

[0067] Figure 3 It is a schematic diagram of the main process of the test evaluation method according to the third embodiment of the present invention. As Figure 3 shown, the test evaluation method includes:

[0068] First, the test scenarios can be identified and input, and then the test data can be collected to generate a scenario library. Preferably, the scenarios are preset and the test scenarios are executed. The data requests during the program execution are collected by a dynamic proxy class (for example, the abbreviation of AOP: Aspect Oriented Programming, which is a technology for unified maintenance of program functions through pre-compilation and dynamic proxy during runtime). The business fields in the data requests are obtained, and the unique business identifier of the data request is obtained by processing the business fields through a hash algorithm. Then, the mapping relationship between the business identifier and the preset scenario is saved to the scenario library. From another perspective, the present invention can transform the scenarios into the expression form of programming language, which can also be called a scenario converter. Of course, the scenario library can be automatically updated during the iterative process of the subsequent test program execution.

[0069] It should be noted that the preset scenarios can include one or more business identifiers.

[0070] Preferably, the information of the execution class is collected by a dynamic proxy class. For each WEB request, the business fields are set, and a unique business identifier BUID is generated through hashing. For multiple business fields, multiple BUIDs are generated. Among them, if the business field is empty, a UUID is generated. Then, the BUID or UUID is passed through the ThreadLocal thread-shared variable. In the case of cross-layer calls and RPC calls, the BUID or UUID is set as a hidden parameter for passing. In addition, the methods on the request call chain use the same business identifier, that is, <BUID or UUID, {method set}>. Several different web requests are defined as a scenario, and a single web request can also be used as a scenario. Finally, the key-value pair of the scenario and the business identifier is generated, <scenario key, {BUID, UUID...}>. Also, a request count threshold can be preset for each scenario, that is, each scenario includes at most N requests (i.e., the maximum number of requests).

[0071] Further, data requests during program execution can be collected by a data collector and imported into the scenario library.

[0072] After the scenario library is prepared, during the testing process or online monitoring process of the program, the dynamic proxy class collects the input parameters, output parameters, and code instrumentation information of method execution, and can obtain the <BUID or UUID, {method set}> of each data request, as well as the set of business identifiers {BUID, UUID...} of the test. If there is a single business identifier corresponding to the scenario information in the scenario library, it is matched with the set of business identifiers according to the single business identifier to obtain the corresponding scenario information, and then the preset label is marked on the scenario information. Or if there are multiple business identifiers corresponding to the scenario information in the scenario library, they are respectively matched with the set of business identifiers. If all multiple business identifiers are successfully matched, the corresponding scenario information is obtained to mark the preset label on the scenario information.

[0073] Statistically sum the scenario information marked with the preset label to obtain the number of covered scenarios, and then calculate the ratio of the number of covered scenarios to the total number of scenarios in the scenario library to obtain the scenario coverage rate of the test. At the same time, according to the scenario library, obtain the unmatched scenario information, generate an uncovered scenario set, and send a test insufficiency warning message.

[0074] It should be noted that if there is a business identifier in the set of business identifiers that fails to match successfully in the scenario library, the code static analysis method is called to generate a directed graph of the call relationship of the data request corresponding to the business identifier, and the label of each node in the directed graph of the call relationship is mapped to a positive integer in a preset interval segment through hash encoding, and then the mapped directed graph of the call relationship is converted into an adjacency matrix. Then, according to the adjacency matrix, it is matched with the adjacency matrix corresponding to the business identifier in the scenario library to determine whether the business identifier that matches successfully in the scenario library is obtained. If so, the covered scenario of the match is determined; if not, the unmatched business identifier and the adjacency matrix corresponding to the business identifier are stored in the scenario library, and the scenario key is configured, and then the scenario library is updated.

[0075] Figure 4 is a schematic diagram of the main modules of the test evaluation device according to an embodiment of the present invention, as Figure 4 shown, the test evaluation device 400 includes an acquisition module 401 and a processing module 402. Among them, the acquisition module 401 monitors the trigger of the test, calls the dynamic proxy class, and collects data during the test; the processing module 402 performs scenario matching on the data based on a preset scenario library and a preset matching model to obtain the covered scenarios; calls a preset coverage model to calculate the scenario coverage rate of the test.

[0076] In some embodiments, the acquisition module 401 collects data during the test process, including:

[0077] According to the data request in the test process, the business field in the data request is obtained to generate a corresponding business identifier, and then the business identifier set of the test is obtained; wherein all methods on each data request call chain correspond to the same business identifier;

[0078] The processing module 402 performs scene matching on the data based on a preset scene library and a preset matching model to obtain covered scenes, including:

[0079] According to the service identifier set, the covered scenarios are matched based on a preset scenario library; wherein the scenario library includes a mapping relationship between the service identifier and the scenario information.

[0080] In some embodiments, the acquisition module 401 acquires the business field in the data request to generate a corresponding business identifier, including:

[0081] Obtain the business field in the data request, determine whether the business field is empty, if so, generate a universal unique identification code as a business identifier according to the data request, otherwise generate a corresponding business identifier according to the business field through a preset algorithm model.

[0082] In some embodiments, the processing module 402 obtains covered scenarios based on the business identification set and matching based on a preset scenario library, including:

[0083] If there is a single service identifier corresponding to the scene information in the scene library, matching the single service identifier with the set of service identifiers is performed to obtain corresponding scene information, and then marking the scene information with a preset label; or

[0084] If there are multiple business identifiers corresponding to the scene information in the scene library, the multiple business identifiers are matched with the business identifier set respectively. If all the multiple business identifiers are matched successfully, the corresponding scene information is obtained to mark the scene information with a preset label.

[0085] In some embodiments, the processing module 402 calls a preset coverage model to calculate the scenario coverage of the test, including:

[0086] The scene information marked with preset tags is counted and summed to obtain the number of covered scenes, and then the ratio of the number of covered scenes to the total number of scenes in the scene library is calculated to obtain the scene coverage rate of the test.

[0087] In some embodiments, the processing module 402 obtains covered scenarios based on the business identification set and matching based on a preset scenario library, and further includes:

[0088] If there is a business identifier in the business identifier set that has not been successfully matched in the scenario library, the code static analysis method is called to generate a call relationship directed graph of the data request corresponding to the business identifier, so as to map the label of each node in the call relationship directed graph to a positive integer of a preset interval through hash coding, and then convert the mapped call relationship directed graph into an adjacency matrix;

[0089] According to the adjacency matrix, a match is performed with the adjacency matrix corresponding to the service identifier in the scenario library to obtain the successfully matched service identifier in the scenario library, and then determine the matched coverage scenario.

[0090] In some embodiments, after the processing module 402 calculates the scenario coverage of the test, it includes:

[0091] According to the scenario library, unmatched scenario information is obtained, an uncovered scenario set is generated, and an insufficient test warning message is sent.

[0092] It should be noted that the test evaluation method and the test evaluation device of the present invention have a corresponding relationship in terms of specific implementation contents, so the repeated contents will not be described again.

[0093] Figure 5 An exemplary system architecture 500 is shown to which the test evaluation method or test evaluation device according to the embodiment of the present invention can be applied.

[0094] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0095] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0096] The terminal devices 501 , 502 , and 503 may be various electronic devices having a test evaluation screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0097] It should be noted that the test evaluation method provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the computing device is generally disposed in the server 505 .

[0098] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0099] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device suitable for implementing an embodiment of the present invention. Figure 6 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0100] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0101] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal test evaluator (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0102] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0103] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, it may be described as follows: a processor includes an acquisition module and a processing module. The names of these modules do not, in some cases, limit the modules themselves.

[0106] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes monitoring the trigger test, calling the dynamic proxy class, and collecting data during the test; performing scene matching on the data based on a preset matching model according to a preset scene library to obtain a covered scene; calling a preset coverage model to calculate the scene coverage of the test.

[0107] The technical solution according to the embodiment of the present invention can solve the problems of high cost, low efficiency, opaque data and lack of persuasiveness in existing test scenario coverage evaluation.

[0108] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A test evaluation method, characterized in that: include: When a test is triggered, the dynamic proxy class is called to collect data during the test and obtain the service identifier set of the test; According to a preset scenario library, the data is scene-matched based on a preset matching model to obtain covered scenarios, including: according to a business identifier set, the covered scenarios are matched based on a preset scenario library; if there is a business identifier in the business identifier set that has not been successfully matched in the scenario library, a code static analysis method is called to generate a call relationship directed graph of the data request corresponding to the business identifier, so as to map the label of each node in the call relationship directed graph to a positive integer of a preset interval through hash coding, and then convert the mapped call relationship directed graph into an adjacency matrix; according to the adjacency matrix, matching is performed with the adjacency matrix corresponding to the business identifier in the scenario library to obtain the successfully matched business identifier in the scenario library, and then determining the matched coverage scenarios; The preset coverage model is called to calculate the scenario coverage of the test.

2. The method according to claim 1, characterized in that Collect data during the test, including: According to the data request in the test process, the business field in the data request is obtained to generate a corresponding business identifier, and then the business identifier set of the test is obtained; wherein all methods on each data request call chain correspond to the same business identifier; According to the preset scene library, scene matching is performed on the data based on a preset matching model to obtain covered scenes, including: The scenario library includes a mapping relationship between a service identifier and scenario information.

3. The method according to claim 2, characterized in that Obtaining the business field in the data request to generate a corresponding business identifier includes: Obtain the business field in the data request, determine whether the business field is empty, if so, generate a universal unique identification code as a business identifier according to the data request, otherwise generate a corresponding business identifier according to the business field through a preset algorithm model.

4. The method according to claim 2, characterized in that: According to the business identification set, the covered scenarios are matched based on the preset scenario library, including: If there is a single service identifier corresponding to the scene information in the scene library, matching the single service identifier with the set of service identifiers is performed to obtain corresponding scene information, and then marking the scene information with a preset label; or If there are multiple business identifiers corresponding to the scene information in the scene library, the multiple business identifiers are matched with the business identifier set respectively. If all the multiple business identifiers are matched successfully, the corresponding scene information is obtained to mark the scene information with a preset label.

5. The method according to claim 4, characterized in that Calling a preset coverage model to calculate the scenario coverage of the test includes: The scene information marked with preset tags is counted and summed to obtain the number of covered scenes, and then the ratio of the number of covered scenes to the total number of scenes in the scene library is calculated to obtain the scene coverage rate of the test.

6. The method according to any one of claims 1 to 5, characterized in that: After calculating the scenario coverage of the test, it includes: According to the scenario library, unmatched scenario information is obtained, an uncovered scenario set is generated, and an insufficient test warning message is sent.

7. A test and evaluation device, characterized in that: include: The acquisition module is used to monitor the touch test, call the dynamic proxy class, collect data during the test, and obtain the service identification set of the test; A processing module is used to perform scene matching on the data based on a preset scene library and a preset matching model to obtain covered scenes, including: matching the covered scenes based on a preset scene library according to a business identification set; if there is a business identification in the business identification set that has not been successfully matched in the scene library, calling a code static analysis method to generate a call relationship directed graph of the data request corresponding to the business identification, so as to map the label of each node in the call relationship directed graph to a positive integer of a preset interval through hash coding, and then converting the mapped call relationship directed graph into an adjacency matrix; matching the adjacency matrix with the adjacency matrix corresponding to the business identification in the scene library to obtain the successfully matched business identification in the scene library, and then determining the matched coverage scenes; calling a preset coverage model to calculate the scene coverage of the test.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images