Test methods and devices

Abstract

The embodiments of this specification provide a testing method and device, wherein the testing method includes: pre-marking a unique identifier for a test case to be tested, so that the system log generated by the system called by the test case to be tested contains the correspondence between the unique identifier and the system; obtaining the logs of each system on the system call link, and generating a test scenario according to the correspondence contained in the log; in response to receiving a test case request for one or more systems in the test scenario, executing a test on the test case corresponding to the one or more systems. Through the above method, in response to receiving a test case request for one or more systems in the test scenario, executing a test on the test case corresponding to the one or more systems, the effect of automatic linkage of the downstream with the upstream test can be achieved, effectively improving the test efficiency and test coverage, and avoiding the problems of inefficiency and missing risk points caused by manually combing the scenarios.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a testing method. Background Art

[0002] With the development of Internet technology, various projects based on the Internet have also exploded, and accordingly, the systems that handle tasks have become more and more complex. Generally speaking, when the mid- and downstream systems of the Internet platform are upgraded, it usually affects the upstream, so the upstream needs to be tested.

[0003] Currently, the mid- and downstream test managers manually sort out the upstream scenarios they expect to cover based on their experience, seek support from the upstream quality manager, and the upstream system manager manually triggers relevant use cases for testing. However, this testing method is not only inefficient, but may also fail to cover risk points due to mistakes made by the manager. Summary of the invention

[0004] In view of this, an embodiment of the present specification provides a testing method. One or more embodiments of the present specification also relate to a testing device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a testing method is provided, including:

[0006] Pre-marking a unique identifier for the test case to be tested, so that the system log generated by the system called by the test case to be tested contains the corresponding relationship between the unique identifier and the system;

[0007] Obtaining logs of each system on the system call link, and generating a test scenario according to the corresponding relationship contained in the log, wherein the test scenario is used to describe the corresponding relationship between the test case to be tested and the system;

[0008] In response to receiving a use case test request for one or more systems in the test scenario, a test is performed on the use cases to be tested corresponding to the one or more systems.

[0009] According to a second aspect of an embodiment of this specification, a testing device is provided, including:

[0010] A marking module is configured to mark a unique identifier for the test case in advance, so that the system log generated by the system called by the test case contains the corresponding relationship between the unique identifier and the system;

[0011] A generating module is configured to obtain logs of each system on the system call link, and generate a test scenario according to the corresponding relationship contained in the log, wherein the test scenario is used to describe the corresponding relationship between the test case to be tested and the system;

[0012] The execution module is configured to execute a test on the test cases to be tested corresponding to the one or more systems in response to receiving a test case request for the one or more systems in the test scenario.

[0013] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0014] Memory and processor;

[0015] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned testing method are implemented.

[0016] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned test method are implemented.

[0017] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned testing method.

[0018] The embodiments of this specification pre-mark a unique identifier for the use case to be tested, so that the system log generated by the system called by the use case to be tested contains the correspondence between the unique identifier and the system. Since the logs of each system on the system call link are obtained and a test scenario is generated according to the correspondence contained in the log, and the test scenario is used to describe the correspondence between the use case to be tested and the system, the use case can be perceived according to the system through the test scenario. Therefore, in response to receiving a use case test request for one or more systems in the test scenario, the test can be executed on the use cases to be tested corresponding to one or more systems, so as to achieve the effect of automatic linkage between downstream and upstream testing, effectively improve test efficiency and test coverage, and avoid the inefficiency and missing risk points caused by manual combing of scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a platform architecture of an Internet platform is shown;

[0020] Figure 2 A schematic diagram of another test processing process of an Internet platform is shown;

[0021] Figure 3 A schematic diagram of the analysis of the full link / check status is shown;

[0022] Figure 4 A flow chart of a testing method provided according to an embodiment of this specification is shown;

[0023] Figure 5 A schematic diagram of another testing method provided according to an embodiment of this specification is shown;

[0024] Figure 6 A schematic diagram of a verification process in a testing method provided according to an embodiment of this specification is shown;

[0025] Figure 7 A diagram showing a display interface of each system set provided according to an embodiment of this specification is shown;

[0026] Figure 8 A use case set display interface diagram of a system provided according to an embodiment of this specification is shown;

[0027] Fig. 9 An interface diagram for executing a use case test provided according to an embodiment of the present specification is shown;

[0028] Fig.10 A schematic diagram of a test architecture provided according to an embodiment of this specification is shown;

[0029] Fig.11 A schematic diagram of a field check effect provided according to an embodiment of this specification is shown;

[0030] Fig.12 A schematic diagram of the structure of a testing device provided by an embodiment of this specification is shown;

[0031] Fig.13 A structural block diagram of a computing device provided according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0032] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0033] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] See also Figure 1 , Figure 1 A schematic diagram of a platform architecture of an Internet platform is shown.

[0036] With the rapid development of the Internet, the complexity of the entire platform project system of many Internet service providers is also increasing. For example, the mid- and downstream systems of some platforms such as Figure 1 As shown in the figure, it supports many upstream projects, and may even connect to tens of thousands of project scenarios, carrying massive amounts of data. However, due to the high reuse of platform technical capabilities, the enterprise-level capability reuse platform supports many upstream projects, and the impact of capability changes is wide, which leads to low efficiency of quality project regression in the testing phase. In addition, the platform has high inter-domain complexity, which is mainly reflected in the increase in the scale of microservices. Microservices transfer the complexity within the system to the complexity between systems, which increases the difficulty of inter-domain contract testing. Contract testing is a testing activity conducted to discover contract breaking (Contract Breaking). For example, from the perspective of consumer project implementation, the consumer side defines the required data format and interaction details to generate a contract document. Then the producer implements its own logic based on the contract document and continuously verifies whether the implementation result is correct in the continuous integration environment. Another aspect is that the platform capability evolves quickly: the standardized precipitation of capabilities of multiple similar projects is completed, and the reusable capabilities are continuously upgraded, which makes it difficult to quickly deliver quality in the testing phase. It can be seen that the current testing status of the entire platform of Internet service providers has caused a certain impact on the stability of the platform.

[0037] In this context, mid- and downstream systems need to upgrade their capabilities quickly while also completing architecture upgrades. Therefore, how to quickly test and ensure that a large number of projects meet quality standards has become a challenge.

[0038] See also Figure 2 , Figure 2 A schematic diagram of the test processing process of another Internet platform is shown.

[0039] like Figure 2 As shown in the figure, when the underlying system is upgraded, the person in charge of mid- and downstream testing manually sorts out the upstream scenarios that are expected to be covered based on experience, and seeks support from the person in charge of upstream project system quality. Then the person in charge of the upstream system manually triggers the relevant use cases, and the person in charge of the underlying system manually confirms whether the use cases cover the expected logic. Figure 2 In the test process shown, the following problems exist:

[0040] First, the cost of upstream and downstream communication is high: Upstream and downstream projects are complex, and it is difficult for systems to perceive each other's project details.

[0041] Second, the maintenance cost of combing through use cases is high: the cost of manually combing through use cases is high, but it cannot be automatically perceived.

[0042] Third, the rhythm is highly dependent on projects: it relies on upstream scheduling and regression verification, resulting in low R&D efficiency.

[0043] Based on the above analysis, the current Internet platform cannot effectively achieve the effect of rapid delivery during testing. Figure 3 , Figure 3 The analysis diagram of the current status of the full link / verification is shown. It can be seen from the analysis diagram that the full link between the systems is currently fragmented, that is, each system maintains its own link and verification, and cannot perceive each other, which brings certain difficulties to the test.

[0044] Therefore, based on the above problems, a testing method is provided in this specification. This specification also involves a testing device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0045] See also Figure 4 , Figure 4 A flow chart of a testing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0046] Step 402: Mark the test case with a unique identifier in advance, so that the system log generated by the system called by the test case contains the corresponding relationship between the unique identifier and the system.

[0047] The use case refers to a scenario for testing a software program, so that the software program can run normally and achieve the execution results designed by the program in this scenario. The system refers to the software program that supports the upstream project in the Internet platform.

[0048] In this embodiment, before testing the use case, it is necessary to mark the use case to be tested with a unique identifier in advance, so that the system log generated by the system called by the use case to be tested contains the corresponding relationship between the unique identifier and the system, wherein there may be multiple use cases to be tested, and one use case to be tested corresponds to one unique identifier. The system called by the use case to be tested may include: the system directly called by the use case to be tested, and the system indirectly called in the process of calling the systems one by one from top to bottom.

[0049] Specifically, the unique identifier can be composed of numbers, letters, special characters, etc. For example, the unique identifier corresponding to the test case A is "1001", the unique identifier corresponding to the test case B is "1002", the unique identifier corresponding to the test case C is "1003", and so on.

[0050] For example, when the full link is scheduled for execution, the full link scheduling system can automatically mark the full link use case with a unique identifier. When the use case is called from top to bottom, the unique identifier will be automatically passed from the upstream system to the downstream system and the current system call information and use case information will be printed in a standard format in the public log. Thus, the log of the system will contain the corresponding relationship between the unique identifier of the use case to be tested and the system.

[0051] For example, if the system that needs to execute the test case is "system X", when the test case A calls system X, the corresponding relationship between the unique identifier "1001" of the test case A and system X will be recorded in the log of the system X. That is, it can be understood that if the log of system X contains the log record of the call of the unique identifier "1001" of the test case A, it means that the test case A calls system X, and the two have the corresponding relationship.

[0052] Step 404: Obtain the logs of each system on the system call link, and generate a test scenario according to the corresponding relationship contained in the log, wherein the test scenario is used to describe the corresponding relationship between the test case to be tested and the system.

[0053] The system call link refers to a link for calling one or more systems in an Internet platform. For example, the system call link can be any call link branch, multiple call link branches, or the entire call link of the platform, i.e., the full link.

[0054] For example, in one or more embodiments of this specification, the logs of each system on the entire link are obtained to achieve full-link regression. Full-link regression means that the test scenario not only includes the domain where the tested case is located, but also multiple other upstream and downstream domains. Of course, testing can also be performed on a single domain or multiple domains, and the method provided in the embodiments of this specification does not limit this. In an embodiment of full-link regression, the logs of all systems on the entire link can be obtained before testing.

[0055] In this embodiment, after a unique identifier is pre-marked for the test case so that the system log generated by the system called by the test case contains the correspondence between the unique identifier and the system, the logs of each system on the system call link can be obtained, and then a test scenario can be generated based on the correspondence between the test case and the system contained in the log.

[0056] For example, the logs of system Y and system Z on the system call link are obtained. The log of system Y contains the unique identifier "1002" of the test case B; the log of system Z contains the unique identifier "1003" of the test case C.

[0057] According to the above example, a test scenario is generated to describe the corresponding relationship between the test case and the system. It can be understood that the generated test scenario includes: system Y corresponds to test case B; system Z corresponds to test case C.

[0058] Specifically, in a possible implementation of the embodiments of this specification, the logs of each system on the system call link are obtained, and a test scenario is generated according to the corresponding relationship contained in the log, including:

[0059] Regularly update the logs of each system on the system call link to the database;

[0060] The correspondence between the test cases and the system is accumulated in the test scenario by using the correspondence contained in the logs updated regularly in the database.

[0061] In this embodiment, for the sake of system stability, it is necessary to synchronously update the logs of each system on the system call link to the database. Then, the corresponding relationship between the test case and the system is accumulated in the test scenario by using the corresponding relationship contained in the logs updated regularly in the database. The database may be an offline database.

[0062] Therefore, first of all, it is necessary to regularly collect the logs of each system on the system call link, and then synchronously update the collected logs of each system on the call link to the database for storage.

[0063] Here, log service technology can be used to collect logs of various systems on the system call link. Log service technology is a service for collecting, storing, querying and analyzing logs.

[0064] After the collected logs of each system on the call link are periodically updated and stored in the database, the correspondence contained in the logs regularly updated in the database can be used to accumulate the correspondence between the test cases and the systems in the test scenario.

[0065] For example, with a 24-hour cycle, the logs of each system on the call link are collected every 24 hours, and the logs of each system are synchronously updated to the database for storage. Then, the correspondence contained in the logs updated regularly in the database can be used to accumulate the correspondence between the test cases and the systems in the test scenarios.

[0066] In the above implementation, the logs of each system on the system call link are periodically updated to the database, and then the corresponding relationships contained in the logs regularly updated in the database are used to accumulate the corresponding relationships between the test cases and the systems in the test scenario. Through the above method, the corresponding relationships between the test cases and the systems can be updated to ensure the accuracy of the corresponding relationships between the test cases and the systems in the logs of each system.

[0067] The testing method provided in this specification also includes: deduplication processing of log records in the database.

[0068] In this embodiment, when the logs of each system on the system call link are periodically updated to the database, duplicate log records may occur. Therefore, it is necessary to perform deduplication processing on the log records in the database, that is, to filter and delete the duplicate log records in the logs of each system.

[0069] Through the above method, the corresponding relationship between the test case and the system in the test scenario can be guaranteed to be updated in time, achieving the effect of regular automatic preservation.

[0070] Step 406: In response to receiving a use case test request for one or more systems in the test scenario, execute a test on the use cases to be tested corresponding to the one or more systems.

[0071] In this embodiment, by obtaining the logs of each system on the system call link and generating a test scenario based on the corresponding relationship contained in the log, when a use case test request for one or more systems in the test scenario is received, a test can be performed on the use cases to be tested corresponding to the one or more systems in response to the use case test request.

[0072] In a possible implementation of an embodiment of the present specification, in response to receiving a use case test request for one or more systems in a test scenario, executing a test on the use cases to be tested corresponding to the one or more systems may be specifically implemented in the following manner:

[0073] According to the test scenario, the test cases corresponding to the system are displayed on the interface so that users can select the test cases according to the system.

[0074] In response to receiving a test case request sent by a user, a test is performed on the test case selected by the user.

[0075] In this embodiment, an interface for displaying “the use cases to be tested corresponding to the system” is provided, and the user can select the use cases to be tested corresponding to the system for execution according to the interface.

[0076] Specifically, after obtaining the logs of each system on the system call link and generating a test scenario according to the corresponding relationship contained in the log, the test case corresponding to the system can be displayed on the interface based on the test scenario. Then the user can select the test case according to the system.

[0077] It should be noted that the user can select one or more test cases corresponding to the system, and after the user selects one or more test cases corresponding to the system, a test case request can be generated. Then, in response to receiving the test case request issued by the user, the test is performed on the test case selected by the user.

[0078] In the above implementation, according to the test scenario, the test case corresponding to the system is displayed on the interface so that the user can select the test case according to the system. Then, in response to receiving the test case test request sent by the user, the test is performed on the test case selected by the user. Through the above method, the user can select the test case according to the system and test the test case, thereby ensuring the effectiveness of the execution of the test case.

[0079] In a possible implementation of the embodiments of this specification, according to the test scenario, the test case corresponding to the system is displayed on the interface, which can be specifically implemented in the following manner:

[0080] According to the test scenario, the system information of each system is displayed on the interface, wherein one system corresponds to one system information display area.

[0081] In response to the system display area corresponding to any system being selected and opened, one or more test cases corresponding to any system and a selection control corresponding to each test case are displayed according to the test scenario, so that the user can select the test case using the selection control.

[0082] In this embodiment, the test scenario is used to describe the correspondence between the test case and the system. Therefore, based on the test scenario, the system information of each system can be displayed on the display interface, wherein the system information includes specific information of the system, for example, the system information can include the system name corresponding to the system, the number of test cases to be tested, etc. And on the interface, one system corresponds to one system information display area, and the system information display area refers to the area in the interface that displays the system information.

[0083] When the system display area corresponding to any system is selected and opened, one or more test cases corresponding to any system and a selection control corresponding to each test case will be displayed, so that the user can select the test case using the selection control.

[0084] For example, according to the test scenario, the system information display area of ​​system A and system B is displayed on the interface. Specifically, the system information corresponding to system A is "the number of test cases to be tested is 100"; the system information corresponding to system B is "the number of test cases to be tested is 150".

[0085] Then, in response to the system display areas corresponding to system A and system B being selected and opened, the test cases corresponding to system A and system B are displayed according to the test scenario, as well as the selection controls corresponding to each test case, so that the user can select the test case using the selection controls.

[0086] In the above implementation, according to the test scenario, the system information of each system is displayed on the interface, and then in response to the system display area corresponding to any system being selected and opened, one or more test cases corresponding to any system and the selection control corresponding to each test case are displayed according to the test scenario, so that the user can select the test case using the selection space. Through the above method, the user can select the test case according to the system and test the test case, thereby ensuring the effectiveness of the execution of the test case.

[0087] In another possible implementation of this embodiment, according to the test scenario, the test case corresponding to the system is displayed on the interface, which can be specifically implemented in the following manner:

[0088] In response to receiving a use case query request for a specified system, the test cases corresponding to the specified system and the selection controls corresponding to each test case are displayed on the interface according to the test scenario, so that the user can select the test case using the selection controls.

[0089] In this embodiment, a query box for querying use cases of a specified system may also be provided in the interface, and the user may enter any system in the query box for query, and may obtain the use case to be tested corresponding to the system.

[0090] For example, after the user enters "System A" in the query box, a use case query request for System A will be generated based on "System A". Then, in response to receiving the use case query request for the specified system, the use cases to be tested corresponding to System A and the selection controls corresponding to each use case to be tested will be displayed on the interface according to the test scenario, so that the user can use the selection space to select the use case to be tested.

[0091] In the above implementation, in response to receiving a use case query request for a specified system, the test case corresponding to the specified system and the selection control corresponding to each test case are displayed on the interface according to the test scenario, so that the user can select the test case using the selection control. Through the above method, the user can select the test case according to the system and test the test case, thereby ensuring the effectiveness of the execution of the test case.

[0092] The test method provided in the embodiment of this specification also includes:

[0093] A verification script for each system on the system call link is collected from the verification platform, wherein the verification script is a script for verifying the consistency of associated data between systems.

[0094] Use verification scripts to identify relationships between related data.

[0095] The relationship between the identified related data is checked for validity.

[0096] In this embodiment, firstly, a verification script for each system on the system call link is collected from the verification platform, wherein the verification script is a script for verifying the consistency of associated data between systems.

[0097] Then the relationship between the associated data is identified, and then the validity of the relationship between the identified associated data is checked.

[0098] For example, after collecting the verification scripts for each system on the system call link from the verification platform, the verification scripts can be used to identify the relationship between the related data, obtain the related data between the fields in the system logs on the system call link, and then verify the validity of the relationship between the fields in the system logs to determine whether the fields between the systems are consistent. Fig.11 As shown in the schematic diagram of the field verification effect, through the above embodiment, the fields between various systems such as the "middle platform F" and the "bottom layer G", and the fields between the "middle platform F" and the "platform Q" can be checked, so as to check the missing fields, check the invalid fields, and so on.

[0099] It is understandable that, in the verification process, according to the above embodiment, the data between all cross-end systems can be verified, also known as reconciliation, and the inconsistency of data in the flow process can be checked and discovered. For example, the information flow can be verified and the capital flow can be verified. The information flow verification is mainly the verification between project data, and the capital flow is the verification of capital transaction data. For example, after the verification, the coverage risk points determined by the verification validity check can also be displayed on the interface, so that the upstream and downstream systems can perceive the coverage risk fields from each other, so as to establish a linked data and capital security line.

[0100] In the above implementation, by collecting the verification scripts for each system on the system call link from the verification platform, and then using the scripts to identify the relationship between the associated data, and verifying the validity of the relationship between the identified associated data, the above method can make the systems on the system call link perceive each other's risks.

[0101] In a possible implementation of the embodiments of this specification, after executing the test case to be tested corresponding to one or more systems, the following steps are also included:

[0102] Display the test results of the test case.

[0103] In this embodiment, after executing the test case corresponding to one or more systems, the test results need to be displayed so that the user can timely understand whether one or more systems have risks according to the test results of the test case.

[0104] The embodiments of this specification pre-mark a unique identifier for the test case, so that the system log generated by the system called by the test case contains the correspondence between the unique identifier and the system; then obtain the logs of each system on the system call link, and generate a test scenario based on the correspondence contained in the log, wherein the test scenario is used to describe the correspondence between the test case and the system; then, in response to receiving a test case test request for one or more systems in the test scenario, perform a test on the test case corresponding to one or more systems. Through the above method, pre-marking a unique identifier for the test case can make the system log generated by the system called by the test case contain the correspondence between the unique identifier and the system. Based on the correspondence, a test scenario for describing the correspondence between the test case and the system can be further generated. Based on the test scenario, each system can perceive the test case of each system on the entire link. Further, by receiving a test case test request for one or more systems in the test scenario, the risks of each system can be effectively identified.

[0105] Below, Figure 5Taking the example of the present invention, an embodiment of the test method provided in the embodiments of the present specification is schematically described in combination with the implementation method of the contract test. Figure 5 A schematic diagram of another testing method provided according to an embodiment of the present specification is shown.

[0106] like Figure 5 As shown, the test method of this embodiment can be divided into a data production end and a data consumption end. On the data production end, first, when the full-link use case project is written, the test end implemented according to the method provided in the embodiment of this specification can automatically determine a unique identifier for each use case; when the full-link use case is executed, the full-link scheduling system can automatically mark the use case on the full link with a unique identifier, and then when the use case is called from top to bottom one by one, the unique identifier can be automatically passed from the upstream system to the downstream system and recorded in the public log to indicate the system call information and the corresponding use case unique identifier. Specifically, as Figure 5 As shown in the figure, the data production end can include four stages: "identification definition", "use case labeling", "full link execution" and "full link use case data production", as follows:

[0107] Identification definition phase: In this phase, unique identifiers are pre-labeled for the test cases. Figure 5 As shown, a unique identifier is defined first, for example, the defined unique identifier may be "id=1", "id=2", "id=3" and so on.

[0108] Use case labeling phase: In this phase, unique identifiers are given to the use cases to be tested. Figure 5 As shown, a use case is marked as "id=1", and the systems called by the use case include: system A, system B, and system C.

[0109] Full-link execution phase: In this phase, the execution of the use case "id=1" calls system A, system B, and system C accordingly. During the calling process, "id=1" is passed from top to bottom to system A, system B, and system C.

[0110] Full-link use case data production phase: In this phase, the log system generates logs for each system. The log generated by system A contains the corresponding relationship between system A and the use case "id=1", the log generated by system B contains the corresponding relationship between system B and the use case "id=1", and the log generated by system C contains the corresponding relationship between system C and the use case "id=1".

[0111] Through the above four stages, after each system on the system call link executes the test case, the system call link produces the corresponding full-link use case data, that is, the log of each system. The log of each system contains the correspondence between the unique identifier of the use case and the system.

[0112] On the data consumption side, for performance and system stability considerations, public logs can be synchronized with offline databases, bypassing the public log screening and deduplication, to obtain full-link use cases and call system related information. For example, the offline database can be run once a day, merging historical data with T-1 day incremental data, automatically generating use case sets and use case lists, and automatically preserving them daily. Specifically, Figure 5 As shown in the figure, the data consumption end can include four stages: "synchronization offline", "filtering and deduplication", "use case use" and "expansion empowerment", as follows:

[0113] Synchronous offline phase: In this phase, the logs of each system on the system call link are synchronized to the database. For example, the system RPC call log, then the log is collected, and real-time calculation is performed, and finally stored in the database of the data processing platform.

[0114] Screening and deduplication stage: Screening and deduplication refers to deduplication of log records in the database. Specifically, duplicate information in log records can be deduplicated to reduce the amount of data processing. The information in the log records includes but is not limited to the unique identifier of the use case, the application, i.e., the system, interface, service, measurement, scenario, and other information. After screening and deduplication, a use case set formed by the correspondence between the system and the use case and specific use case information, such as the use case name, are obtained.

[0115] Use case usage phase: The use case usage phase includes providing full set display of use cases, daily insurance, coverage measurement, one-click test execution and other functions.

[0116] Extension and empowerment stage: This stage is the function expanded after executing the use case test, for example, it can include: full-scenario verification, scenario analysis, scenario supplement, etc.

[0117] Below, Figure 6 Taking as an example, the processing process of checking in the test method provided in the embodiment of this specification is schematically described. Figure 6 A schematic diagram of a verification process in a testing method provided according to an embodiment of the present specification is shown.

[0118] like Figure 6 As shown, the verification process can include five stages: "data collection", "relationship identification", "validity verification", "scenario insurance" and "use case use", as follows:

[0119] Data collection phase: In this phase, the verification scripts for each system on the system call link are collected from the verification platform.

[0120] Relationship identification phase: In this phase, the relationship between the associated data of each system is identified, for example, the relationship between systems, tables, and fields on the system call link is identified.

[0121] Validity verification stage: In this stage, the comprehensiveness of the verification of the relationship between the associated data and the effectiveness of the verification can be analyzed. Among them, the comprehensiveness analysis can include: scenario analysis, system analysis, relationship analysis, risk analysis and other aspects. Among them, the effectiveness analysis can include: scenario combing, self-test verification, rule reversal, reverse attack and other aspects.

[0122] Scenario preservation stage: In this stage, scenario preservation refers to updating the test cases to be tested. Specifically, for example, data can be collected, then the collected data can be synchronized to the database, and then relationship identification can be performed, and finally validity verification can be performed.

[0123] Use case usage phase: In this phase, tests can be performed on the test cases corresponding to one or more systems, thereby achieving functions such as full set display, daily preservation, coverage measurement, and one-click execution.

[0124] In the use case usage phase, see Figure 7 , Figure 7 FIG. 1 shows a system set display interface diagram provided according to an embodiment of this specification. Figure 7 As shown in the display interface, one system corresponds to one system information display area. Each system information display area displays the system name and the number of corresponding test cases to be tested. Users can click on any system information display area to enter the following Figure 8 As shown, the use case set display interface for the system.

[0125] See also Figure 8 , Figure 8 FIG. 1 shows a use case set display interface diagram of a system provided according to an embodiment of this specification. Figure 8 As shown, taking the system "fcassetfux" as an example, the use case set display interface diagram of the system "fcassetfux" displays all the use cases to be tested that have a corresponding relationship with the system "fcassetfux", and each use case to be tested has use case information such as the use case name and the corresponding unique identifier. Figure 8 As shown, a query input box may also be provided in the interface for querying test cases for a specified system.

[0126] When the user Figure 8 When one or more use cases are selected in the interface shown, you can click the "Execute" button to trigger the test of the selected use cases. Fig. 9 , Fig. 9FIG. 1 shows an interface diagram for executing a use case test according to an embodiment of the present specification. Fig. 9 As shown, after clicking "Execute", a small window may pop up to display the relevant execution information of the test case, such as "Test Case Name", "Run Status", "Run Link", etc. When the user clicks the Run Link, the user may further enter the page for viewing the detailed run information of the test case, which may include input parameters, return parameters, coverage, results, etc.

[0127] It can be seen from the above embodiments that according to the testing method provided in the embodiments of this specification, a large number of unknown use cases can be automatically perceived, and the mid- and downstream systems can jointly execute the testing of the upstream use cases. There is no need to manually collect use case information to trigger the test, and the unknown use cases of the persons in charge of the mid- and downstream systems can be automatically excavated, so as to effectively preserve the use cases in the test scenarios, make up for the problem that the original mid- and downstream systems cannot perceive the project verification risk fields, and cannot check whether the project covers the risk points of the mid- and downstream systems comprehensively and effectively, so as to achieve the purpose of verifying the validity of the verification between the systems.

[0128] Fig.10 FIG. 1 shows a schematic diagram of a test architecture provided according to an embodiment of this specification. In this embodiment, the above-mentioned multiple embodiments are combined. Fig.10 As shown, the test architecture can produce full-link logs, automatically collect, extract, and preserve the logs, and then use unique identifiers to build the correspondence between the system and the use case, and form a test scenario. Based on the test scenario, many functions such as full set display, full-link use case testing, and verification are performed, and the results of the effectiveness analysis are obtained based on the results of the use case testing and verification. More specifically, for example, for the full link, the logs of the mid- and downstream systems can be collected to identify the full-link use cases passing through the mid- and downstream, and the corresponding scenarios can be identified by tagging the log keywords. Whenever regression is required, the corresponding use cases are executed according to the scenarios. After the test is completed, the coverage is automatically analyzed, and effective coverage is determined based on the analysis results. For the verification process, the table storing the verification scripts in the database of the verification platform can be cleaned to obtain the verification associated with the upstream system and the downstream system. The project characteristics and key risk fields of these verification contents can be identified to verify the comprehensiveness of the verification, and the verification can be verified by constructing abnormal scenarios. Of course, Fig.10 The system architecture shown may also have other implementation details. For details not described in detail, please refer to the introduction of other embodiments of the testing method, which will not be repeated here.

[0129] Corresponding to the above method embodiment, this specification also provides a test device embodiment, Fig.12 FIG. 1 is a schematic diagram showing the structure of a testing device provided by an embodiment of the present specification. Fig.12 As shown, the device comprises:

[0130] The marking module 1202 is configured to mark the test case with a unique identifier in advance, so that the system log generated by the system called by the test case contains the corresponding relationship between the unique identifier and the system;

[0131] The generating module 1204 is configured to obtain the logs of each system on the system call link, and generate a test scenario according to the corresponding relationship contained in the log, wherein the test scenario is used to describe the corresponding relationship between the test case and the system;

[0132] The execution module 1206 is configured to execute a test on the test cases to be tested corresponding to the one or more systems in response to receiving a test case request for the one or more systems in the test scenario.

[0133] Optionally, the generating module 1204 is further configured to:

[0134] Regularly update the logs of each system on the system call link to the database;

[0135] The correspondence between the test cases and the system is accumulated in the test scenario by using the correspondence contained in the logs updated regularly in the database.

[0136] Optionally, the device further comprises:

[0137] The processing module is configured to perform deduplication processing on log records in the database.

[0138] Optionally, the device further comprises:

[0139] A collection module is configured to collect a verification script for each system on the system call link from the verification platform, wherein the verification script is a script for verifying the consistency of associated data between systems;

[0140] an identification module configured to identify the relationship between the associated data using the verification script;

[0141] The verification module is configured to verify the validity of the relationship between the identified related data.

[0142] Optionally, the execution module 1206 is further configured to:

[0143] According to the test scenario, the test cases corresponding to the system are displayed on the interface so that the user can select the test cases according to the system;

[0144] In response to receiving a test case request sent by a user, a test is performed on the test case selected by the user.

[0145] Optionally, the execution module 1206 is further configured to:

[0146] According to the test scenario, the system information of each system is displayed on the interface, where one system corresponds to one system information display area;

[0147] In response to the system display area corresponding to any system being selected and opened, one or more test cases corresponding to any system and a selection control corresponding to each test case are displayed according to the test scenario, so that the user can select the test case using the selection control.

[0148] Optionally, the execution module 1206 is further configured to:

[0149] In response to receiving a use case query request for a specified system, the test cases corresponding to the specified system and the selection controls corresponding to each test case are displayed on the interface according to the test scenario, so that the user can select the test case using the selection controls.

[0150] Optionally, the execution module 1206 is further configured to:

[0151] Display the test results of the test cases.

[0152] The test device provided in the embodiment of this specification pre-marks the use case to be tested with a unique identifier, so that the system log generated by the system called by the use case to be tested contains the correspondence between the unique identifier and the system; then obtains the logs of each system on the system call link, and generates a test scenario based on the correspondence contained in the log, wherein the test scenario is used to describe the correspondence between the use case to be tested and the system; then, in response to receiving a use case test request for one or more systems in the test scenario, performs a test on the use case to be tested corresponding to one or more systems. Through the above method, pre-marking the use case to be tested with a unique identifier can make the system log generated by the system called by the use case to be tested contain the correspondence between the unique identifier and the system. Based on the correspondence, a test scenario for describing the correspondence between the use case to be tested and the system can be further generated. Based on the test scenario, each system can perceive the use cases to be tested of each system on the entire link. Further, by receiving use case test requests for one or more systems in the test scenario, the risks of each system can be effectively identified.

[0153] The above is a schematic scheme of a test device of this embodiment. It should be noted that the technical scheme of the test device and the technical scheme of the above test method belong to the same concept, and the details of the technical scheme of the test device that are not described in detail can be found in the description of the technical scheme of the above test method.

[0154] Fig.13The block diagram of a computing device 1300 according to one embodiment of the present specification is shown. The components of the computing device 1300 include but are not limited to a memory 1310 and a processor 1320. The processor 1320 is connected to the memory 1310 via a bus 1330, and the database 1350 is used to store data.

[0155] The computing device 1300 also includes an access device 1340 that enables the computing device 1300 to communicate via one or more networks 1360. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1340 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0156] In one embodiment of the present specification, the above components of the computing device 1300 and Fig.13 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.13 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0157] Computing device 1300 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 1300 may also be a mobile or stationary server.

[0158] The processor 1320 is used to execute the following computer executable instructions, which implement the steps of the above-mentioned test method when executed by the processor.

[0159] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned testing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned testing method.

[0160] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned test method when executed by a processor.

[0161] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned test method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned test method.

[0162] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned testing method.

[0163] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above test method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be found in the description of the technical scheme of the above test method.

[0164] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0165] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0166] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0167] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0168] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A testing method comprising: Pre-mark the test case with a unique identifier so that the system log generated by the system called by the test case contains the corresponding relationship between the unique identifier and the system, wherein the system called by the test case includes the system directly called by the test case and the system indirectly called in the process of calling the system one by one from top to bottom, and the test case is a full-link test case; Obtain the logs of each system on the entire link of the system call link, and generate a test scenario according to the corresponding relationship contained in the log, wherein the test scenario is used to describe the corresponding relationship between the test case to be tested and the system, and the test scenario includes the domain where the test case to be tested is located and multiple other upstream and downstream domains; According to the test scenario, the test cases corresponding to the system are displayed on the interface so that the user can select the test cases according to the system; in response to receiving a test case request issued by the user, the test is performed on the test case selected by the user, wherein the test case request is generated based on the test case selected by the user according to the system.

2. According to the method of claim 1, obtaining the logs of each system on the entire link of the system call link, and generating a test scenario according to the corresponding relationship contained in the log, comprises: Regularly update the logs of each system on the entire link of the system call link to the database synchronously; The corresponding relationship between the test case and the system is accumulated in the test scenario by using the corresponding relationship contained in the log regularly updated in the database.

3. The method according to claim 2, further comprising: Deduplication processing is performed on the log records in the database.

4. The method according to claim 1, further comprising: Collecting verification scripts for each system on the entire link of the system call link from the verification platform, wherein the verification script is a script for verifying the consistency of associated data between systems; Using the verification script to identify the relationship between the associated data; The relationship between the identified related data is checked for validity.

5. The method according to claim 1, wherein displaying the test case corresponding to the system on the interface according to the test scenario comprises: According to the test scenario, the system information of each system is displayed on the interface, wherein one system corresponds to one system information display area; In response to the system information display area corresponding to any system being selected and opened, one or more test cases corresponding to any system and a selection control corresponding to each test case are displayed according to the test scenario, so that the user can select the test case using the selection control.

6. The method according to claim 1, wherein displaying the test case corresponding to the system on the interface according to the test scenario comprises: In response to receiving a use case query request for a specified system, the use cases to be tested corresponding to the specified system and the selection controls corresponding to each use case to be tested are displayed on the interface according to the test scenario, so that the user can select the use case to be tested using the selection controls.

7. The method according to claim 1, after executing the test on the test case selected by the user in response to receiving the test case request issued by the user, comprises: Display the test results of the test case.

8. A testing device comprising: A marking module is configured to mark a unique identifier for the test case in advance, so that the system log generated by the system called by the test case contains the corresponding relationship between the unique identifier and the system, wherein the system called by the test case includes the system directly called by the test case and the system indirectly called in the process of calling the system one by one from top to bottom, and the test case is a full-link test case; A generation module is configured to obtain logs of each system on the entire link of the system call link, and generate a test scenario according to the corresponding relationship contained in the log, wherein the test scenario is used to describe the corresponding relationship between the test case to be tested and the system, and the test scenario includes the domain where the test case to be tested is located and multiple other upstream and downstream domains; The execution module is configured to display the test cases corresponding to the system on the interface according to the test scenario so that the user can select the test cases according to the system; in response to receiving a test case request from the user, the execution module performs a test on the test case selected by the user, wherein the test case request is generated based on the test case selected by the user according to the system.

9. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the testing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the testing method according to any one of claims 1 to 7.

Images