Test case processing method, device, equipment and storage medium

Abstract

The present application provides a test case processing method, device, equipment and storage medium, which relates to the field of test case testing technology. The method includes: receiving a first test case removal notification sent by a first server; calling a test task to run the test cases in a second test case set, and obtaining a first test run result of the second test case set; if the test run of the removed test case in the first test run result fails, then determining that the removed test case is a first false positive test case. Compared with the prior art, the problem of low efficiency and high cost caused by manual troubleshooting of false positive test cases is avoided.

Description

Technical Field

[0001] The present application relates to the field of use case testing technology, and in particular to a test case processing method, apparatus, device and storage medium. Background Art

[0002] Today, with the rapid development of the Internet application product testing field, the competition is increasingly fierce. The product function iteration speed of the Internet is fast, and the frequency of R&D technology going online is very high. After each R&D technology reaches the upper limit, it is necessary to test the quality of the online project to check whether each use case can be executed normally, that is, whether each use case can be executed successfully.

[0003] In the prior art, the online project is generally tested manually, each use case is tested, and the use cases that fail to execute are determined by manual screening, and then the failed use cases are processed to ensure that each use case in the online project can be successfully executed.

[0004] However, this processing method is costly and inefficient. In actual applications, false alarms may occur due to the test environment, use case design and other factors. A large number of false alarms may affect the availability and online efficiency of the use case, further reducing efficiency. Summary of the invention

[0005] In view of this, the purpose of this application is to provide a test case processing method, device, equipment and storage medium, which can solve the problems of high cost and low efficiency of testing test cases in the prior art, thereby improving the testing efficiency of test cases and automatically determining the function of false positive test cases.

[0006] In a first aspect of the present application, the present application provides a test case processing method, the method comprising:

[0007] Receive a first use case removal notification sent by a first server, the first use case removal notification including: an identifier of a test case that has been removed in the first server, the test case that has been removed being a service running result of the test case set obtained by the first server calling a service task to run a test case in a first test case set to test a code of a project to be tested, and a test case determined from the first test case set that has failed in service running;

[0008] Calling the test task to run the test cases in the second test case set, and obtaining the first test running result of the second test case set;

[0009] If the test run of the removed test case in the first test run result fails, it is determined that the removed test case is a first false positive test case.

[0010] Optionally, the method further comprises:

[0011] Calling the test task to run the test cases in the second test case set, and obtaining a second test running result of the second test case set;

[0012] If there is a test case in the second test running result that fails to run the test, determining the test case that failed to run the test is a second false positive test case;

[0013] A second use case removal notification is sent to the first server, wherein the second use case removal notification includes: an identifier of the second false alarm test case, and the second use case removal notification is used to instruct the first server to perform a removal operation on the second false alarm test case.

[0014] Optionally, the method further comprises:

[0015] Sending a use case repair notification, wherein the use case repair notification is used to notify the user to repair the false positive test case;

[0016] Receive input of the repaired test case;

[0017] The false positive test case in the second test case set is replaced with the repaired test case.

[0018] Optionally, the method further comprises:

[0019] Calling the test task multiple times to run the test cases in the second test case set to obtain a third test running result of the second test case set;

[0020] If the repaired test case in the third test running result has been successfully tested for a first preset number of times in a row, a use case release notification is sent to the first server, and the use case release notification includes: the repaired test case, and the use case release notification is used to enable the first server to replace the false alarm test case in the first test case set with the repaired test case, and resume the running of the repaired test case.

[0021] Optionally, the method further comprises:

[0022] Calling the test task multiple times to run the test cases in the second test case set to obtain a third test running result of the second test case set;

[0023] If the stable test cases other than the false alarm test cases in the third test running result are tested and run successfully for a second preset number of times in a row, a removal operation is performed on the stable test cases in the second test case set.

[0024] Optionally, calling the test task to run the test cases in the second test case set includes:

[0025] According to a preset interval, the test task is called to run the test cases in the second test case set.

[0026] In a second aspect of the present application, the present application also provides a test case processing method, the method comprising:

[0027] Calling a service task to run the test cases in the first test case set to test the code of the project to be tested to obtain a service running result of the test case set;

[0028] According to the service operation result, removing the test cases in the first test case set whose service operation fails;

[0029] A first use case removal notification is sent to a second server, wherein the first use case removal notification includes: an identifier of a test case that has been removed from the first server, the first use case removal notification is used to enable the second server to call a test task to run the test cases in a second test case set, and obtain a first test running result of the second test case set; if the test running of the removed test case in the first test running result fails, then the removed test case is determined to be a first false alarm test case.

[0030] Optionally, the method further comprises:

[0031] Receive a second use case removal notification sent by the second server, the second use case removal notification including: an identifier of a second false positive test case; the second false positive test case is a test case in the second test case set that is run by the second server when calling the test task, obtain a second test run result of the second test case set, and determine the test case that failed the test run according to the second test run result;

[0032] Perform a removal operation on the second false positive test case.

[0033] Optionally, the method further comprises:

[0034] receiving a use case return notification sent by the second server, wherein the use case return notification includes: a repaired test case;

[0035] Replace the false positive test cases in the first test case set with the repaired test cases.

[0036] Optionally, the service running result of the test case set obtained by calling the service task to run the test cases in the first test case set to test the code of the project to be tested includes:

[0037] In response to the online request of the code of the project to be tested, the service task is called to run the test cases in the first test case set.

[0038] In a third aspect of the present application, the present application further provides a test case processing device, the device comprising: a receiving module, a calling module and a determining module, wherein:

[0039] The receiving module is used to receive a first use case removal notification sent by the first server, wherein the first use case removal notification includes: an identifier of a test case that has been removed in the first server, the test case that has been removed is a service running result of the test case set obtained by the first server calling a service task to run a test case in a first test case set to test the code of the project to be tested, and a test case determined from the first test case set that has failed in service running;

[0040] The calling module is used to call the test task to run the test cases in the second test case set, and obtain the first test running result of the second test case set;

[0041] The determination module is used to determine that the removed test case is a first false positive test case if the test running of the removed test case in the first test running result fails.

[0042] Optionally, the device further includes: a sending module, wherein:

[0043] The calling module is specifically used to call the test task to run the test cases in the second test case set to obtain a second test running result of the second test case set;

[0044] The determination module is specifically configured to determine that the test case that failed in the test running is a second false positive test case if there is a test case that failed in the test running result;

[0045] The sending module is used to send a second use case removal notification to the first server, wherein the second use case removal notification includes: an identifier of the second false alarm test case, and the second use case removal notification is used to instruct the first server to perform a removal operation on the second false alarm test case.

[0046] Optionally, the device further comprises: a replacement module, wherein:

[0047] The sending module is specifically used to send a use case repair notification, where the use case repair notification is used to notify the user to repair the false positive test case;

[0048] The receiving module is specifically used to receive the input repaired test case;

[0049] The replacement module is used to replace the false positive test case in the second test case set with the repaired test case.

[0050] Optionally, the calling module is specifically configured to call the test task multiple times to run the test cases in the second test case set, and obtain a third test running result of the second test case set;

[0051] The sending module is specifically used to send a test case release notification to the first server if the repaired test case in the third test running result has been successfully tested and run for a first preset number of times in a row. The test case release notification includes: the repaired test case. The test case release notification is used to enable the first server to replace the false alarm test case in the first test case set with the repaired test case, and resume the running of the repaired test case.

[0052] Optionally, the device further comprises: a removal module, wherein:

[0053] The calling module is specifically used to call the test task multiple times to run the test cases in the second test case set to obtain a third test running result of the second test case set;

[0054] The removal module is used to perform a removal operation on the stable test cases in the second test case set if the stable test cases other than the false alarm test cases in the third test running results are tested and run successfully for a second preset number of consecutive times.

[0055] Optionally, the calling module is specifically configured to call the test task to run the test cases in the second test case set according to a preset interval time.

[0056] In a fourth aspect of the present application, the present application further provides a test case processing device, the device comprising: a calling module, a removing module and a sending module, wherein:

[0057] The calling module is used to call a service task to run the test cases in the first test case set to test the code of the project to be tested and obtain the service running result of the test case set;

[0058] The removal module is used to perform a removal operation on the test cases in the first test case set that failed in service operation according to the service operation result;

[0059] The sending module is used to send a first use case removal notification to the second server, and the first use case removal notification includes: an identifier of a test case that has been removed in the first server, and the first use case removal notification is used to enable the second server to call a test task to run the test cases in a second test case set, and obtain a first test running result of the second test case set. If the test running of the removed test case in the first test running result fails, it is determined that the removed test case is a first false alarm test case.

[0060] Optionally, the device further includes: a receiving module, configured to receive a second use case removal notification sent by the second server, wherein the second use case removal notification includes: an identifier of a second false positive test case; the second false positive test case is a test case in the second test case set that is run by the second server when calling the test task, and a second test run result of the second test case set is obtained, and a test case that fails the test run is determined according to the second test run result;

[0061] The removal module is specifically used to perform a removal operation on the second false alarm test case.

[0062] Optionally, the device further comprises: a replacement module, wherein:

[0063] The receiving module is specifically used to receive a use case return notification sent by the second server, and the use case return notification includes: a repaired test case.

[0064] The replacement module is specifically used to replace the false positive test cases in the first test case set with the repaired test cases.

[0065] Optionally, the calling module is specifically configured to respond to an online request for the code of the project to be tested, and then call the service task to run the test cases in the first test case set.

[0066] In the fifth aspect of the present application, the present application also provides a server, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, when the server is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of any method described in the first aspect above.

[0067] In the seventh aspect of the present application, the present application also provides a server, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, when the server is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of any method described in the second aspect above.

[0068] In the eighth aspect of the present application, the present application also provides a test case processing system, including: a first server and a second server; the first server and the second server are communicatively connected, wherein the first server is used to execute the method described in any one of the first aspect above, and the second server is used to execute the method described in any one of the second aspect above.

[0069] In a ninth aspect of the present application, the present application further provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any method described in the first aspect are executed.

[0070] Based on any of the above aspects, a test case processing method provided by the present application can be used to remove the first test case sent by the first server, and call the test task to run the test case in the second test case set to obtain a first test running result. If in the first test running result, the test case that failed to run is the same as the removed test case included in the removal notification, then the removed test case is determined to be the first false alarm test case. This setting method makes it possible to determine the first false alarm test case in the test case through the cooperation between the first server and the second server, thereby solving the problem of low efficiency and high cost caused by manual troubleshooting of false alarm test cases in the prior art, thereby improving the testing efficiency of the test case, and automatically determining the role of the false alarm test case. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 A schematic diagram of the structure of a test case processing system provided in one embodiment of the present application;

[0073] Figure 2 A flowchart of a test case processing method provided in one embodiment of the present application;

[0074] Figure 3A flowchart of a test case processing method provided by another embodiment of the present application;

[0075] Figure 4 A flowchart of a test case processing method provided by another embodiment of the present application;

[0076] Figure 5 A flowchart of a test case processing method provided by another embodiment of the present application;

[0077] Figure 6 A flowchart of a test case processing method provided by another embodiment of the present application;

[0078] Figure 7 A flowchart of a test case processing method provided by another embodiment of the present application;

[0079] Figure 8 A flowchart of a test case processing method provided by another embodiment of the present application;

[0080] Fig. 9 A schematic diagram of the structure of a test case processing device provided in one embodiment of the present application;

[0081] Fig.10 A schematic diagram of the structure of a test case processing device provided in another embodiment of the present application;

[0082] Fig.11 A schematic diagram of the structure of a test case processing device provided in another embodiment of the present application;

[0083] Fig.12 A schematic diagram of the structure of a test case processing device provided in another embodiment of the present application;

[0084] Fig.13 A schematic diagram of the structure of a first server provided in an embodiment of the present application;

[0085] Fig.14 A schematic diagram of the structure of a second server provided in one embodiment of the present application. DETAILED DESCRIPTION

[0086] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0087] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0088] In order to enable those skilled in the art to use the content of this application, the following implementation is provided in combination with a specific application scenario automated test case scenario. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0089] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0090] To facilitate understanding of the embodiments of the present application, some of the terms involved in the present application are explained below:

[0091] Automated test cases: automated test cases that serve R&D and online development.

[0092] Extraction: prohibit the execution of a certain automated test case, or block the execution result of a certain test case so that it is not perceived by the online tester.

[0093] False positive: The automated execution fails, but it is not caused by a code failure, but by other factors such as the test environment and use case design. This type of failure does not have much reference value for intercepting code failures and is a false positive test case.

[0094] Jenkins is an open source software project. It is a continuous integration tool developed based on Java. It is used to monitor continuous repetitive work. It aims to provide an open and easy-to-use software platform that enables continuous software integration.

[0095] One aspect of the present application relates to a test case processing system. The system can implement the operation of a first test case set and a second test case set based on the cooperation between a user's first server and a second server, obtain the removed test case and the first test operation result, and determine the first false positive test case based on the removed test case and the first test operation result, thereby implementing the automated testing of the test case set and the automated determination of the false positive test case.

[0096] It is worth noting that before the present application was filed, the prior art usually tested the online project manually, tested each use case, determined the failed use cases by manual screening, and then processed the failed use cases to ensure that each use case in the online project can be successfully executed. However, this processing method is costly and inefficient, and in actual applications, due to the test environment, use case design and other factors, false alarms may occur. A large number of false alarms may affect the availability and online efficiency of the use cases, further reducing efficiency.

[0097] The test case processing method provided in the present application can remove the first test case from the notification received from the first server, and call the test task to run the test cases in the second test case set to obtain the first test running result. If in the first test running result, the test case that failed to run is the same as the removed test case included in the removal notification, then the removed test case is determined to be the first false alarm test case. This setting method makes it possible to determine the first false alarm test case in the test case through the cooperation between the first server and the second server, thereby solving the problem of low efficiency and high cost caused by manual troubleshooting of false alarm test cases in the prior art, thereby improving the testing efficiency of the test cases, and can automatically determine the role of the false alarm test case.

[0098] Figure 1 1 is a schematic diagram of the architecture of a test case processing system 100 provided in an embodiment of the present application. For example, the test case processing system 100 may be a service for automating test cases, or any platform or scenario involving test case testing. Figure 1 As shown, the test case processing system 100 may include a first server 110 and a second server 120 , wherein the first server 110 and the second server 120 are communicatively connected with each other.

[0099] In some embodiments, the first server 110 and the second server 120 may each include a processor. In some embodiments, the processor may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (M)). By way of example only, the processor may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), or a microprocessor, or any combination thereof.

[0100] In some embodiments, the test case processing system 100 may further include a database 130, wherein the database 130 may be connected to a network to communicate with one or more components (e.g., the first server 110, the second server 120, a service provider, etc.) in the test case processing system 100. One or more components in the test case processing system 100 may access data or instructions stored in the database 130 via the network. In some embodiments, the database 130 may be directly connected to one or more components in the test case processing system 100, or the database 130 may also be a part of the first server 110 or the second server 120.

[0101] For example, the first server 110 and the second server 120 are two independent servers, and the first server 110 and the second server 120 can be connected by remote procedure call (RPC) communication, so that the service tasks on the first server 110 and the second server 120 can communicate. The first server 110 and the second server 120 are respectively established with a set of tasks, and the two sets of tasks are used to execute two sets of automated test case sets respectively, and the two servers transmit the extracted or put back information through RPC according to the test results of the two sets of automated test case sets and the business logic. In the implementation process, an idempotent mechanism is deployed on each server. Through the design of the idempotent mechanism, the uniqueness of the transmitted removal information is guaranteed when the RPC is retried due to network problems; the storage persistence and data integrity of the removal information are guaranteed through the pre-write log system / …… (WAL: Write-Ahead Logging / ……, WAL / AOF) mechanism to avoid power failure loss.

[0102] In one embodiment of the present application, the first server 110 may be, for example, Odin. During the R&D process, if the server detects that the R&D personnel have pre-deployed the R&D product, such as the code of the project to be tested, an open source software project, such as Jenkins, is called to call the service task Server Job to run the test cases in the first test case combination to test the code of the project to be tested, and obtain the service operation result of the test case combination, and determine from the first test case combination that the test cases that failed the service operation are the removed test cases; the second server 120 may be, for example, carrot, and the second server 120 may trigger the call of the test task Test Job to run the test cases in the second test case set according to preset rules, such as every preset time interval, to obtain the first test operation result of the second test case set; and determine the first false alarm test case according to the first test operation result and the removed test case, wherein the first false alarm test case is the removed test case in the first test operation result.

[0103] The service task and the test task run independently without interfering with each other, but the second server 120 analyzes the running results of the service task and the test task and communicates between the two running results, thereby realizing the removal of the control case.

[0104] The following is combined with the above Figure 1The contents described in the test case processing system 100 shown in the figure provide a detailed description of the test case processing method provided in the embodiment of the present application. The following test case processing method is applied to the above system. The execution subject may be a service terminal or a server. The preset scenarios may be designed and adjusted according to user needs. Any scenario involving automated case testing or case testing may be used, and is not limited to the two scenarios given in the embodiment.

[0105] Reference Figure 2 FIG. 1 is a flow chart of a test case processing method provided by an embodiment of the present application. The method may be executed by a second server. The method may include:

[0106] S101: Receive a first use case removal notification sent by a first server.

[0107] The first test case removal notification includes: the identification of the test case that has been removed in the first server, such as a timestamp identification, or a unique identification of the test case, etc. The removed test case is the service running result of the test case set obtained by the first server calling the service task to run the test case in the first test case set to test the code of the project to be tested, which is the test case that failed to run the service determined from the first test case set. The service task is used to produce a test report for the first test case set to provide quality assessment results for R&D launch.

[0108] S102: Calling a test task to run the test cases in the second test case set, and obtaining a first test running result of the second test case set.

[0109] The test cases included in the first test case set and the second test case set are the same, and the same test cases are deployed on different servers, but the second test case set is not controlled by the R&D online; the test task runs the test cases in the second test case set and obtains the first test running result, which includes the identification of each test case and the corresponding running result, such as running successfully or running failed. The test task is used to monitor each test case in the second test case set.

[0110] If the test execution from which the test case has been removed in the first test execution result fails, S103 is executed.

[0111] S103: Determine that the removed test case is the first false positive test case.

[0112] For example, in some possible embodiments, for example, based on the identifier of the removed test case, in the first test running result, the corresponding removed test case in the first test case set can be determined according to the identifier, and the running result of the test case can be obtained. If the running result of the test case is failure, it means that the test case failed in the tests of the first server and the second server, and the removed test case is determined to be the first false alarm test case.

[0113] If the test run of the test case that has been removed from the first test run result is successful, the test case can continue to be monitored, for example, the test case can be run again for the first preset number of times, the first preset number of times can be, for example, 3 times, if the result of each run is successful, it means that the test case can run stably. At this time, the second server can notify the first server to put the test case back into the first test case set, where the setting of the first preset number of times can be flexibly adjusted according to user needs and is not limited to the above embodiment.

[0114] By adopting a test case processing method provided in the present application, a first test case removal notification sent by a first server can be received, and a test task can be called to run the test cases in a second test case set to obtain a first test running result. If in the first test running result, the test case that failed to run is the same as the removed test case included in the removal notification, then the removed test case is determined to be the first false alarm test case. This setting method makes it possible to determine the first false alarm test case in the test case through the cooperation between the first server and the second server, thereby solving the problem of low efficiency and high cost caused by manual troubleshooting of false alarm test cases in the prior art, thereby improving the testing efficiency of the test cases, and automatically determining the role of false alarm test cases.

[0115] Optionally, based on the above embodiment, the embodiment of the present application may also provide a test case processing method, and the implementation process of the above method is illustrated below in conjunction with the accompanying drawings. Figure 3 A flow chart of a test case processing method provided in another embodiment of the present application is as follows: Figure 3 As shown, the method may also include:

[0116] S204: Calling the test task to run the test cases in the second test case set, and obtaining a second test running result of the second test case set.

[0117] For example, in some possible embodiments, the second server can call the test task to run the test cases in the second test case set according to a preset interval time, and obtain the second test running results, where the preset time interval can be, for example, 10 minutes, etc. The setting of the specific preset time interval can be flexibly adjusted according to user needs and is not limited to the above embodiments.

[0118] If there is a test case in the second test running result that fails to run, S205 is executed.

[0119] S205: Determine that the test case that failed the test run is a second false positive test case.

[0120] For example, if the first test case set and the second test case combination both include M test cases, then during the continuous running of the test task for the test case combination, if P test cases fail, it means that the test task has discovered P false positive cases in advance.

[0121] S206: Send a second use case removal notification to the first server.

[0122] At this time, in order to avoid false alarms affecting the launch of R&D, the second server needs to send a second use case removal notification to the first server. The second use case removal notification includes: the identifier of the second false alarm test case. The second use case removal notification is used to instruct the first server to perform a removal operation on the second false alarm test case; so that the service task in the first server removes these P second false alarm test cases. At this time, the test cases running in the first test case set running by the service task only include MP test cases.

[0123] For example, in some possible embodiments, if the value of P is too large, for example, the ratio between P and M is greater than a preset ratio, such as 40%, this indicates that the test case may have a major fault, and the problem cannot be solved by removal, and manual troubleshooting is required. At this time, no testing will be performed and the manual troubleshooting stage will be entered. It should be understood that the preset ratio can be flexibly adjusted according to user needs, and can also be any ratio such as 10% or 30%, and is not limited to the several ratios given in the above embodiments.

[0124] This setting allows the second server to pre-mine the second test case set, pre-mine the second false positive test cases in the second test case set, and promptly feed back to the first server, thereby avoiding the problem of false positives affecting R&D launch.

[0125] Optionally, based on the above embodiment, the embodiment of the present application may also provide a test case processing method, and the implementation process of the above method is illustrated below in conjunction with the accompanying drawings. Figure 4A flow chart of a test case processing method provided in another embodiment of the present application is as follows: Figure 4 As shown, the method may also include:

[0126] S207: issuing a use case repair notification, where the use case repair notification is used to notify the user to repair the false positive test case.

[0127] The false alarm test case may be the first false alarm test case or the second false alarm test case, and the issued test case repair notification is used to notify troubleshooting personnel to manually check and repair the false alarm test case.

[0128] S208: Receive the input repaired test case.

[0129] In some possible embodiments, for example, the repaired test case may be uploaded to the second server by a troubleshooter.

[0130] S209: Replace the false positive test cases in the second test case set with the repaired test cases.

[0131] For example, in some possible embodiments, after the false alarm test cases in the second test case set are replaced with the repaired test cases, the test task can be called multiple times again to run the test cases in the second test case set to obtain a third test running result of the second test case set; if the repaired test cases in the third test running results are successfully tested and run for a first preset number of times in a row, it means that the operation of the repaired test cases has recovered stably. At this time, a case return notification is sent to the first server, and the case return notification includes: the repaired test cases. The case return notification is used to enable the first server to replace the false alarm test cases in the first test case set with the repaired test cases, and resume the operation of the repaired test cases.

[0132] Among them, the running result of the test case is still checked in the third test running result according to the identification of the repaired test case. If the running result is a successful running, and the number of successful running is greater than the first preset number of times, for example, it can be 3 times, then the second server sends a case return notification to the first server, so that the service task in the first server replaces the false alarm test case in the first test case set with the repaired test case, and resumes the running of the repaired test case. It should be understood that the specific number of the first preset number of times can be flexibly adjusted according to user needs, and is not limited to the above embodiment.

[0133] Optionally, based on the above embodiment, the embodiment of the present application may also provide a test case processing method, and the implementation process of the above method is illustrated below in conjunction with the accompanying drawings. Figure 5A flow chart of a test case processing method provided in another embodiment of the present application is as follows: Figure 5 As shown, the method may also include:

[0134] S210: Call the test task multiple times to run the test cases in the second test case set, and obtain a third test running result of the second test case set.

[0135] If the stable test cases other than the false alarm test cases in the third test running result are tested and run successfully for a second preset number of consecutive times, S211 is executed.

[0136] S211: Perform a removal operation on stable test cases in the second test case set.

[0137] Optionally, in one embodiment of the present application, the second preset number of times may be, for example, 1000 times, that is, if there is a test case in the second test case set that is run successfully for 1000 consecutive times, it is determined that the test case can continue to run stably and can be removed from the second test case set, thereby reducing the burden of the test task. If the stable test case fails to run when running in the service case, the first server will send the identifier of the failed stable test case to the second server, so that the second server will resume the test task monitoring of the failed stable test case until it is removed after the next stability.

[0138] By adopting a test case processing method provided by the present application, a first test case removal notification sent by a first server can be received, and a test task can be called to run the test cases in the second test case set to obtain a first test running result. If in the first test running result, the test case that failed to run is the same as the removed test case included in the removal notification, the removed test case is determined to be the first false alarm test case; in addition, the second server can also test the second test case according to a preset interval, determine that the test case that failed to run is the second false alarm test case, and send a second false alarm notification to the first server to notify the first server to remove the second false alarm test case from the first test case set. This setting method enables the method provided by the present application to determine the first false alarm test case in the test case through the cooperation between the first server and the second server, thereby solving the problem of low efficiency and high cost caused by manual investigation of false alarm test cases in the prior art, thereby improving the testing efficiency of the test case, and automatically determining the false alarm test case.

[0139] Optionally, based on the above embodiment, the embodiment of the present application may also provide a test case processing method, and the implementation process of the above method is illustrated below in conjunction with the accompanying drawings. Figure 6A flow chart of a test case processing method provided in another embodiment of the present application, the method is applied to a first server, such as Figure 6 As shown, the method may include:

[0140] S301: Calling a service task to run the test cases in the first test case set to test the code of the project to be tested to obtain the service running result of the test case set.

[0141] Optionally, in one embodiment of the present application, the first server responds to the online request of the project code to be tested, and then calls the service task to run the test cases in the first test case set.

[0142] S302: According to the service running result, remove the test cases in the first test case set that have failed in service running.

[0143] S303: Send a first use case removal notification to the second server.

[0144] Among them, the first use case removal notification includes: the identification of the test case that has been removed in the first server, the first use case removal notification is used to enable the second server to call the test task to run the test cases in the second test case set, and obtain the first test running result of the second test case set. If the test running of the removed test case in the first test running result fails, it is determined that the removed test case is the first false alarm test case.

[0145] The above method is Figure 1-Figure 5 The method provided is the method of executing the main body on the opposite side, so the beneficial effects brought by the method are the same as those mentioned above. Figure 1-Figure 5 The methods provided are the same and will not be described in detail in this application.

[0146] Optionally, based on the above embodiment, the embodiment of the present application may also provide a test case processing method, and the implementation process of the above method is illustrated below in conjunction with the accompanying drawings. Figure 7 A flow chart of a test case processing method provided in another embodiment of the present application is as follows: Figure 7 As shown, the method may also include:

[0147] S304: Receive a second use case removal notification sent by the second server.

[0148] Among them, the second use case removal notification includes: the identifier of the second false alarm test case; the second false alarm test case is the test case in the second test case set run by the second server when calling the test task, and obtaining the second test running result of the second test case set, and according to the second test running result, determining the test case that failed the test running.

[0149] S305: Perform a removal operation on the second false positive test case.

[0150] Optionally, based on the above embodiment, the embodiment of the present application may also provide a test case processing method, and the implementation process of the above method is illustrated below in conjunction with the accompanying drawings. Figure 8 A flow chart of a test case processing method provided in another embodiment of the present application is as follows: Figure 8 As shown, the method may also include:

[0151] S306: Receive a use case return notification sent by the second server.

[0152] The use case release notification includes: the repaired test case.

[0153] S307: Replace the false positive test cases in the first test case set with the repaired test cases.

[0154] The above method is Figure 1-Figure 5 The method provided is the method of executing the main body on the opposite side, so the beneficial effects brought by the method are the same as those mentioned above. Figure 1-Figure 5 The methods provided are the same and will not be described in detail in this application.

[0155] Based on the same inventive concept, the present application also provides a test case processing device corresponding to the test case processing method. Since the principle of solving the problem by the device in the present application is the same as that in the above-mentioned embodiment of the present application, Figure 1-Figure 5 The test case processing method is similar, so the implementation of the device can refer to the implementation of the method, and the repetition of the beneficial effects will not be repeated.

[0156] Fig. 9 A schematic diagram of a test case processing device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the device includes: a receiving module 401, a calling module 402 and a determining module 403, wherein:

[0157] Receiving module 401 is used to receive a first use case removal notification sent by the first server, the first use case removal notification includes: an identifier of the removed test case in the first server, the removed test case is a service running result of the test case set obtained by the first server calling a service task to run the test case in the first test case set to test the code of the project to be tested, and a test case determined from the first test case set that failed in the service running.

[0158] The calling module 402 is used to call the test task to run the test cases in the second test case set and obtain the first test running result of the second test case set.

[0159] The determination module 403 is configured to determine that the removed test case is a first false positive test case if the test run of the removed test case in the first test run result fails.

[0160] Fig.10 A structural diagram of a test case processing device provided in another embodiment of the present application is shown as follows: Fig.10 As shown, the device also includes: a sending module 404, wherein:

[0161] The calling module 402 is specifically used to call the test task to run the test cases in the second test case set, and obtain the second test running result of the second test case set;

[0162] The determination module 403 is specifically configured to determine that the test case that failed in the test running is a second false positive test case if there is a test case that failed in the test running result;

[0163] The sending module 404 is used to send a second use case removal notification to the first server, wherein the second use case removal notification includes: an identifier of the second false alarm test case, and the second use case removal notification is used to instruct the first server to perform a removal operation on the second false alarm test case.

[0164] like Fig.10 As shown, the device further includes: a replacement module 405, wherein:

[0165] The sending module 404 is specifically used to send a use case repair notification, where the use case repair notification is used to notify the user to repair the false positive test case.

[0166] The receiving module 401 is specifically configured to receive an input repaired test case.

[0167] The replacement module 405 is used to replace the false positive test case in the second test case set with a repaired test case.

[0168] Optionally, the calling module 402 is specifically configured to call the test task multiple times to run the test cases in the second test case set to obtain a third test running result of the second test case set.

[0169] The sending module 404 is specifically used to send a test case release notification to the first server if the repaired test case in the third test running result has been successfully tested and run for a first preset number of times in a row. The test case release notification includes: the repaired test case. The test case release notification is used to enable the first server to replace the false positive test case in the first test case set with the repaired test case, and resume the running of the repaired test case.

[0170] like Fig.10 As shown, the device further includes: a removal module 406, wherein:

[0171] The calling module 402 is specifically configured to call the test task multiple times to run the test cases in the second test case set, and obtain a third test running result of the second test case set.

[0172] The removal module 406 is used to perform a removal operation on the stable test cases in the second test case set if the stable test cases other than the false positive test cases in the third test running result are tested and run successfully for a second preset number of consecutive times.

[0173] Optionally, the calling module 402 is specifically configured to call the test task to run the test cases in the second test case set according to a preset interval time.

[0174] Based on the same inventive concept, the present application also provides a test case processing device corresponding to the test case processing method. Since the principle of solving the problem by the device in the present application is the same as that in the above-mentioned embodiment of the present application, Figure 6-Figure 8 The test case processing method is similar, so the implementation of the device can refer to the implementation of the method, and the repetition of the beneficial effects will not be repeated.

[0175] Fig.11 A structural diagram of a test case processing device provided in another embodiment of the present application is shown as follows: Fig.11 As shown, the device includes: a calling module 501, a removing module 502 and a sending module 503, wherein:

[0176] The calling module 501 is used to call a service task to run the test cases in the first test case set to test the code of the project to be tested and obtain the service running result of the test case set.

[0177] The removal module 502 is used to remove the test cases in the first test case set that failed in service operation according to the service operation result.

[0178] The sending module 503 is used to send a first use case removal notification to the second server, and the first use case removal notification includes: an identifier of the test case that has been removed in the first server, and the first use case removal notification is used to enable the second server to call a test task to run the test cases in the second test case set, and obtain a first test running result of the second test case set. If the test running of the removed test case in the first test running result fails, it is determined that the removed test case is a first false alarm test case.

[0179] Fig.12 A structural diagram of a test case processing device provided in another embodiment of the present application is shown as follows: Fig.12As shown, the device also includes: a receiving module 504, which is used to receive a second use case removal notification sent by the second server, and the second use case removal notification includes: an identifier of a second false alarm test case; the second false alarm test case is a test case in a second test case set run by the second server when calling a test task, and a second test running result of the second test case set is obtained, and according to the second test running result, a test case that fails the test running is determined.

[0180] The removal module 502 is specifically configured to perform a removal operation on the second false positive test case.

[0181] like Fig.12 As shown, the device further includes: a replacement module 505, wherein:

[0182] The receiving module 504 is specifically configured to receive a use case return notification sent by the second server, where the use case return notification includes: a repaired test case.

[0183] The replacement module 505 is specifically configured to replace the false positive test cases in the first test case set with the repaired test cases.

[0184] Optionally, the calling module 501 is specifically configured to respond to an online request for the code of the project to be tested, and then call a service task to run the test cases in the first test case set.

[0185] Fig.13 A schematic diagram of the structure of a first server provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the first server includes: a processor 601, a memory 602 and a bus 603; the memory 602 stores machine-readable instructions executable by the processor 601. When the first server is running, the processor 601 communicates with the memory 602 via the bus 603, and the processor 601 executes the machine-readable instructions to perform the method embodiment as described above. Figure 1-Figure 5 The steps of the test case processing method provided.

[0186] Specifically, the machine-readable instructions stored in the memory 602 are the execution steps of the test case processing method described in the aforementioned embodiment of the present application. The processor 601 can execute the test case processing method to process the test case set. Therefore, the first server also has all the beneficial effects described in the aforementioned method embodiment, and this application will not repeat the description.

[0187] It should be noted that the first server can be a general-purpose computer or a special-purpose computer, as well as other servers for processing data, etc., all of which can be used to implement the first server method of the present application. Although the present application only describes the first server method through a computer and a server respectively, for the sake of convenience, the functions described in the present application can also be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0188] For example, the memory 602 may include one or more processors, communication buses, and storage media of different forms, such as disks, ROMs, or RAMs, or any combination thereof, for executing program instructions. Exemplarily, the computer platform may also include program instructions stored in ROMs, RAMs, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented according to these program instructions.

[0189] For ease of description, only one processor is described in the above embodiment. However, it should be noted that the first server in the present application may also include multiple processors, so the steps performed by one processor described in the present application may also be performed jointly or individually by multiple processors.

[0190] Fig.14 A schematic diagram of the structure of a second server provided in an embodiment of the present application is shown in FIG. Fig.14 As shown, the second server includes: a processor 701, a memory 702 and a bus 703; the memory 702 stores machine-readable instructions executable by the processor 701. When the second server is running, the processor 701 communicates with the memory 702 via the bus 703, and the processor 701 executes the machine-readable instructions to perform the method embodiment as described above. Figure 6-Figure 8 The steps of the test case processing method provided.

[0191] Specifically, the machine-readable instructions stored in the memory 702 are the execution steps of the test case processing method described in the aforementioned embodiment of the present application. The processor 701 can execute the test case processing method to process the test case set. Therefore, the second server also has all the beneficial effects described in the aforementioned method embodiment, and this application will not repeat the description.

[0192] It should be noted that the second server can be a general-purpose computer or a special-purpose computer, as well as other servers for processing data, etc., all of which can be used to implement the second server method of the present application. Although the present application only describes the second server method through a computer and a server respectively, for convenience, the functions described in the present application can also be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0193] For example, the memory 702 may include one or more processors, communication buses, and storage media of different forms, such as disks, ROMs, or RAMs, or any combination thereof, for executing program instructions. Exemplarily, the computer platform may also include program instructions stored in ROMs, RAMs, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented according to these program instructions.

[0194] For ease of description, only one processor is described in the above embodiment. However, it should be noted that the second server in the present application may also include multiple processors, so the steps performed by one processor described in the present application may also be performed jointly or individually by multiple processors.

[0195] The present application also provides a storage medium on which a computer program is stored. The computer program is executed by a processor. Figure 1-Figure 8 The steps of the method are provided.

[0196] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the above-mentioned Figure 1-Figure 8 The provided method thus solves the problem of high cost and low efficiency in testing use cases in the prior art.

[0197] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0198] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0199] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0200] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0201] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A test case processing method, characterized in that: The method is applied to a second server; The method comprises: Receive a first use case removal notification sent by a first server, the first use case removal notification including: an identifier of a test case that has been removed in the first server, the test case that has been removed being a service running result of the test case set obtained by the first server calling a service task to run a test case in a first test case set to test a code of a project to be tested, and a test case determined from the first test case set that has failed in service running; Calling a test task to run the test cases in a second test case set, and obtaining a first test running result of the second test case set; the test cases included in the first test case set and the second test case set are the same; If the test run of the removed test case in the first test run result fails, it is determined that the removed test case is a first false positive test case.

2. The method according to claim 1, characterized in that The method further comprises: Calling the test task to run the test cases in the second test case set, and obtaining a second test running result of the second test case set; If there is a test case in the second test running result that fails to run the test, determining the test case that failed to run the test is a second false positive test case; A second use case removal notification is sent to the first server, wherein the second use case removal notification includes: an identifier of the second false alarm test case, and the second use case removal notification is used to instruct the first server to perform a removal operation on the second false alarm test case.

3. The method according to claim 2, characterized in that The method further comprises: Sending a use case repair notification, wherein the use case repair notification is used to notify the user to repair the false positive test case; Receive input of the repaired test case; The false positive test case in the second test case set is replaced with the repaired test case.

4. The method according to claim 3, characterized in that The method further comprises: Calling the test task multiple times to run the test cases in the second test case set to obtain a third test running result of the second test case set; If the repaired test case in the third test running result has been successfully tested for a first preset number of times in a row, a use case release notification is sent to the first server, and the use case release notification includes: the repaired test case, and the use case release notification is used to enable the first server to replace the false alarm test case in the first test case set with the repaired test case, and resume the running of the repaired test case.

5. The method according to claim 1, characterized in that The method further comprises: Calling the test task multiple times to run the test cases in the second test case set to obtain a third test running result of the second test case set; If the stable test cases other than the false alarm test cases in the third test running result are tested and run successfully for a second preset number of times in a row, a removal operation is performed on the stable test cases in the second test case set.

6. The method according to any one of claims 1 to 5, characterized in that: The calling test task to run the test cases in the second test case set includes: According to a preset interval, the test task is called to run the test cases in the second test case set.

7. A test case processing method, characterized in that: The method is applied to a first server; the method comprises: Calling a service task to run the test cases in the first test case set to test the code of the project to be tested to obtain a service running result of the test case set; According to the service operation result, removing the test cases in the first test case set whose service operation fails; A first use case removal notification is sent to a second server, wherein the first use case removal notification includes: an identifier of a test case that has been removed from the first server, the first use case removal notification is used to enable the second server to call a test task to run the test cases in a second test case set, and obtain a first test running result of the second test case set; if the test running of the removed test case in the first test running result fails, the removed test case is determined to be a first false alarm test case; the test cases included in the first test case set and the second test case set are the same.

8. The method according to claim 7, characterized in that The method further comprises: Receive a second use case removal notification sent by the second server, the second use case removal notification including: an identifier of a second false positive test case; the second false positive test case is a test case in the second test case set that is run by the second server when calling the test task, obtain a second test run result of the second test case set, and determine the test case that failed the test run according to the second test run result; Perform a removal operation on the second false positive test case.

9. The method according to claim 7, characterized in that The method further comprises: receiving a use case return notification sent by the second server, wherein the use case return notification includes: a repaired test case; Replace the false positive test cases in the first test case set with the repaired test cases.

10. The method according to any one of claims 7 to 9, characterized in that: The calling service task runs the test cases in the first test case set to test the code of the project to be tested, and obtains the service running result of the test case set, including: In response to the online request of the project code to be tested, the service task is called to run the test cases in the first test case set.

11. A test case processing device, characterized in that: The device is applied to a second server; the device comprises: a receiving module, a calling module and a determining module, wherein: The receiving module is used to receive a first use case removal notification sent by the first server, wherein the first use case removal notification includes: an identifier of a test case that has been removed in the first server, the test case that has been removed is a service running result of the test case set obtained by the first server calling a service task to run a test case in a first test case set to test the code of the project to be tested, and a test case determined from the first test case set that has failed in service running; The calling module is used to call the test task to run the test cases in the second test case set to obtain the first test running result of the second test case set; the test cases included in the first test case set and the second test case set are the same; The determination module is used to determine that the removed test case is a first false positive test case if the test running of the removed test case in the first test running result fails.

12. A test case processing device, characterized in that: The device is applied to a first server; the device comprises: a calling module, a removing module and a sending module, wherein: The calling module is used to call a service task to run the test cases in the first test case set to test the code of the project to be tested and obtain the service running result of the test case set; The removal module is used to perform a removal operation on the test cases in the first test case set that failed in service operation according to the service operation result; The sending module is used to send a first use case removal notification to the second server, and the first use case removal notification includes: an identifier of a test case that has been removed in the first server, and the first use case removal notification is used to enable the second server to call a test task to run the test cases in a second test case set to obtain a first test running result of the second test case set; if the test running of the removed test case in the first test running result fails, it is determined that the removed test case is a first false alarm test case; the test cases included in the first test case set and the second test case set are the same.

13. A server, characterized in that: The server includes: a processor, a storage medium and a bus, the storage medium stores machine-readable instructions executable by the processor, and when the server is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the method described in any one of claims 1 to 6.

14. A server, characterized in that: The server includes: a processor, a storage medium and a bus, the storage medium stores machine-readable instructions executable by the processor, when the server is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to execute the method described in any one of claims 7 to 10.

15. A test case processing system, characterized in that: include: A first server and a second server; the first server and the second server are communicatively connected, wherein the first server is used to execute the method described in any one of claims 1 to 6 above, and the second server is used to execute the method described in any one of claims 7 to 10 above.

16. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is executed.

Images