Automated Testing Method and Distributed System

Abstract

The present invention discloses an automated testing method for a distributed system. The distributed system includes a task scheduling service, an image production service, a test case distribution service, a test case running service, and a result collection service. The method includes: the task scheduling service receives a test task and parses task information from the test task; the image production service configures a running environment required for testing according to the task information, generates an image file of the running environment, and downloads test cases to be tested; the test case distribution service starts the test case running service according to the task information and the number of test cases; the test case running service runs the test cases according to the image file of the running environment; the result collection service collects the test results of the test cases. The present invention also discloses a distributed system for automated testing.

Description

Technical Field

[0001] The present invention relates to the technical field of automated testing, and particularly relates to an automated testing method for a distributed system and a distributed system for automated testing. Background Art

[0002] Automated testing refers to automatically running a system or an application program under preset conditions and evaluating the running results. In the prior art, the Jemeter master-slave mode is often used to implement the automated testing process. However, the Jemeter master-slave mode has great drawbacks: during the testing process, the master node assigns tasks to each slave node and aggregates the task results, resulting in excessive pressure on the master node and easy collapse, thus affecting the testing progress. Summary of the Invention

[0003] The purpose of the present invention is to provide an automated testing method for a distributed system and a distributed system for automated testing, which can solve the technical problem that once the master node collapses due to excessive pressure during testing using the Jemeter master-slave mode, it will seriously affect the testing progress.

[0004] One aspect of the present invention provides an automated testing method for a distributed system. The distributed system includes a task scheduling service, an image making service, a test case distribution service, a test case running service, and a result collection service. The method includes: the task scheduling service receives a test task and parses task information from the test task; the image making service configures a running environment required for testing according to the task information, generates an image file of the running environment, and downloads test cases to be tested; the test case distribution service starts the test case running service according to the task information and the number of test cases; the test case running service runs the test cases according to the image file of the running environment; the result collection service collects the test results of the test cases.

[0005] Optionally, the test case distribution service starts the test case running service according to the task information and the number of test cases, including: the test case distribution service determines the number of test cases downloaded by the image making service; the test case distribution service judges whether the number of the test case running services is defined in the task information to obtain a judgment result; the test case distribution service determines the final number of the test case running services according to the number of test cases and the judgment result; the test case distribution service starts the test case running services with the final number.

[0006] Optionally, the test case distribution service determines the final number of the test case running services according to the number of the test cases and the judgment result, including: when the judgment result indicates that the number of the test case running services is not defined, the test case distribution service obtains a first calculation rule associated with the judgment result; the test case distribution service calculates the number of the test case running services according to the first calculation rule and the number of the test cases; the test case distribution service directly uses the calculated number as the final number of the test case running services.

[0007] Optionally, the test case distribution service determines the final number of the test case running services according to the number of the test cases and the judgment result, including: when the judgment result indicates that the number of the test case running services is defined, the test case distribution service obtains a second calculation rule associated with the judgment result; the test case distribution service calculates the number of the test case running services according to the second calculation rule and the number of the test cases; the test case distribution service determines the final number of the test case running services according to the defined number and the calculated number.

[0008] Optionally, the test case distribution service calculates the number of the test case running services according to the second calculation rule and the number of the test cases, including: the test case distribution service obtains the test case numerical range defined in the second calculation rule; the test case distribution service filters out the test case numerical range having a mapping relationship with the number of the test cases from the obtained test case numerical range; the test case distribution service determines a sub-rule associated with the filtered test case numerical range in the second calculation rule; the test case distribution service calculates the number of the test case running services according to the determined sub-rule.

[0009] Optionally, the test case distribution service starts the test case running services with the final number, including: the test case distribution service obtains the configuration information of the test case running services defined in the task information; the test case distribution service starts the test case running services with the final number according to the configuration information.

[0010] Optionally, before the use case distribution service starts the use case running service according to the task information and the number of test cases, the method further includes: the image production service stores the image file of the running environment in the image sharing server; the image production service stores the downloaded test cases in a preset database; the use case running service runs the test cases according to the image file of the running environment, including: the use case running service sends a use case request for applying for test cases to the use case distribution service; after the use case running service sends a use case request for applying for test cases to the use case distribution service, the method further includes: the use case distribution service filters out several test cases from the database according to the use case request; the use case distribution service assigns the filtered test cases to the use case running service and deletes the filtered test cases from the database; the use case running service runs the test cases according to the image file of the running environment, further including: the use case running service receives the test cases assigned by the use case distribution service; the use case running service obtains the image file of the running environment from the image sharing server; the use case running service runs the received test cases according to the obtained image file of the running environment.

[0011] Optionally, the use case running service sends a use case request for applying for test cases to the use case distribution service, including: the use case running service determines whether it is the first time to apply for test cases to the use case distribution service; if it is not the first application, the use case running service determines the identifier of the most recently received test case and sends the use case request carrying the identifier to the use case running service; if it is the first application, the use case running service sends a use case request without carrying any identifier of test cases to the use case running service.

[0012] Optionally, the use case distribution service filters out several test cases from the database according to the use case request, including: the use case distribution service determines whether the use case request carries an identifier of a test case; if it carries, the use case distribution service determines a set of test cases associated with the identifier in the database and randomly filters out the several test cases from the determined set of test cases; if it does not carry, the use case distribution service determines whether there is an unassigned set of test cases in the database, if there is, the use case distribution service randomly filters out the several test cases from any unassigned set of test cases; if there is not, the use case distribution service randomly filters out the several test cases from any set of test cases in the database.

[0013] Optionally, after the result collection service collects the test results of the test cases, the method further includes: the result collection service stores the collected test results in the database; when the task scheduling service monitors that the number of test results stored in the database is equal to the number of test cases, the task scheduling service notifies the result collection service of the message that the test cases have been run; after receiving the notification, the result collection service calculates the pass rate of all test cases based on the collected test results; the task scheduling service determines whether the pass rate is greater than the pass rate threshold; if it is less, the task scheduling service obtains the re-run rule defined in the task information; the task scheduling service determines whether the pass rate meets the re-run rule; if it meets, the task scheduling service obtains the maximum number of re-runs defined in the task information; the task scheduling service determines whether the maximum number of re-runs is greater than 0; if it is greater, the task scheduling service parses the task information from the test task.

[0014] Optionally, the task scheduling service, the image production service, the test case distribution service, the test case running service, and the result collection service are different virtual machines or Docker containers.

[0015] Another aspect of the present invention provides a distributed system for automated testing. The distributed system includes a task scheduling service, an image production service, a test case distribution service, a test case running service, and a result collection service, wherein: the task scheduling service is used to receive a test task and parse task information from the test task; the image production service is used to configure the running environment required for testing according to the task information, generate an image file of the running environment, and download the test cases to be tested; the test case distribution service is used to start the test case running service according to the task information and the number of test cases; the test case running service is used to run the test cases according to the image file of the running environment; the result collection service is used to collect the test results of the test cases.

[0016] The automated testing method of the distributed system provided by the present invention. The distributed system includes a task scheduling service, an image production service, a test case distribution service, a test case running service, and a result collection service. There is no master-slave hierarchy among these services, and each undertakes part of the operation links in the testing process, and they cooperate with each other to complete the entire process of automated testing. The whole process does not require manual intervention, accelerates the running speed of test cases, improves the testing efficiency, and solves the problem that once the master node collapses due to excessive pressure during testing using the Jemeter master-slave mode in the prior art, it will seriously affect the testing progress. Description of the Drawings

[0017] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of illustrating the preferred embodiments and are not considered to be a limitation of the present invention. Also, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0018] Figure 1 shows a flowchart of an automated testing method for a distributed system provided in the first embodiment of the present invention;

[0019] Figure 2 shows a schematic diagram of a mirror production service generating a mirror file for a running environment provided in the first embodiment of the present invention;

[0020] Figure 3 shows a schematic diagram of the specific working process of a mirror production service provided in the first embodiment of the present invention;

[0021] Figure 4 shows a schematic diagram of a test case running service running test cases provided in the first embodiment of the present invention;

[0022] Figure 5 shows a schematic diagram of the complete life cycle of a test task provided in the first embodiment of the present invention;

[0023] Figure 6 shows an operation flowchart of each stage corresponding to a test task provided in the first embodiment of the present invention;

[0024] Figure 7 shows a schematic diagram of the business scenario of an automated testing process provided in the first embodiment of the present invention;

[0025] Figure 8 shows a schematic diagram of the flow process of the software under test provided in the first embodiment of the present invention;

[0026] Figure 9 shows a block diagram of a distributed system for automated testing provided in the second embodiment of the present invention.

[0027] Figure 10 shows a block diagram of a computer device for controlling a distributed system to execute an automated testing method provided in the third embodiment of the present invention. Detailed Embodiments

[0028] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0029] It should be noted that in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising such element.

[0030] Embodiment 1

[0031] Figure 1 The flowchart of the automated testing method for the distributed system provided in Embodiment 1 of the present invention is shown. The distributed system includes a task scheduling service, an image building service, a test case distribution service, a test case running service, and a result collection service. Among them, the task scheduling service, the image building service, the test case distribution service, the test case running service, and the result collection service are different virtual machines or Docker containers. As Figure 1 shown, this automated testing method includes steps S1 to S5, where:

[0032] Step S1, the task scheduling service receives a test task and parses task information from the test task.

[0033] After the task scheduling service receives the test task, it decomposes the test process into an image building stage, a waiting-to-run stage, a test case running stage, and a task end stage according to the test task. In addition, the task scheduling service also parses task information from the test task. This task information includes the test script download method, the test script address, the number of test case running services to be started, the configuration information of each test case running service, the maximum number of retries, and the retry rule, etc. Among them, the configuration information includes the hardware resources required by each test case running service, such as CPU resources, memory resources, etc.; the maximum number of retries is the maximum number of times to re-execute the test task when all test cases corresponding to the test task meet certain conditions, and the retry rule is that after running the test cases corresponding to the test task, when the passing rate of these test cases meets the retry rate, the test task will be retried.

[0034] Step S2, the image building service configures the running environment required for testing according to the task information, generates an image file of the running environment, and downloads the test cases to be tested.

[0035] In the image building stage, as Figure 2 shown, taking the image building service as a virtual machine as an example, Figure 2The left part is the process of starting a virtual machine, which is prior art and will not be elaborated in detail in this embodiment. Figure 2 The right part is the process of generating an image file of the running environment. Specifically, after the image making service obtains the task of making the image file from the task scheduling service, it downloads and installs the software under test, and then obtains the test script download method and the test script address defined in the task information. It downloads the test script from the test script address according to the test script download method, and downloads the test cases required for the test according to the downloaded test script. The image making service also copies the engine used to automatically operate the software under test to implement the test of the software under test, and then performs the customized operations defined by the tester. After the above preparations are completed, the running environment is configured and the test cases are downloaded. Further, the running environment is saved in the form of an image file. Then, the image making service stores the image file of the running environment in the image sharing server, and stores the downloaded test cases in a preset database. In addition, after the image file is made, the image making service notifies the task scheduling service of the completion of the production. As Figure 3 shown, Figure 3 is the workflow of the image making service. After the image making service is powered on, it obtains its own mac address, and then determines whether the task information is loaded. If the task information is loaded, it continues to determine whether the test task has ended. If the test task has not ended, it determines whether the image making has ended. If the image making has not ended, it continues to make the image; if the task information is not loaded, it determines whether the current task status is waiting to make the image. If so, it starts the image making service, otherwise it shuts down; if the test task ends, it shuts down; if the image making ends, it notifies the task scheduling service.

[0036] Step S3, the use case distribution service starts the use case running service for running the test cases according to the task information and the number of the test cases.

[0037] In the waiting to run stage, the task scheduling service will notify the use case distribution service to execute the tasks in the waiting to run stage. Before that, the task scheduling service will first determine the use case distribution service. That is, before step S3, the method further includes steps A1 to A5, where:

[0038] Step A1, the task scheduling service determines all candidate use case distribution services deployed in the distributed system;

[0039] Step A2, the task scheduling service verifies the availability of each candidate use case distribution service;

[0040] Step A3, the task scheduling service screens out the available candidate use case distribution services from all candidate use case distribution services;

[0041] Step A4, the task scheduling service determines the load of each available candidate use case distribution service;

[0042] Step A5, the task scheduling service uses the candidate use case distribution service with the minimum load as the use case distribution service, and sends the task information to the use case distribution service.

[0043] Specifically, the candidate use case distribution services in the distributed system adopt multi-service deployment to ensure the high availability of the candidate use case distribution services. The load of each candidate use case distribution service is determined by the number of tasks running or to be run on the candidate use case distribution service. Before the task scheduling service designates the use case distribution service, the availability of the candidate use case distribution service will be verified first. For example, the candidate use case distribution service provides an interface similar to a health check (i.e., the health interface). If the candidate use case distribution service receives a verification request sent by the task scheduling service through this interface, it will return an OK value through this interface. If the task scheduling service receives the OK value returned by this interface, it indicates that this candidate use case distribution service is available, otherwise it is considered unavailable; further, the task scheduling service selects the candidate use case distribution service with the current minimum load from all available candidate use case distribution services, and sends the task information to the candidate use case distribution service with the minimum load. During the testing process, if the selected use case distribution service is abnormal, the task scheduling service will automatically load the real-time data of the task information into the candidate use case distribution service with the current lowest load to ensure that the test results can be normally returned and continue to distribute subsequent test cases.

[0044] Further, after determining the use case distribution service, the use case distribution service will start the use case running service for running the test cases according to the task information and the number of test cases. Specifically, step S3 includes steps S31 to S34, where:

[0045] Step S31, the use case distribution service determines the number of test cases downloaded by the image production service;

[0046] Step S32, the use case distribution service determines whether the number of use case running services is defined in the task information, and obtains a judgment result;

[0047] Step S33, the use case distribution service determines the final number of use case running services according to the number of test cases and the judgment result;

[0048] Step S34, the use case distribution service starts the final number of use case running services.

[0049] Step S34 includes steps S341 to S342, where:

[0050] Step S431, the use case distribution service obtains the configuration information of the use case running service defined in the task information;

[0051] Step S432, the use case distribution service starts the final number of use case running services according to the configuration information.

[0052] Before the test is executed, there is no use case running service in the distributed system. This is to avoid the problem that the test process has not reached the use case running service, but due to some reason, the test process ends prematurely, resulting in the premature start of the use case running service occupying system resources without reason. At the same time, it avoids the problem that the number of prematurely started use case running services does not match the actual needs. For example, if the prematurely started number is greater than the actual needs, it also causes resource waste. Based on this, in this embodiment, during the execution of the test, the final number of use case running services is determined from two dimensions, which can improve the accuracy of determination and avoid resource waste. Further, the use case distribution service can start the final number of use case running services according to the configuration information of each use case running service defined in the task information.

[0053] Optionally, step S33 may include steps S331 to S333, where:

[0054] Step S331, when the judgment result indicates that the number of use case running services is not defined, the use case distribution service obtains the first calculation rule associated with the judgment result;

[0055] Step S332, the use case distribution service calculates the number of use case running services according to the first calculation rule and the number of test cases;

[0056] Step S333, the use case distribution service directly uses the calculated number as the final number of use case running services.

[0057] The first calculation rule may be: allocate one use case running service for N test cases and round up. N is a positive integer. For example, when N is 30 and the number of test cases is 100, the calculated number of use case running services is 4, and at this time, the final number of use case running services is also determined to be 4.

[0058] Optionally, step S33 may include steps S331' to S333', where:

[0059] Step S331', when the judgment result indicates that the number of use case running services is defined, the use case distribution service obtains the second calculation rule associated with the judgment result;

[0060] Step S332’, the use case distribution service calculates the number of use case running services according to the second calculation rule and the number of test cases;

[0061] Step S333’, the use case distribution service determines the final number of use case running services according to the defined number and the calculated number.

[0062] In this embodiment, the second calculation rule includes: multiple test case value ranges and sub-rules associated with each test case value range, where each test case value range is different, and the union of all test case value ranges is [0, +∞), and the lower limit value and the lower limit value of the test case value range are both the specific values of the test case. Optionally, step S332’ specifically includes:

[0063] The use case distribution service obtains the test case value ranges defined in the second calculation rule;

[0064] The use case distribution service filters out the test case value ranges that have a mapping relationship with the number of test cases from the obtained test case value ranges;

[0065] The use case distribution service determines the sub-rules associated with the filtered test case value ranges in the second calculation rule;

[0066] The use case distribution service calculates the number of use case running services according to the determined sub-rules.

[0067] For example, the test case value ranges and sub-rules included in the second calculation rule can be: when the specific value of the test case is less than 10: start one use case running service; when the specific value of the test case is greater than or equal to 11 and less than 100: start one use case running service for every 5 test cases; when the specific value of the test case is greater than or equal to 101 and less than 500: start one use case running service for every 8 test cases; when the specific value of the test case is greater than or equal to 501 and less than 1000: start one use case running service for every 20 test cases; when the specific value of the test case is greater than or equal to 1001 and less than 3500: the number of use case running services is 200; when the specific value of the test case is greater than or equal to 3501: the number of use case running services is 300.

[0068] In order to reduce the resource consumption while ensuring the processing speed, the smallest number can be selected from the defined number and the calculated number as the above-mentioned final number.

[0069] For example, if the number of downloaded test cases is 100, the number calculated according to the second calculation rule is 19, and if the defined number is 20, the final number is determined to be 19.

[0070] Optionally, during the waiting-to-run phase, it may not be possible to start the final number of test case running services simultaneously due to insufficient current system resources. In this case, the test case distribution service will first start a part of the test case running services, and then start the remaining test case running services after the system processes some data and releases the corresponding resources. However, when the remaining test case running services are started later, it is possible that the test task has been completed. Therefore, to ensure the smooth progress of the test process, for each started test case running service, actively check the accessibility of the test task, that is, check whether the task is within the valid period. When it is within the valid period, the test case running service performs initialization operations and then enters the test case running phase to execute step S4. That is, before step S4, the method further includes steps B1 to B4, where:

[0071] The test case running service sends a verification request to the task scheduling service to verify whether the test task is valid;

[0072] The task scheduling service looks up the number of test results in the database according to the verification request;

[0073] The task scheduling service determines the size relationship between the number of test results and the number of test cases;

[0074] If the number of test results is less than the number of test cases, the task scheduling service returns information indicating that the test task is valid to the test case running service;

[0075] If the number of test results is equal to the number of test cases, the task scheduling service returns information indicating that the test task is invalid to the test case running service;

[0076] When the test case running service receives information indicating that the test task is valid, it runs the test cases according to the image file of the running environment;

[0077] When the test case running service receives information indicating that the test task is invalid, it stops working.

[0078] Step S4, the test case running service runs the test cases according to the image file of the running environment.

[0079] When the test case running service receives information indicating that the test task is valid, it enters the test case running phase. In this phase, as Figure 4As shown, the use case running service obtains task information and sets up environment variables, such as configuring the startup command of the software under test according to the startup parameters defined in the task information, and configuring the command for how to shut down the software under test in case of an exception according to the shutdown parameters defined in the task information; if a customized script is set in the task information, the use case running service can also run the customized script, and then loop to apply for test cases, automatically run the test cases and send the test results to the result collection service.

[0080] Specifically, in the use case running stage:

[0081] The use case running service runs the test cases according to the image file of the running environment, including:

[0082] The use case running service sends a use case request for applying for test cases to the use case distribution service;

[0083] After the use case running service sends a use case request for applying for test cases to the use case distribution service, the method further includes:

[0084] The use case distribution service filters out several test cases in the database according to the use case request;

[0085] The use case distribution service allocates the filtered test cases to the use case running service and deletes the filtered test cases in the database;

[0086] The use case running service runs the test cases according to the image file of the running environment, further including:

[0087] The use case running service receives the test cases allocated by the use case distribution service;

[0088] The use case running service obtains the image file of the running environment from the image sharing server;

[0089] The use case running service runs the received test cases according to the obtained image file of the running environment.

[0090] Several means one, two or more.

[0091] Among them, the use case running service sends a use case request for applying for test cases to the use case distribution service, including:

[0092] The use case running service determines whether it is the first time to apply for test cases to the use case distribution service;

[0093] If it is not the first application, the use case running service determines the identifier of the most recently received test case and sends the use case request carrying the identifier to the use case running service;

[0094] If it is a first application, the use case running service sends a use case request without any identification of test cases to the use case running service.

[0095] The use case distribution service filters out a number of test cases in the database according to the use case request, including:

[0096] The use case distribution service determines whether the use case request carries the identification of test cases;

[0097] If it carries, the use case distribution service determines a set of test cases associated with the identification in the database, and randomly filters out the number of test cases from the determined set of test cases;

[0098] If it does not carry, the use case distribution service determines whether there is an unassigned set of test cases in the database. If there is, the use case distribution service randomly filters out the number of test cases from any unassigned set of test cases; if not, the use case distribution service randomly filters out the number of test cases from any set of test cases in the database.

[0099] In this embodiment, test cases with an association relationship are classified into a test case set (also called a class or a group). For example, all test cases under a functional module are put into a test case set. The principle for the use case distribution service to allocate test cases to the use case running service is: preferentially allocate test cases within the same test case set to the same use case running service. At this time, the use case running service also needs to provide some help for the above principle. Specifically: if the use case running service applies for test cases from the use case distribution service not for the first time, it sends a use case request carrying the identifier of the most recently received test case to the use case running service. If the use case running service applies for test cases from the use case distribution service for the first time, it sends a use case request without carrying any test case identifier to the use case running service. Further, when the use case running service detects the identifier carried in the use case request, it determines whether there is a test case set associated with the identifier in the database. If there is, it randomly selects several test cases from the determined test case set; if not, the use case distribution service determines whether there is an unallocated test case set in the database. If there is, the use case distribution service randomly selects several test cases from any unallocated test case set; if not, the use case distribution service randomly selects several test cases from any test case set in the database. When the use case running service detects that the use case request does not carry an identifier, it preferentially randomly selects several test cases from any unallocated test case set in the database and allocates them to the use case running service. If there is no unallocated test case set in the database, it randomly selects several test cases from any allocated test case set in the database and allocates them to the use case running service. In addition, to avoid duplicate allocation of test cases, the use case running service can also delete the allocated test cases from the database.

[0100] Further, after receiving the test cases, the use case running service copies the mirror file of the running environment from the mirror sharing server and runs the received test cases under this mirror file, and then sends the test results to the result collection service. To ensure timeliness, it is preferred to synchronously backhaul the test results to the result collection service via an http request; however, considering service connection problems caused by some unstable factors, when there is a problem with the synchronous backhaul, the test results will be sent to the message queue asynchronously to ensure data accuracy.

[0101] Step S5, the result collection service collects the test results of the test cases.

[0102] After the result collection service collects the test results of the test cases, it stores the test results in the database and summarizes and calculates the collected test results in real time.

[0103] The task scheduling service has a monitoring function. When the task scheduling service monitors that the number of test results stored in the database is equal to the number of test cases, it indicates that the current test task has been completed. The task scheduling service notifies the result collection service of the message that the test cases have been run. After receiving the notification, the result collection service calculates the pass rate of all test cases based on the collected test results. The task scheduling service determines whether the pass rate is greater than the pass rate threshold. If it is less, the task scheduling service obtains the rerun rule defined in the task information. The task scheduling service determines whether the pass rate meets the rerun rule. If it meets, the task scheduling service obtains the maximum number of reruns defined in the task information. The task scheduling service determines whether the maximum number of reruns is greater than 0. If it is greater, the task scheduling service parses the task information from the test task. The pass rate threshold is used to determine whether to execute the rerun process. When the pass rate is greater than the pass rate threshold, it is considered that the current test results are available and can be based on the current test results. When the pass rate is less than the pass rate threshold, it is considered that the current test results are unavailable and the test process needs to be executed again. However, it is not necessary to execute the rerun process every time the pass rate is less than the pass rate threshold. In this embodiment, a rerun rule is predefined in advance. Only when the pass rate meets the rerun rule, the rerun process is executed. Specifically, the rerun rule is: whether the pass rate is greater than the rerun rate. If it is greater, it indicates that although the current test results are not accurate enough, they still have certain reference value and the test process can be executed again to improve the accuracy of the test results. Specifically, obtain the maximum number of reruns from the task information, and then loop for the maximum number of reruns until the pass rate meets the pass rate threshold or the number of loops reaches the maximum number of reruns.

[0104] Furthermore, when the pass rate meets the pass rate threshold, or the number of loops reaches the maximum number of reruns, or the maximum number of reruns is not defined and the current test task has been completed, it enters the task end stage. The result collection service calculates the pass rate based on the received test results and generates a test report. The task scheduling service aggregates and persists the intermediate data generated during the task execution (such as the start time and end time of each stage, the pass rate) to the database, and at the same time releases the relevant resources opened during the test process. In addition, the task scheduling service also has some timeout mechanisms and warning rules. The timeout mechanism is used to forcibly end the running process of the test case running service when the test case running service does not return test results for a long time, so that the test case running service feedbacks the test results of the failure of the test case running to the result collection service, thus ensuring the smoothness of the entire process and preventing the entire test task from being affected by certain special exceptions. The warning rule is used to prompt the user whether to continue running when the pass rate is too low, thereby indirectly improving the running efficiency.

[0105] The life cycle of an automated test task is as Figure 5 shown, and the automated test process is as Figure 6As shown in the figure, in Figure 5 and Figure 6 , the operation process is as follows: A. Task startup, such as starting a test task through the Figure 6 management platform; B. Task scheduling, where the task scheduling service receives the test task and parses the task information, and at the same time decomposes the test process into an image making stage, a waiting-to-run stage, a test case running stage, and a task ending stage; C. Image making, that is, the image making stage, where the image making service completes the image file of the running environment and downloads the test cases; D. Waiting to run, that is, the waiting-to-run stage, where the task scheduling service sends the task information to the test case distribution service, and the test case distribution service starts the test case running service according to the task information and the number of test cases; E. Test case running, that is, the test case running stage, where the test case running service applies to the test case distribution service for test cases and runs the applied test cases based on the image file of the running environment; F. Retry, that is, re-execute the test task according to the maximum number of retries and retry rules defined in the task information; G. Task ending, that is, the task ending stage, where the result collection service performs some data cleaning work on the test results according to different test types, so as to facilitate the subsequent generation of different test reports, and sends the test reports to the testers defined in the task information by email.

[0106] For example, Figure 7 as shown in the figure, after the user starts a task through the management platform, the running platform (i.e., the distributed system) executes the test task and displays the data generated by the test in real time, such as displaying the running time axis, the start time of each stage, the passing rate, the total number of test cases (the number of test cases), the total number of returns (the number of test results), the number of successes (the number of test cases that passed the test), the number of failures (the number of test cases that failed the test), etc.

[0107] For example, Figure 8 as shown in the figure, each service occupies a node, that is, a server, where: the software to be tested in the plate-making server is stored in the platform version storage server, the test task is started through the cloud test platform entry service, and the test process is executed based on the cloud test running platform (i.e., the distributed system). Specifically, the task scheduling server receives the test task, schedules resources from the resource scheduling server, and constructs a test environment. The image making server downloads and installs the software to be tested from the platform version storage server to make the image file of the running environment, and downloads the test cases from the test script server. The test case distribution service starts the test case running server, and the test case running server applies to the test case distribution service for test cases. The distributed system also includes a result collection server, which generates a test report (i.e., the task running report).

[0108] Embodiment 2

[0109] Embodiment 2 of the present invention also provides a distributed system for automated testing. This distributed system for automated testing corresponds to the method provided in Embodiment 1 above. The corresponding technical features and technical effects will not be elaborated in detail in this embodiment, and the relevant parts can refer to Embodiment 1 above. Specifically, Figure 9 shows a block diagram of the distributed system for automated testing provided in Embodiment 2 of the present invention. As Figure 9 shown, the distributed system 900 may include a task scheduling service 901, an image making service 902, a test case distribution service 903, a test case running service 904, and a result collection service 905, where:

[0110] The task scheduling service 901 is used to receive a test task and parse task information from the test task;

[0111] The image making service 902 is used to configure a running environment required for testing according to the task information, generate an image file of the running environment, and download test cases to be tested;

[0112] The test case distribution service 903 is used to start the test case running service 904 according to the task information and the number of test cases;

[0113] The test case running service 904 is used to run the test cases according to the image file of the running environment;

[0114] The result collection service 905 is used to collect test results of the test cases.

[0115] Optionally, when the test case distribution service starts the test case running service according to the task information and the number of test cases, it is further used to: determine the number of test cases downloaded by the image making service; judge whether the number of the test case running service is defined in the task information, and obtain a judgment result; determine the final number of the test case running service according to the number of test cases and the judgment result; start the test case running service with the final number.

[0116] Optionally, when the test case distribution service determines the final number of the test case running service according to the number of test cases and the judgment result, it is further used to: when the judgment result indicates that the number of the test case running service is not defined, obtain a first calculation rule associated with the judgment result; calculate the number of the test case running service according to the first calculation rule and the number of test cases; directly use the calculated number as the final number of the test case running service.

[0117] Optionally, when determining the final number of use case running services according to the number of test cases and the judgment result, the use case distribution service is further configured to: when the judgment result represents that the number of use case running services is defined, obtain a second calculation rule associated with the judgment result; calculate the number of use case running services according to the second calculation rule and the number of test cases; determine the final number of use case running services according to the defined number and the calculated number.

[0118] Optionally, when calculating the number of use case running services according to the second calculation rule and the number of test cases, the use case distribution service is further configured to: obtain the test case value range defined in the second calculation rule; screen out the test case value range having a mapping relationship with the number of test cases from the obtained test case value range; determine the sub-rule associated with the screened test case value range in the second calculation rule; calculate the number of use case running services according to the determined sub-rule.

[0119] Optionally, when starting the use case running services with the final number, the use case distribution service is further configured to: obtain the configuration information of the use case running services defined in the task information; start the use case running services with the final number according to the configuration information.

[0120] Optionally, before the use case distribution service starts the use case running services according to the task information and the number of test cases, the image production service is further configured to: store the image file of the running environment in an image sharing server; the image production service stores the downloaded test cases in a preset database; when running the test cases according to the image file of the running environment, the use case running service is further configured to: send a use case request for applying for test cases to the use case distribution service; after the use case running service sends a use case request for applying for test cases to the use case distribution service, the use case distribution service is further configured to: screen out several test cases from the database according to the use case request; allocate the screened test cases to the use case running service and delete the screened test cases from the database; when running the test cases according to the image file of the running environment, the use case running service is further configured to: receive the test cases allocated by the use case distribution service; obtain the image file of the running environment from the image sharing server; run the received test cases according to the obtained image file of the running environment.

[0121] Optionally, when the use case running service sends a use case request for applying for test cases to the use case distribution service, it is further used to: determine whether it is the first time to apply for test cases to the use case distribution service; if it is not the first application, determine the identifier of the most recently received test case, and send the use case request carrying the identifier to the use case running service; if it is the first application, send the use case request without carrying any test case identifier to the use case running service.

[0122] Optionally, when the use case distribution service filters out several test cases from the database according to the use case request, it is further used to: determine whether the use case request carries the identifier of a test case; if it carries, determine the set of test cases associated with the identifier in the database, and randomly filter out the several test cases from the determined set of test cases; if it does not carry, determine whether there is an unallocated set of test cases in the database, if there is, randomly filter out the several test cases from any unallocated set of test cases; if not, randomly filter out the several test cases from any set of test cases in the database.

[0123] Optionally, after the result collection service collects the test results of the test cases: the result collection service is further used to store the collected test results in the database; the task scheduling service is further used to notify the result collection service of the message that the test cases have been run when it monitors that the number of test results stored in the database is equal to the number of test cases; after receiving the notification, the result collection service calculates the passing rate of all test cases according to the collected test results; the task scheduling service determines whether the passing rate is greater than the passing rate threshold; if it is less, the task scheduling service obtains the rerun rule defined in the task information; the task scheduling service determines whether the passing rate meets the rerun rule; if it meets, the task scheduling service obtains the maximum number of reruns defined in the task information; the task scheduling service determines whether the maximum number of reruns is greater than 0; if it is greater, the task scheduling service parses the task information from the test task.

[0124] Optionally, the task scheduling service, the image building service, the use case distribution service, the use case running service, and the result collection service are different virtual machines or Docker containers.

[0125] Embodiment III

[0126] Figure 10The block diagram of the computer device for controlling the distributed system to execute the automated test method provided in the third embodiment of the present invention is shown. In this embodiment, the computer device 1000 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including an independent server or a server cluster composed of multiple servers) that executes a program, etc. As Figure 10 shown, the computer device 1000 of this embodiment at least includes, but is not limited to: a memory 1001 control, a processor 1002, and a network interface 1003 that can communicate with each other through the system bus. It should be noted that Figure 10 only the computer device 1000 with components 1001-1003 is shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0127] In this embodiment, the memory 1003 includes at least one type of computer-readable storage medium. The readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 1001 may be an internal storage unit of the computer device 1000, such as the hard disk or memory of the computer device 1000. In other embodiments, the memory 1001 may also be an external storage device of the computer device 1000, such as a plug-in hard disk equipped on the computer device 1000, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Of course, the memory 1001 may also include both the internal storage unit and the external storage device of the computer device 1000. In this embodiment, the memory 1001 is generally used to store the operating system and various application software installed on the computer device 1000, such as program codes for controlling the distributed system to execute the automated test method. The program codes include: sending test tasks. The distributed system includes a task scheduling service, an image making service, a use case distribution service, a use case running service, and a result collection service. The automated test method includes: the task scheduling service receives a test task and parses task information from the test task; the image making service configures the running environment required for the test according to the task information, generates an image file of the running environment, and downloads the test cases to be tested; the use case distribution service starts the use case running service according to the task information and the number of test cases; the use case running service runs the test cases according to the image file of the running environment; the result collection service collects the test results of the test cases.

[0128] In some embodiments, the processor 1002 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 1002 is generally used to control the overall operation of the computer device 1000. For example, it performs control and processing related to data interaction or communication with the computer device 1000. In this embodiment, the processor 1002 is used to run the program codes stored in the memory 1001 for controlling the steps of the automated test method executed by the distributed system.

[0129] In this embodiment, the steps for controlling the distributed system to execute the automated test method stored in the memory 1001 can also be divided into one or more program modules and executed by one or more processors (processor 1002 in this embodiment) to complete the present invention.

[0130] The network interface 1003 can include a wireless network interface or a wired network interface. This network interface 1003 is generally used to establish a communication link between the computer device 1000 and other computer devices. For example, the network interface 1003 is used to connect the computer device 1000 to an external terminal through a network, and establish a data transmission channel and a communication link between the computer device 1000 and the external terminal, etc. The network can be an enterprise internal network (Intranet), the Internet, the Global System of Mobile communication (abbreviated as GSM), Wideband Code Division Multiple Access (abbreviated as WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi and other wireless or wired networks.

[0131] Embodiment 4

[0132] This embodiment also provides a computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disc, server, App application mall, etc., on which a computer program is stored, and the computer program is executed by a processor to control the distributed system to execute the program code of the automated test method.

[0133] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the embodiments of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0134] It should be noted that the serial numbers of the embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0135] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0136] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied to other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An automated testing method for a distributed system, characterized in that, The distributed system includes a task scheduling service, an image building service, a use case distribution service, a use case running service, and a result collection service. The method includes: The task scheduling service receives a test task and parses task information from the test task; The image building service configures a running environment required for testing according to the task information, generates an image file of the running environment, and downloads test cases to be tested; The use case distribution service starts the use case running service according to the task information and the number of test cases; The use case running service sends a verification request to the task scheduling service to verify whether the test task is valid; The task scheduling service looks up the number of test results in the database according to the verification request and determines the size relationship between the number of test results and the number of test cases; If the number of test results is less than the number of test cases, the task scheduling service returns information indicating that the test task is valid to the use case running service; when the use case running service receives the information indicating that the test task is valid, it runs the test cases according to the image file of the running environment; If the number of test results is equal to the number of test cases, the task scheduling service returns information indicating that the test task is invalid to the use case running service; when the use case running service receives the information indicating that the test task is invalid, it stops working; The result collection service collects the test results of the test cases.

2. The method according to claim 1, wherein The use case distribution service starts the use case running service according to the task information and the number of test cases, including: The use case distribution service determines the number of test cases downloaded by the image building service; The use case distribution service determines whether the number of the use case running service is defined in the task information and obtains a judgment result; The use case distribution service determines the final number of the use case running service according to the number of test cases and the judgment result; The use case distribution service starts the use case running service with the final number.

3. The method according to claim 2, wherein The use case distribution service determines the final number of the use case running service according to the number of test cases and the judgment result, including: When the judgment result indicates that the number of the use case running service is not defined, the use case distribution service obtains a first calculation rule associated with the judgment result; The use case distribution service calculates the number of the use case running service according to the first calculation rule and the number of test cases; The use case distribution service directly uses the calculated number as the final number of the use case running service.

4. The method according to claim 2, wherein The use case distribution service determines the final number of the use case running service according to the number of test cases and the judgment result, including: When the judgment result indicates that the number of the use case running service is defined, the use case distribution service obtains a second calculation rule associated with the judgment result; The use case distribution service calculates the number of the use case running service according to the second calculation rule and the number of test cases; The use case distribution service determines the final quantity of the use case running service according to the defined quantity and the calculated quantity.

5. The method according to claim 4, characterized in that The use case distribution service calculates the quantity of the use case running service according to the second calculation rule and the quantity of the test cases, including: The use case distribution service obtains the test case value range defined in the second calculation rule; The use case distribution service filters out the test case value range having a mapping relationship with the quantity of the test cases from the obtained test case value range; The use case distribution service determines the sub-rule associated with the filtered test case value range in the second calculation rule; The use case distribution service calculates the quantity of the use case running service according to the determined sub-rule.

6. The method according to claim 2, wherein The use case distribution service starts the use case running service of the final quantity, including: The use case distribution service obtains the configuration information of the use case running service defined in the task information; The use case distribution service starts the use case running service of the final quantity according to the configuration information.

7. The method according to claim 1, wherein: Before the use case distribution service starts the use case running service according to the task information and the quantity of the test cases, the method further includes: The image production service stores the image file of the running environment to the image sharing server; The image production service stores the downloaded test cases to a preset database; The use case running service runs the test cases according to the image file of the running environment, including: The use case running service sends a use case request for applying for test cases to the use case distribution service; After the use case running service sends a use case request for applying for test cases to the use case distribution service, the method further includes: The use case distribution service filters out a number of test cases from the database according to the use case request; The use case distribution service allocates the filtered test cases to the use case running service and deletes the filtered test cases from the database; The use case running service runs the test cases according to the image file of the running environment, further including: The use case running service receives the test cases allocated by the use case distribution service; The use case running service obtains the image file of the running environment from the image sharing server; The use case running service runs the received test cases according to the obtained image file of the running environment.

8. The method according to claim 7, wherein The use case running service sends a use case request for applying for test cases to the use case distribution service, including: The use case running service determines whether it is the first time to apply for test cases to the use case distribution service; If it is not the first application, the use case running service determines the identifier of the test case received most recently and sends the use case request carrying the identifier to the use case running service; If it is the first application, the use case running service sends the use case request without carrying any test case identifier to the use case running service.

9. The method according to claim 8, wherein The use case distribution service filters out a number of test cases from the database according to the use case request, including: The use case distribution service determines whether the use case request carries the identifier of the test case; If it does, the use case distribution service determines a set of test cases associated with the identifier in the database, and randomly selects the several test cases from the determined set of test cases; If it does not, the use case distribution service determines whether there is an undistributed set of test cases in the database. If there is, the use case distribution service randomly selects the several test cases from any undistributed set of test cases; if not, the use case distribution service randomly selects the several test cases from any set of test cases in the database.

10. The method according to claim 1, characterized in that, After the result collection service collects the test results of the test cases, the method further includes: The result collection service stores the collected test results in the database; When the task scheduling service monitors that the number of test results stored in the database is equal to the number of test cases, it notifies the result collection service of the message that the test cases have been run; After receiving the notification, the result collection service calculates the pass rate of all test cases based on the collected test results; The task scheduling service determines whether the pass rate is greater than the pass rate threshold; If it is less, the task scheduling service obtains the rerun rule defined in the task information; The task scheduling service determines whether the pass rate meets the rerun rule; If it meets, the task scheduling service obtains the maximum number of reruns defined in the task information; The task scheduling service determines whether the maximum number of reruns is greater than 0; If it is greater, the task scheduling service parses the task information from the test task.

11. The system according to claim 1, wherein, The task scheduling service, the image production service, the use case distribution service, the use case running service, and the result collection service are different virtual machines or Docker containers.

12. A distributed system for automated testing of the method according to any one of claims 1 to 11, characterized in that, The distributed system includes a task scheduling service, an image production service, a use case distribution service, a use case running service, and a result collection service, where: The task scheduling service is used to receive a test task and parse the task information from the test task; The image production service is used to configure the running environment required for testing according to the task information, generate an image file of the running environment, and download the test cases to be tested; The use case distribution service is used to start the use case running service according to the task information and the number of test cases; The use case running service is used to run the test cases according to the image file of the running environment; The result collection service is used to collect the test results of the test cases.

Images