Application test method, device and equipment, readable storage medium and program product

Abstract

The embodiment of the invention provides an application test method and device, equipment, a readable storage medium and a program product, and the method comprises the steps that a client selects a plurality of test cloud real machines from a plurality of cloud real machines, and stores an obtained test instruction, test configuration data and a reference program interface image into a database; the test cloud real machine obtains test data including a test code and a reference program interface image according to the related data in the database; running the to-be-tested application program according to the test code; acquiring a test interface image; determining a test result of the to-be-tested application program according to the test interface image and the reference program interface image, and storing the test result in a database; and the client determines a target test result according to the test result of the to-be-tested application returned by each test cloud real machine acquired from the database. According to the method provided by the embodiment of the invention, the application program can be tested by using the cloud real machine to obtain the test result, so that the application test efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an application testing method, an application testing device, a computer equipment, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of Internet technology, mobile devices (such as mobile phones) are becoming more and more powerful, and there are more and more applications in mobile devices. Multi-object interactive applications can allow multiple objects to participate in a scene in the application at the same time (for example, some online mobile games can allow multiple users to participate at the same time). After the development of multi-object interactive applications is completed, application testing is usually required.

[0003] When testing multi-object interaction scenarios of an application, a corresponding number of testers and mobile devices are usually required to complete the test task. This testing method requires a great deal of manpower and material resources, and the testing efficiency is very low. Therefore, finding an efficient application testing method is an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide an application testing method, apparatus, device, readable storage medium, and program product, which can use a cloud real machine to test the application program to obtain test results, effectively improving the efficiency of application testing.

[0005] On the one hand, an embodiment of the present application provides an application testing method, the method comprising:

[0006] Acquire test data for the application to be tested, wherein the test data includes test code and a reference program interface image;

[0007] Running the application to be tested according to the test code, wherein, during the running of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code;

[0008] Acquire a test interface image, wherein the test interface image is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code;

[0009] A matching process is performed based on the test interface image and the reference program interface image, and a test result of the application program to be tested is determined based on the matching result.

[0010] Accordingly, the present application embodiment provides another application testing method, the method comprising:

[0011] Acquire cloud machine status information of multiple cloud machines, and select multiple test cloud machines from the multiple cloud machines according to the cloud machine status information of the multiple cloud machines, wherein the cloud machine status information is determined according to the running status of the application to be tested in the cloud machine;

[0012] Acquire test instructions, test configuration data, and a reference program interface image, and store the test instructions, the test configuration data, and the reference program interface image in an instruction database, so that each of the test cloud real machines determines a test code according to the test instructions and the test configuration data, and runs the application to be tested according to the test code;

[0013] According to the test results of the application to be tested returned by each of the test cloud real machines, the target test result of the application to be tested is determined, and the test result is determined by the test cloud real machine based on the matching result of the test interface image and the reference program interface. The test interface image is obtained by the test cloud real machine taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code.

[0014] On the one hand, an embodiment of the present application provides an application testing device, the device comprising:

[0015] An acquisition unit, used to acquire test data for the application to be tested, wherein the test data includes a test code and a reference program interface image;

[0016] A processing unit, configured to run the application to be tested according to the test code, wherein, during the running of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code;

[0017] The acquisition unit is further used to acquire a test interface image, wherein the test interface image is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code;

[0018] A matching unit is used to perform matching processing according to the test interface image and the reference program interface image, and determine the test result of the application to be tested according to the matching result.

[0019] Accordingly, an embodiment of the present application provides another application testing device, the device comprising:

[0020] A selection unit, configured to obtain cloud real machine status information of a plurality of cloud real machines, and select a plurality of test cloud real machines from the plurality of cloud real machines according to the cloud real machine status information of the plurality of cloud real machines, wherein the cloud real machine status information is determined according to the running status of the application to be tested in the cloud real machine;

[0021] A storage unit, used to obtain test instructions, test configuration data and reference program interface images, and store the test instructions, the test configuration data and the reference program interface images in an instruction database, so that each of the test cloud real machines determines the test code according to the test instructions and the test configuration data, and runs the application to be tested according to the test code;

[0022] A determination unit is used to determine the target test result of the application to be tested based on the test results of the application to be tested returned by each of the test cloud real machines, wherein the test result is determined by the test cloud real machine based on the matching result of the test interface image and the reference program interface, wherein the test interface image is obtained by taking a screenshot of the program interface of the application to be tested by the test cloud real machine when the target code is executed, and the target code is included in the test code.

[0023] On the one hand, an embodiment of the present application provides a computer device, comprising: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores executable program code, and the processor is used to call the executable program code to implement the application testing method provided in the embodiment of the present application.

[0024] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer implements the application testing method provided by the embodiment of the present application.

[0025] Accordingly, an embodiment of the present application further provides a computer program product, the computer program product comprising a computer program or computer instructions, the computer program or computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device implements the application testing method provided in the embodiment of the present application.

[0026] In the present application, the client can select multiple test cloud real machines from multiple cloud real machines based on the cloud real machine status information of multiple cloud real machines obtained from the database, and store the obtained test instructions, test configuration data and reference program interface images in the instruction database; the test cloud real machine can obtain test data including test code and reference program interface images, and run the application to be tested according to the test code; the test cloud real machine can obtain a test interface image obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and determine the test result based on the test interface image and the reference program interface image; the client can determine the target test result of the application to be tested based on the test results returned by each test cloud real machine. Through the application testing method provided in the embodiment of the present application, multiple test cloud real machines can be used to test the same application to be tested, thereby realizing the testing of multi-object interaction scenarios of the application and effectively reducing the resources consumed by the application testing; the test cloud real machine can be used to execute the test code, and the test interface image and the reference program interface image can be matched to obtain the test result, thereby effectively improving the testing efficiency of the application; the test cloud real machine can be determined based on the cloud real machine status information, so that the same cloud real machine can be flexibly called by multiple clients, which is conducive to improving the resource utilization of the cloud real machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 is a schematic diagram of the system architecture of an application testing system provided in an embodiment of the present application;

[0029] Figure 2 It is a flowchart of an application testing method provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of a test code determination method provided in an embodiment of the present application is shown;

[0031] Figure 4 It is a flowchart of another application testing method provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of a method for determining a test cloud real machine provided in an embodiment of the present application is shown;

[0033] Figure 6is a schematic diagram of a data acquisition method provided in an embodiment of the present application;

[0034] Figure 7 It is a flowchart of another application testing method provided in an embodiment of the present application;

[0035] Figure 8 is a system architecture diagram of another application testing system provided in an embodiment of the present application;

[0036] Fig. 9 is a schematic diagram of an application testing method provided in an embodiment of the present application;

[0037] Fig.10 is a structural block diagram of an application testing device provided in an embodiment of the present application;

[0038] Fig.11 is a structural block diagram of another application testing device provided in an embodiment of the present application;

[0039] Fig.12 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] It should be noted that the descriptions of "first", "second", etc. involved in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0042] At present, the types and number of mobile applications are increasing. Some mobile applications can provide multi-object interaction scenarios. For example, a mobile game program can allow 32 game objects to participate in the same game level at the same time. When testing multi-object interaction scenarios, a corresponding number of testers and test equipment are usually required to complete the application testing, which greatly wastes resources and the application testing efficiency is extremely low.

[0043] Based on this, the embodiment of the present application provides an application testing method, in which the client can select multiple test cloud real machines from multiple cloud real machines according to the cloud real machine status information of multiple cloud real machines, and store the obtained test instructions, test configuration data and reference program interface images in the instruction database; the test cloud real machine can obtain test data including test code and reference program interface images, and run the application to be tested according to the test code; the test cloud real machine can obtain the test interface image obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and determine the test result according to the test interface image and the reference program interface image; the client can determine the target test result of the application to be tested according to the test results returned by each test cloud real machine. Through the method provided by the embodiment of the present application, multiple test cloud real machines can be used to test the same application to be tested, realizing the testing of multi-object interaction scenarios of the application, effectively reducing the resources consumed by the application test, and effectively improving the testing efficiency of the application.

[0044] The application testing method provided by the embodiment of the present application can be applied to the field of cloud storage. Cloud storage (cloudstorage), that is, a distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of storage devices of various types (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide data storage and business access functions to the outside world. In the application testing method provided by the embodiment of the present application, cloud storage devices can be used to store instructions, configuration data, reference program interface images and other data required for application testing. The client can store test instructions, test configuration data and reference program interface images in the cloud storage device, and the cloud real machine then obtains test instructions, test configuration data and reference program interface images from the cloud storage device, and generates test code; runs the application to be tested according to the test code, and determines the test results of the application to be tested according to the test interface image and the reference program interface image; the test cloud real machine stores the test results in the cloud storage device, and the client obtains the test results of each test cloud real machine from the cloud storage device. Through the method provided by the embodiment of the present application, the test efficiency of multi-object interaction scenarios can be greatly improved and resource consumption can be reduced.

[0045] The architecture of the application testing system provided in the embodiment of the present application will be introduced below in conjunction with the accompanying drawings.

[0046] See also Figure 1 , which is a schematic diagram of the system architecture of an application testing system provided in an embodiment of the present application, wherein the application testing system includes a client 101, a database 102, and multiple test cloud real machines 103 (such as Figure 1The client 101 and multiple test cloud real machines 103 can access the database 102. Among them:

[0047] The client 101 can interact with the object, receive external input data, store relevant data in the database 102, and obtain data from the database 102. The client 101 can be a handheld device with communication function (such as a smart phone, a tablet computer), a computing device (such as a personal computer (PC), an intelligent voice interaction device, a wearable device or other intelligent device, but is not limited thereto.

[0048] Database 102 is used to store data related to application testing. Database 102 can be a physical database or a cloud database (i.e., a database deployed in the cloud). Specifically, it can be deployed based on any of private cloud, public cloud, hybrid cloud, edge cloud, etc., so that the cloud database focuses on different functions. For example, for a database deployed in a private cloud, the basic cloud hardware is the user's personal equipment, and it focuses more on serving a small number of users. The database deployed in the public cloud is deployed based on a cloud platform provided by a third party, which allows the data stored in the database to achieve data sharing. Any user's data can be stored in the database, and any user can also use the data in the database.

[0049] The test cloud real machine 103 is a mobile device that can perform related tasks according to remote instructions. A real physical device is connected to the Internet through cloud technology, and the physical device is called a cloud real machine. The user can operate the cloud real machine in the cloud. The test cloud real machine 103 can obtain the test data of the application from the database 102. The device models of the test cloud real machine 1031, the test cloud real machine 1032 and the test cloud real machine 1033 can be the same or different.

[0050] The following will explain in detail Figure 1 The application test system shown works as follows:

[0051] The client 101 can obtain the cloud real machine status information of multiple cloud real machines from the database 102, and select multiple test cloud real machines from the multiple cloud real machines according to the cloud real machine status information of the multiple cloud real machines (assuming that the selected test cloud real machines are test cloud real machine 1031, test cloud real machine 1032 and test cloud real machine 1033); the client 101 can interact with the use object and obtain test instructions, test configuration data and reference program interface images. The client 101 stores the test-related data (i.e., test instructions, test configuration data and reference program interface images) in the database 102 so that the test cloud real machine can generate test code according to the test instructions and test configuration data. Test cloud real machine 1031, test cloud real machine 1032 and test cloud real machine 1033 respectively obtain test related data related to the application to be tested from database 102, and determine test data according to the test related data, and the test data includes test code and reference program interface image; each test cloud real machine runs the application to be tested according to the test code, and obtains the test interface image, which is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code; each test cloud real machine can determine the test result according to the test interface image and the reference program interface image, and store the test result in database 102. Client 101 can obtain the test results returned by each test cloud real machine from database 102, and determine the target test result of the application to be tested according to the test result. Through the application testing method provided in the embodiment of the present application, the cloud real machine can be used to implement the test of multi-object interaction scenarios, effectively improving the efficiency of application testing and reducing resource consumption.

[0052] It is to be understood that the schematic diagram of the application testing system described in the embodiment of the present application is to more clearly illustrate the application testing method of the embodiment of the present application, and does not constitute a limitation on the application testing method provided in the embodiment of the present application. Figure 1 The number of clients 101 and test cloud real machines 103 is only illustrative. Any number of devices and nodes can be configured according to business implementation needs. In addition, with the evolution of system architecture and the emergence of new business scenarios, the application testing method provided in the embodiment of the present application is also applicable to similar technical problems.

[0053] See also Figure 2 , Figure 2 A flowchart of an application testing method provided in an embodiment of the present application. The application testing method can be implemented by the test cloud real machine 103 in the above-mentioned application testing system. The process of the application testing method provided in the embodiment of the present application includes but is not limited to:

[0054] S201: Acquire test data for an application to be tested, wherein the test data includes test code and a reference program interface image.

[0055] In an embodiment of the present application, the test cloud real machine is a physical device that can be remotely controlled and relies on network communication technology to achieve real-time operation. The test cloud real machine can obtain test data for the application to be tested, and the test data may include test code and reference program interface images. The test code is used to implement the processing corresponding to the test instructions of the application to be tested. The reference program interface image is the expected program interface image, which is used to determine whether the application test is completed normally. The reference program interface image can have a corresponding relationship with the code in the test code. For example: the operation indicated by a certain section of code in the test code is to open the setting interface of the application to be tested. The reference program interface image may include the setting interface image of the application to be tested, so that the cloud real machine can determine whether the test code is executed normally.

[0056] In one embodiment, the implementation method of obtaining test data for the application to be tested can be: obtaining test instructions, test configuration data and reference program interface images for the application to be tested, the reference program interface image has a corresponding relationship with the test instructions, and the test configuration data includes the authentication data required to run the application to be tested, the running data of the application to be tested and the program interface matching data; generating test code according to the test instructions and the test configuration data; using the test code and the reference program interface image as test data for the application to be tested. The test cloud real machine can obtain test instructions, test configuration data and reference program interface images for the application to be tested from the database. The test instructions may include multiple instructions, such as: cmd instructions, level instructions, game instructions, etc. The test instructions correspond to the test configuration data, for example: for cmd instructions, its test configuration data may include additional parameters of the cmd instruction and instruction type data (for example: single-click instructions, double-click instructions, sliding instructions, etc.). The test configuration data may include the authentication data required to run the application to be tested, the running data of the application to be tested and the program interface matching data. The authentication data required to run the application to be tested can enable the cloud real machine system to have the right to access the system. For example, if the application to be tested is a game application, the authentication data required to run the application to be tested can be the game account and account password. The running data of the application to be tested is used to limit the test process of the application to be tested. For example, the running data of the application to be tested can be the time limit of the judgment logic. If the cloud real machine does not complete the judgment search within the time limit of the judgment logic, the test result can be determined to be abnormal. The program interface matching data is used to indicate the judgment conditions in the test process, for example, judgment search based on text recognition, image or control information. When the test cloud real machine performs application testing, it can use the authentication data required by the application to be tested to obtain the access rights of the application, and perform judgment search in the program interface according to the program interface matching data. After finding the specified control, image, text, etc., the test instruction can be executed. The reference program interface image has a corresponding relationship with the test instruction, and one test instruction can correspond to one reference program interface image. The test cloud real machine can generate test code according to the test instruction and test configuration data, and use the test code and the reference program interface image as test data for the application to be tested. Through the method provided in the embodiment of the present application, test code can be generated according to the acquired instructions and configuration data, which can effectively improve the efficiency of application testing.

[0057] In one embodiment, the test cloud real machine can periodically obtain test instructions, test configuration data and reference program interface images for the application to be tested from the database. For example, the test cloud real machine can obtain test instructions, test configuration data and reference program interface images for the application to be tested from the database every 1 second.

[0058] In one embodiment, the implementation method of generating test code according to test instructions and test configuration data can be: performing code search processing from a code database according to the test instructions to obtain a search result; if the search result indicates that there is code matching the test instruction, the code matching the test instruction and the test configuration data are combined to obtain the test code; if the search result indicates that there is no code matching the test instruction, an instruction implementation code is generated according to the test instruction, and the instruction implementation code is combined with the test configuration data to obtain the test code. After the test cloud real machine obtains the test instruction, it needs to convert the instruction into the corresponding processing logic code. For example: the application to be tested is a game program, and the test instruction is to open the first level. Then the operation process indicated by the processing logic code corresponding to the test instruction can be to open the game program, log in to the game account, enter the game level interface, and select the first game level. Therefore, according to the test instructions, code search processing can be performed from the code database of the cloud real machine to obtain search results; if the search result indicates that there is code that matches the test instruction, the code that matches the test code and the test configuration data are combined to obtain the test code. That is, when there is processing logic code corresponding to the test instruction in the cloud real machine, the test code required for this test can be generated according to the existing code and the test configuration data, without the need to write additional automation code. If the search result indicates that there is no code that matches the test instruction, instruction implementation code can be generated according to the test instruction, and the instruction implementation code and the test configuration data are combined to obtain the test code. That is, for some new test instructions or scenarios, when there is no corresponding processing logic code in the cloud real machine, the method provided in this application can automatically generate processing logic code according to the behavioral events (i.e., test configuration data) and test instructions configured by the user in the eXtensible Markup Language (XML) file, so that the test cloud real machine can complete the corresponding test operations. Please refer to Figure 3 , which shows a schematic diagram of a test code determination method provided in an embodiment of the present application. Figure 3The first line "self.run_machine_en gin(iter)" indicates that the cloud real machine engine obtains test related data (such as test instructions, test configuration data, etc.) from the database; the second line "time.sleep(1)" indicates that the time interval is 1 second; the third line is "Logger.debug('TestCase:run_machine_engin once time')"; the fourth line is "cmd=webcmd['cmd_id']"; the fifth line is "para=webmc['cmd_para']"; the third, fourth and fifth lines indicate that the test cloud real machine determines the test code according to the test instructions and test configuration information. Through the method provided in the embodiment of the present application, the cloud real machine can automatically generate test code according to the test instructions and test configuration data, without the need to write complex code, effectively improving the test efficiency of the application.

[0059] In one embodiment, the cloud real machine may also include special codes for handling some abnormal situations. For example: the application to be tested is a game program, and the processing process indicated by the test code in the test cloud real machine is to open the game program, log in to the game account, enter the game character selection interface, and select the first game character; however, when the game program is running, after logging in to the game account, various announcements, game tasks, game prompts and other pop-up windows will pop up in the program interface. If these pop-up windows cannot be closed, the game character selection interface cannot be entered. Therefore, special codes can be used to handle this abnormal situation, close irrelevant pop-up windows, and allow the test of the application to be tested to proceed normally. Through the method provided in the embodiment of the present application, the testing of the application can be efficiently and smoothly implemented, and the efficiency of application testing can be improved.

[0060] S202: Run the application to be tested according to the test code, wherein, during the running of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code.

[0061] In an embodiment of the present application, the test cloud real machine can run the application to be tested according to the test code, and during the operation of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code. For example: the application to be tested is a game program, and the operation indicated by the test code is to enter the program main interface, enter the game match selection interface, and select a multiplayer game match. Then, the interface display content of the game program changes accordingly according to the operation indicated by the test code (from the game main interface to the game match selection interface, and then to the multiplayer game match startup interface). The method provided by the embodiment of the present application can use the cloud real machine to remotely implement the testing of the application, which effectively improves the efficiency of the application testing, and at the same time, it can also effectively reduce the resources consumed by the application testing.

[0062] In one embodiment, the running status of the application to be tested can be obtained, and the cloud real machine status information can be determined according to the running status of the application to be tested; the cloud real machine status information is stored in the status database, and the cloud real machine status information is used to indicate whether the test cloud real machine is in a callable state. The test cloud real machine can obtain the running status of the application to be tested, and determine the cloud real machine status information according to the running status of the application to be tested. When the display interface of the application to be tested in the cloud real machine changes, it will be in a new state, and the running status of the application to be tested at this time can be obtained, and the cloud real machine status information can be determined. For example: when the application to be tested is a game program, the cloud real machine status information can be mainly divided into two categories: in-game status information and out-of-game status information, which can specifically include unstarted state, login state, novice stage state, lobby idle state, team state, matching state, running state, room state, etc. The cloud real machine status information can be used to indicate whether the test cloud real machine is in a callable state. After determining the cloud real machine status information, the test cloud real machine can store the cloud real machine status information in the status database, so that the client can determine the test cloud real machine that needs to be called according to the cloud real machine status information in the status database, thereby enabling multiple clients to use the same batch of cloud real machines at the same time, effectively improving the resource utilization of the cloud real machines.

[0063] S203, obtaining a test interface image, wherein the test interface image is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code.

[0064] In the embodiment of the present application, the test cloud real machine can obtain the test interface image, which is obtained by taking a screenshot of the program interface of the application to be tested after the target code is executed, and the target code is included in the test code. The cloud real machine can take multiple screenshots of the program interface of the application to be tested, and obtain multiple test interface images. For example, according to the operation indicated by the test code, the display interface of the application to be tested changes from the application main interface to the application setting interface, and then to the application interface version information interface. The cloud real machine can take a screenshot when the display interface of the application to be tested changes, and obtain multiple test interface images. The target code can be a part of the code in the test code, the target code can indicate a certain operation, and there can be a corresponding relationship between the target code and the test interface image. For example, the target code can indicate the start of the application to be tested, so after the target code is executed, the display interface of the application to be tested should be the application main interface. Through the method provided in the embodiment of the present application, the cloud real machine can be used to obtain the test interface image, which is convenient for determining the test result of the application to be tested according to the test interface image later.

[0065] S204: Perform matching processing based on the test interface image and the reference program interface image, and determine the test result of the application to be tested based on the matching result.

[0066] In an embodiment of the present application, the test cloud real machine can perform matching processing based on the test interface image and the reference program interface image to obtain a matching result. There is a corresponding relationship between the test interface image and the reference program interface image. For example, if the operation indicated by a certain part of the code in the test code is to enter a certain specified level, then the reference program interface image includes the reference program interface image of the specified level interface. When the cloud real machine runs the application to be tested according to the test code, the part of the code is executed, and the display interface of the application to be tested is the interface of the specified level. The cloud real machine takes a screenshot of the interface display content of the application to be tested and obtains the test interface image of the specified level interface. The reference program interface image of the specified level interface corresponds to the test interface image of the specified level interface. Matching processing is performed based on the corresponding test interface image and the reference program interface image to obtain a matching result, and the test result of the application to be tested can be determined based on the matching result.

[0067] In one embodiment, matching processing is performed on the test interface image and the reference program interface image, and the implementation method of determining the test result of the application to be tested based on the matching result can be: performing feature extraction processing on the test interface image to obtain test interface image features; performing feature extraction processing on the reference program interface image to obtain reference program interface image features; performing feature matching processing based on the test interface image features and the reference program interface image features to obtain image feature similarity; determining the test result of the application to be tested based on the image feature similarity, and the test result is used to indicate whether the application to be tested is running normally. After acquiring the test interface image, the cloud real machine can perform feature extraction processing on the test interface image to obtain test interface image features; can perform feature extraction processing on the reference program interface image to obtain reference program interface image features; performing feature matching processing based on the test interface image features and the reference program interface image features to obtain image feature similarity, and determining the test result of the application to be tested based on the image similarity, and the test result can be used to indicate whether the application to be tested is running normally.

[0068] It should be noted that the embodiment of the present application exemplarily provides a method for determining the test results of the application to be tested based on the test interface image and the reference program interface image. In the actual application process, in addition to using images, other types of data such as text can be used to determine the test results of the application to be tested. For example: the test data obtained by the test cloud real machine includes test code and reference text; the test cloud real machine runs the application to be tested according to the test code, and obtains the test interface image, which is obtained by taking a screenshot of the program interface of the application to be tested after the target code is executed, and the target code is included in the test code; the test interface image is subjected to text extraction processing to obtain the test text; the test cloud real machine matches the test text and the reference text, and determines the test result of the application to be tested based on the matching result. In some cases, the test cloud real machine can simultaneously implement the test process of two or more different applications to be tested.

[0069] In one embodiment, there are multiple test interface images and multiple reference program interface images. Each test interface image is matched with the corresponding reference program interface image to obtain multiple matching results. If there is a matching result among the multiple matching results indicating that the test interface image does not match the corresponding reference program interface image, then the test result of the application to be tested is determined to be that the application to be tested is running abnormally; if there is no matching result among the multiple matching results indicating that the test interface image does not match the corresponding reference program interface image, then the test result of the application to be tested is determined to be that the application to be tested is running normally. Multiple test interface images are matched with corresponding reference program interface images. If there is a test interface image that does not match the corresponding reference program interface image, then it means that the application to be tested is not running in the method indicated by the test instruction, then it means that the application to be tested is running abnormally. Through the method provided in the embodiment of the present application, the test result of the application to be tested can be effectively determined, the automated testing of the application to be tested is realized, and the testing efficiency of the application to be tested is greatly improved.

[0070] Based on the above embodiments, the beneficial effects of the present application are: the application testing method provided in the embodiments of the present application can enable the cloud real machine to automatically generate corresponding test codes according to various test instructions and test configuration data, without the need for developers to write additional code, effectively ensuring the universality of the method and effectively improving the testing efficiency of the application program; the same batch of cloud real machine devices can respond to test instructions sent by multiple clients at the same time, effectively improving the resource utilization of the cloud real machine; multiple cloud real machines can be used to complete the testing of multi-object interaction scenarios, reducing resource consumption during the testing process.

[0071] See also Figure 4 , Figure 4 A flowchart of another application testing method provided in an embodiment of the present application. The application testing method can be implemented by the client 101 in the above-mentioned application testing system. The process of the application testing method provided in an embodiment of the present application includes but is not limited to:

[0072] S401. Obtain cloud real machine status information of multiple cloud real machines, and select multiple test cloud real machines from the multiple cloud real machines according to the cloud real machine status information of the multiple cloud real machines, wherein the cloud real machine status information is determined according to the running status of the application to be tested in the cloud real machine.

[0073] In the embodiment of the present application, the client can obtain the cloud machine status information of multiple cloud machines from the state database, and select multiple test cloud machines from the multiple cloud machines according to the cloud machine status information, and the cloud machine status information is determined according to the running state of the application to be tested in the cloud machine. According to the cloud machine status information, multiple clients can call the same batch of cloud machines. For example, there are 3 cloud machines in a batch of cloud machines (respectively, cloud machine 1, cloud machine 2 and cloud machine 3), and the cloud machine status information of the current cloud machines are all in a callable state. Client A calls cloud machine 1 and cloud machine 2 to perform application test operations. The cloud machine status information of cloud machine 1 and cloud machine 2 becomes uncallable. At this time, client B can determine that cloud machine 3 is a test cloud machine according to the cloud machine status information of each cloud machine, and use cloud machine 3 to perform application test operations. The method provided in the embodiment of the present application can realize that multiple clients can flexibly call the same batch of cloud machines, realize distributed operation, and effectively improve the resource utilization of cloud machines.

[0074] In one embodiment, the implementation method of selecting multiple test cloud real machines from multiple cloud real machines according to the cloud real machine status information of multiple cloud real machines can be: obtaining test requirement information, the test requirement information includes quantity requirement information and status requirement information, the quantity requirement information is used to indicate the number of test cloud real machines required for application testing, and the status requirement information is used to indicate the status of the test cloud real machine required for application testing; selecting multiple candidate cloud real machines from multiple cloud real machines according to the status requirement information, the quantity requirement information and the cloud real machine status information of multiple cloud real machines, and using the multiple candidate cloud real machines as multiple test cloud real machines. The client can obtain test requirement information, the test requirement information includes quantity requirement information and status requirement information, the quantity requirement information can be used to indicate the number of test cloud real machines required for application testing, and the status requirement information is used to indicate the status of the test cloud real machine required for application testing, for example: for a level in the application to be tested that requires 32 objects to participate, the quantity requirement information is 31 (the tester can also be used as an object), and the status requirement information is that the cloud real machine status information is idle. The client can select multiple candidate cloud machines from multiple cloud machines based on the status requirement information, quantity requirement information, and cloud machine status information of multiple cloud machines, and use the multiple candidate cloud machines as multiple test cloud machines. Figure 5 , Figure 5 A schematic diagram of a method for determining a test cloud real machine provided in an embodiment of the present application is shown. Figure 5In the example, the test cloud machine is numbered "1647", the test cloud machine name is "HWP20", the cloud machine group to which the test cloud machine belongs is "Group 1", the connection device name of the computer device to which the test cloud machine is linked is "HY-PCb", the device model of the test cloud machine is "clt-a100", and the cloud machine status information of the test cloud machine is "idle". The test cloud machine can be "started testing" or "edited" for test instructions. Through the method provided in the embodiment of the present application, multiple cloud machines can be called according to the cloud machine status information, thereby realizing the testing of multi-object interaction scenarios.

[0075] In one embodiment, the application to be tested may be an application for multi-object interaction, for example, the application to be tested may be a competitive game program capable of creating multi-player games, or a casual game program including multi-player levels, etc. The quantity requirement information is determined according to the number of interactive objects required for the test scenario of the application to be tested, for example, if 10 interactive objects are required in a test scenario, the quantity requirement information of the test cloud real machine used for application testing may be 9.

[0076] S402. Obtain test instructions, test configuration data, and a reference program interface image, and store the test instructions, the test configuration data, and the reference program interface image in an instruction database, so that each of the test cloud real machines determines the test code according to the test instructions and the test configuration data, and runs the application to be tested according to the test code.

[0077] In the embodiment of the present application, the client can obtain the test instructions, test configuration data and reference program interface images input externally, and store the obtained test instructions, test configuration data and reference program interface images in the instruction database, so that each test cloud real machine can determine the test code according to the corresponding test instructions and test configuration data, and run the application to be tested according to the test code. Figure 6 , which shows a schematic diagram of a data acquisition method provided in an embodiment of the present application. Figure 6 In the test, the tester can select test instructions from the "preset instruction set" (such as Figure 6 1. cmd: "qualifying" command), you can also configure new test commands according to the specification; after confirming the test command, you can enter the test configuration data, which can include: account information ( Figure 6 "21987903124" in the game), game status information ( Figure 6 "Idle" in the middle), instruction (i.e. the selected test instruction, Figure 6 The command selected in the default command set is cmd: "qualifying" command) and the command-related parameters ( Figure 6for para: "number of people": "single person") Figure 6 The custom state can receive state information different from the existing state input by the user, so as to implement the test of custom scenarios. Figure 6 After the interface shown obtains the test instructions, test configuration data, and reference program interface images, the client can store the test instructions, test configuration data, and reference program interface images in the instruction database, so that the corresponding test cloud real machine can obtain relevant data from the instruction database and perform application test operations. Through the method provided in the embodiment of the present application, personalized application testing can be effectively implemented. Only a small amount of modification of the test instructions, test configuration data, and reference program interface images is required to complete the testing of different scenarios using the cloud real machine, which greatly improves the universality and testing efficiency of the application testing method.

[0078] S403. Determine the target test result of the application to be tested according to the test results of the application to be tested returned by each of the test cloud real machines. The test result is determined by the test cloud real machine based on the matching result of the test interface image and the reference program interface. The test interface image is obtained by the test cloud real machine taking a screenshot of the program interface of the application to be tested when the target code is executed. The target code is included in the test code.

[0079] In the embodiment of the present application, the client can obtain the test results of the application to be tested returned by each test cloud real machine from the database. The test result is determined by the test cloud real machine according to the matching result of the test interface image and the reference program interface. The test interface image is obtained by taking a screenshot of the program interface of the application to be tested by the test cloud real machine when the target code is executed, and the target code is included in the test code. The client can determine the target test result of the application to be tested based on the multiple test results obtained. For example: according to the number of test results with the result of normal operation and the total number of test results, the normal result ratio is obtained, according to the number of test results with the result of abnormal operation and the total number of test results, the abnormal result ratio is obtained, and the target test result of the application to be tested is determined according to the normal result ratio and the abnormal result ratio. The target test result can reflect whether there is a problem in the operation process of the application to be tested to a certain extent. Through the method provided in the embodiment of the present application, the test result of the application to be tested can be accurately determined, which is conducive to improving the application to be tested according to the test results and improving the experience of the application to be tested.

[0080] Based on the above embodiments, the beneficial effects of the present application are: the application testing method provided in the embodiments of the present application can complete the application testing of multi-object interaction scenarios based only on input test instructions, test configuration data, reference program interface images and other data, greatly improving the efficiency of application testing and reducing the manpower and material resources consumed in conducting multi-object interaction scenario testing; for test instructions of different scenarios, the application testing method provided by the present application can also implement the application testing process without the need for developers to write complex code. The application testing method provided by the present application has good universality and can effectively reduce resource consumption.

[0081] See also Figure 7 , Figure 7 A flowchart of another application testing method provided in an embodiment of the present application. The application testing method can be performed by the above Figure 1 The application testing system in the present application is implemented. The process of the application testing method provided in the embodiment of the present application includes but is not limited to:

[0082] S701. The client obtains cloud real machine status information of multiple cloud real machines from a database.

[0083] In the embodiment of the present application, when the client needs to test the application, it can obtain the cloud machine status information of multiple cloud machines from the database, and the cloud machine status information is used to indicate whether the cloud machine is in a callable state. According to the cloud machine status information of the cloud machine, the client can determine which cloud machines are in a callable state. According to the cloud machine status information, multiple clients can call the same batch of cloud machines.

[0084] In one embodiment, after the client obtains the cloud real machine status information of multiple cloud real machines, the cloud real machine status information can be displayed on the display interface of the client so that the tester can understand the status of each cloud real machine.

[0085] In one embodiment, the client may periodically obtain the cloud real machine status information of multiple cloud real machines from the database. For example, the client obtains the cloud real machine status information of multiple cloud real machines from the database every 1 second.

[0086] S702: The client selects multiple test cloud real machines from the multiple cloud real machines according to the cloud real machine status information of the multiple cloud real machines.

[0087] In the embodiment of the present application, the client can select multiple test cloud real machines from multiple cloud real machines according to the cloud real machine status information of the multiple cloud real machines. The test cloud real machines will be used to test the application to be tested.

[0088] In one embodiment, the implementation method of selecting multiple test cloud real machines from multiple cloud real machines according to the cloud real machine status information of multiple cloud real machines can be: obtaining test requirement information, the test requirement information includes quantity requirement information and status requirement information, the quantity requirement information is used to indicate the number of test cloud real machines required for application testing, and the status requirement information is used to indicate the status of the test cloud real machine required for application testing; selecting multiple candidate cloud real machines from multiple cloud real machines according to the status requirement information, the quantity requirement information and the cloud real machine status information of multiple cloud real machines, and using the multiple candidate cloud real machines as multiple test cloud real machines. The client can obtain test requirement information, the test requirement information includes quantity requirement information and status requirement information, the quantity requirement information can be used to indicate the number of test cloud real machines required for application testing, and the status requirement information is used to indicate the status of the test cloud real machine required for application testing. The client can select multiple candidate cloud real machines from multiple cloud real machines according to the status requirement information, the quantity requirement information and the cloud real machine status information of multiple cloud real machines, and use the multiple candidate cloud real machines as multiple test cloud real machines.

[0089] In one embodiment, the application to be tested may be an application for multi-object interaction, for example, the application to be tested may be a competitive game program capable of creating multi-player games, or a casual game program including multi-player levels, etc. The quantity requirement information is determined according to the number of interactive objects required by the test scenario of the application to be tested.

[0090] In one embodiment, the client may include a cloud real machine status display module, which can be used to implement the process of obtaining cloud real machine status information of multiple cloud real machines, and selecting multiple test cloud real machines from multiple cloud real machines based on the cloud real machine status information of the multiple cloud real machines, and displaying the relevant information of the selected multiple test cloud real machines in the display interface of the client.

[0091] S703: The client obtains test instructions, test configuration data, and reference program interface images.

[0092] In an embodiment of the present application, the client can obtain externally input test instructions, test configuration data, and reference program interface images, wherein the test instructions may include multiple instructions, such as: cmd instructions, level instructions, game instructions, etc. The test instructions correspond to the test configuration data. For example, for a cmd instruction, its test configuration data may include additional parameters of the cmd instruction and instruction type data (for example: single-click instructions, double-click instructions, sliding instructions, etc.). The test configuration data may include the authentication data required to run the application to be tested, the operation data of the application to be tested, and the program interface matching data. The reference program interface image has a corresponding relationship with the test instruction, and one test instruction may correspond to one reference program interface image.

[0093] In one embodiment, the client may include a user input module, and the user input module may be used to receive data input by a user (eg, test instructions, test configuration data, reference program interface images, etc.).

[0094] S704: The client sends the test instruction, the test configuration data, and the reference program interface image to the database.

[0095] In the embodiment of the present application, the client can store the acquired test instructions, test configuration data and reference program interface images in the database, so that each test cloud real machine can determine the test code according to the corresponding test instructions and test configuration data, and run the application to be tested according to the test code. Through the method provided in the embodiment of the present application, the test process of the application to be tested can be implemented without investing a lot of manpower and material resources, effectively saving resources.

[0096] In one embodiment, the client may include an external event instruction management module, which can be used to obtain data input by external objects (for example, test instructions, test configuration data, reference program interface images, etc.), and integrate the data input by external objects and store them in a database.

[0097] S705: The test cloud real machine obtains the test instruction, the test configuration data and the reference program interface image from the database.

[0098] In the embodiment of the present application, there may be one or more test cloud real machines for performing the test process of the application to be tested, and one of the test cloud real machines is used as an example for illustration. The test cloud real machine obtains the test instructions, test configuration data, and reference program interface images associated with the application to be tested from the database, so as to facilitate subsequent application testing based on these data.

[0099] S706: The test cloud real machine generates a test code according to the test instruction and the test configuration data.

[0100] In an embodiment of the present application, the test cloud real machine can generate test code according to the test instruction and the test configuration data, and its implementation method can be: according to the test instruction, a code search process is performed from the code database to obtain a search result; if the search result indicates that there is a code that matches the test instruction, the code that matches the test instruction and the test configuration data are combined to obtain the test code; if the search result indicates that there is no code that matches the test instruction, an instruction implementation code is generated according to the test instruction, and the instruction implementation code is combined with the test configuration data to obtain the test code. After the test cloud real machine obtains the test instruction, it is necessary to convert the instruction into the corresponding processing logic code. Therefore, according to the test instruction, a code search process can be performed from the code database of the cloud real machine to obtain a search result; if the search result indicates that there is a code that matches the test instruction, the code that matches the test code and the test configuration data are combined to obtain the test code, that is, when there is a processing logic code corresponding to the test instruction in the cloud real machine, the test code required for this test can be generated according to the existing code and the test configuration data, without the need to write additional automation code. If the search result indicates that there is no code matching the test instruction, the instruction implementation code can be generated according to the test instruction, and the instruction implementation code and the test configuration data can be combined and processed to obtain the test code. Through the method provided in the embodiment of the present application, the cloud real machine can automatically generate test code according to the test instruction and test configuration data, without the need to write complex code, effectively improving the test efficiency of the application.

[0101] In one embodiment, the test cloud real machine may include a cloud real machine automation script core engine layer, and the engine layer may include a state machine management module. The state machine management module can be used to manage the behavior of the test cloud real machine when different instructions are input, that is, to generate different test codes according to the input test instructions, so as to control the application to be tested. For example, when the application to be tested is a game program, the state machine management module can complete the event behaviors in the game such as starting the game, pulling authentication, logging into the game, matching, forming a team, chatting, participating in the game, etc., and manage and trigger real-time events through instruction responses. The test cloud real machine can obtain test instructions, test configuration data and reference program interface images from the database at regular intervals (for example, every 1 second), and pass the obtained data to the state machine management module. The state machine management module can determine the test data for the application to be tested based on the test instructions, test configuration data and reference program interface images (the test data includes test codes and reference program interface images).

[0102] In one embodiment, the state machine management module may also include special codes for handling some abnormal situations. For example: the application to be tested is a game program, and the processing process indicated by the test code in the test cloud real machine is to open the game program, log in to the game account, enter the game character selection interface, and select the first game character; however, when the game program is running, after logging in to the game account, various announcements, game tasks, game prompts and other pop-up windows will pop up in the program interface. If these pop-up windows cannot be closed, the game character selection interface cannot be entered. Therefore, special codes can be used to handle this abnormal situation, close irrelevant pop-up windows, and allow the test of the application to be tested to proceed normally. Through the method provided in the embodiment of the present application, the testing of the application can be efficiently and smoothly implemented, and the efficiency of application testing can be improved.

[0103] In one embodiment, the test cloud real machine may also include a basic interface module, which can be used to implement basic interactive interface processing operations. The basic interface module can be used together with the above-mentioned state machine management module to implement the test process of the application to be tested. The basic interface module can be used to implement functions such as interactive interface recognition, text recognition, control recognition in the interface, bitmap recognition, and key operations. For example: the basic interface module includes the Find_ui function, which can be used to find a specified image based on image information; the click function can be used to implement a click operation; the double_click function can be used to implement a double-click operation; the Slide function can be used to implement a sliding operation; the Long_click function can be used to implement a long press click operation.

[0104] S707. The test cloud real machine runs the application to be tested according to the test code and obtains a test interface image. The test interface image is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code.

[0105] In an embodiment of the present application, the test cloud real machine can run the application to be tested according to the test code, and during the operation of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code. When the interface display content of the application to be tested changes, a test interface image can be obtained, and the test interface image is obtained by taking a screenshot of the program interface of the application to be tested after the target code is executed, and the target code is included in the test code. The cloud real machine can take screenshots of the program interface of the application to be tested multiple times to obtain multiple test interface images. The target code can be a part of the code in the test code, the target code can indicate a certain operation, and there can be a corresponding relationship between the target code and the test interface image. Through the method provided in the embodiment of the present application, the cloud real machine can be used to obtain the test interface image, which is convenient for determining the test result of the application to be tested based on the test interface image later.

[0106] In one embodiment, the test cloud real machine can obtain the running status of the application to be tested, and determine the cloud real machine status information according to the running status of the application to be tested; the cloud real machine status information is stored in a database so that the client can obtain the cloud real machine status information from the database. The cloud real machine status information is used to indicate whether the test cloud real machine is in a callable state.

[0107] S708. The test cloud real machine performs matching processing based on the test interface image and the reference program interface image, and determines the test result of the application to be tested based on the matching result.

[0108] In an embodiment of the present application, the test cloud real machine can perform matching processing according to the test interface image and the reference program interface image to obtain a matching result. There is a corresponding relationship between the test interface image and the reference program interface image. For example, if the operation indicated by a certain part of the code in the test code is to enter a certain specified level, the reference program interface image includes the reference program interface image of the specified level interface. When the cloud real machine runs the application to be tested according to the test code, the part of the code is executed, and the display interface of the application to be tested is the interface of the specified level. The cloud real machine takes a screenshot of the interface display content of the application to be tested and obtains the test interface image of the specified level interface. The reference program interface image of the specified level interface corresponds to the test interface image of the specified level interface. Matching processing is performed according to the corresponding test interface image and the reference program interface image to obtain a matching result. There can be one or more test interface images, and there can also be one or more matching results determined according to the test interface image. Therefore, the test result of the application to be tested can be determined according to one or more matching results.

[0109] In one embodiment, the test cloud real machine includes a result analysis module, which can perform matching processing based on the test interface image and the reference program interface image, and determine the test result of the application to be tested based on the matching result.

[0110] S709: The test cloud real machine sends the test result to the database.

[0111] In the embodiment of the present application, after the test cloud real machine executes the test process of the application to be tested, the test results can be obtained and stored in the database. The database may contain the instruction records executed by the test cloud real machine, for example: the database may include a cloud real machine instruction execution table, which may include record instruction number, instruction name, sequence number, client type, authentication information of the application to be tested, the status of the application to be tested, test instructions, test configuration data, test results, test completion time and other information. This information is conducive to the analysis of the test process of the application to be tested, and when the test cloud real machine fails, it is also conducive to quickly determining the cause of the abnormality.

[0112] S710. The client obtains the test results of the application to be tested returned by each of the test cloud real machines from the database.

[0113] In an embodiment of the present application, the client can periodically (for example, every 1 second) obtain the test results of the application to be tested returned by each test cloud real machine from the database. The test results are determined by the test cloud real machine based on the matching results of the test interface image and the reference program interface. The test interface image is obtained by taking a screenshot of the program interface of the application to be tested by the test cloud real machine when the target code is executed, and the target code is included in the test code.

[0114] See also Figure 8 , which is a system architecture diagram of another application testing system provided in an embodiment of the present application. Figure 8In the figure, the structure of a test cloud real machine is taken as an example to illustrate. The application test system includes a client 801, a database 802 and a test cloud real machine 803, wherein the client 801 includes a user input module 8011 and a cloud real machine status display module 8012; the database 802 includes a cloud real machine information storage module 8021 and a custom instruction storage module 8022; the test cloud real machine 803 includes a cloud real machine script control layer 8031, and the cloud real machine script control layer 8031 ​​includes a state machine management module 8032 and a result detection module 8033. When performing application testing of the application to be tested, the cloud real machine status display module 8012 can obtain multiple cloud real machine status information from the cloud real machine information storage module 8021, and select multiple test cloud real machines from multiple cloud real machines; the user input module 8011 obtains the input application test related data (for example: test instructions, test configuration data and reference program interface images), and stores the application test related data in the custom instruction storage module 8022 of the database 802; the test cloud real machine 803 can obtain application test related data from the custom instruction storage module 8022 at a regular interval, and transmit it to the cloud real machine script control module 803. The state machine management module 8032 in the control layer 8031 ​​executes the test process of the application to be tested, obtains the cloud real machine state information, and stores the cloud real machine state information in the cloud real machine information storage module 8021 of the database 802; the result detection module 8033 can process the data generated by the state machine management module 8032 to obtain the test result, and store the test result in the custom instruction storage module 8022, so that the client can obtain the test result from the custom instruction storage module 8022, and determine the target test result of the application to be tested according to the test result. Through the method provided in the embodiment of the present application, the test process of the multi-object interaction scene can be quickly realized using the cloud real machine, which greatly saves the resources consumed by the test and improves the efficiency of the application test.

[0115] S711. The client determines a target test result of the application to be tested according to the acquired test result.

[0116] In an embodiment of the present application, the client can determine the target test result of the application to be tested based on the multiple test results obtained. For example: the normal result ratio is obtained by calculating the ratio of the number of test results with normal operation results to the total number of test results, the abnormal result ratio is obtained by calculating the ratio of the number of test results with abnormal operation results to the total number of test results, and the target test result of the application to be tested is determined based on the normal result ratio and the abnormal result ratio. The target test result can reflect whether there is a problem in the operation process of the application to be tested to a certain extent. Through the method provided in the embodiment of the present application, the test result of the application to be tested can be accurately determined, which is conducive to improving the application to be tested according to the test results and improving the user experience of the application to be tested.

[0117] See also Fig. 9 , which is a schematic diagram of an application testing method provided by an embodiment of the present application. Fig. 9 It can be seen that the overall process of the application testing method can be: 901, the client periodically obtains multiple cloud real machine status information from the database; 902, the client determines the test cloud real machine and obtains application test related data. Application test related data may include test instructions, test configuration data and reference program interface images; 903, the client stores application test related data in the database; 904, the test cloud real machine periodically obtains application test related data from the database; 905, the test cloud real machine executes the application testing process; 906, the test cloud real machine determines the cloud real machine status information and stores it in the database. After step 906 is executed, step 901 can be executed. When the test cloud real machine completes the application testing process and obtains the test results, the test results can be stored in the database, so that the client can obtain the test results from the database, thereby obtaining the target test results of the application to be tested. Through the application testing method provided in the embodiment of the present application, the cloud real machine can be used to quickly implement the testing process of multi-object interaction scenarios, which greatly saves manpower and material resources and improves the efficiency of application testing. At the same time, the method provided in the embodiment of the present application has good universality. For different testing scenarios, there is no need to write a large amount of complex execution code. Only a small amount of modification of configuration information is required to implement the testing process of the application.

[0118] Based on the above embodiments, the beneficial effects of the present application are: the application testing method provided in the embodiments of the present application can complete the application testing of multi-object interaction scenarios only according to the input test instructions, test configuration data, reference program interface images and other data, which greatly improves the efficiency of application testing and reduces the manpower and material resources consumed in testing multi-object interaction scenarios; it can enable the cloud real machine to automatically generate corresponding test codes according to various test instructions and test configuration data, without the need for developers to write additional code, thereby improving the convenience of the present method; the same batch of cloud real machine devices can respond to test instructions sent by multiple clients at the same time, effectively improving the resource utilization of the cloud real machine; for different test scenarios, there is no need to write a large amount of complex execution code, and only a small amount of modification of configuration information is required to implement the testing process of the application program, effectively ensuring the universality of the application testing method provided by the present application.

[0119] See also Fig.10 , Fig.10 This is a structural block diagram of an application testing device provided in an embodiment of the present application. The application testing device can implement the relevant functions of the above-mentioned test cloud real machine. The device includes:

[0120] The acquisition unit 1001 is used to acquire test data for the application to be tested, wherein the test data includes test code and a reference program interface image;

[0121] The processing unit 1002 is configured to run the application to be tested according to the test code, wherein, during the running of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code;

[0122] The acquisition unit 1001 is further used to acquire a test interface image, wherein the test interface image is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code;

[0123] The matching unit 1003 is used to perform matching processing according to the test interface image and the reference program interface image, and determine the test result of the application to be tested according to the matching result.

[0124] In one embodiment, when acquiring test data for an application to be tested, the acquisition unit 1001 is specifically used to: acquire test instructions, test configuration data, and a reference program interface image for the application to be tested, the reference program interface image having a corresponding relationship with the test instructions, the test configuration data including authentication data required to run the application to be tested, running data of the application to be tested, and program interface matching data; generate test code according to the test instructions and the test configuration data; and use the test code and the reference program interface image as test data for the application to be tested.

[0125] In one embodiment, when the acquisition unit 1001 generates the test code according to the test instruction and the test configuration data, it is specifically used to: perform code search processing from a code database according to the test instruction to obtain a search result; if the search result indicates that there is code matching the test instruction, the code matching the test instruction and the test configuration data are combined to obtain the test code; if the search result indicates that there is no code matching the test instruction, an instruction implementation code is generated according to the test instruction, and the instruction implementation code is combined with the test configuration data to obtain the test code.

[0126] In one embodiment, the acquisition unit 1001 is also used to: obtain the running status of the application to be tested, and determine the cloud real machine status information according to the running status of the application to be tested; store the cloud real machine status information in a status database, and the cloud real machine status information is used to indicate whether the test cloud real machine is in a callable state.

[0127] In one embodiment, when the matching unit 1003 performs matching processing on the test interface image and the reference program interface image, and determines the test result of the application to be tested based on the matching result, it is specifically used to: perform feature extraction processing on the test interface image to obtain test interface image features; perform feature extraction processing on the reference program interface image to obtain reference program interface image features; perform feature matching processing based on the test interface image features and the reference program interface image features to obtain image feature similarity; determine the test result of the application to be tested based on the image feature similarity, and the test result is used to indicate whether the application to be tested is running normally.

[0128] See also Fig.11 , Fig.11 A schematic block diagram of another application testing device provided in an embodiment of the present application. The application testing device can implement the relevant functions of the above-mentioned client. The device includes:

[0129] A selection unit 1101 is used to obtain cloud machine status information of multiple cloud machines, and select multiple test cloud machines from the multiple cloud machines according to the cloud machine status information of the multiple cloud machines, wherein the cloud machine status information is determined according to the running status of the application to be tested in the cloud machine;

[0130] The storage unit 1102 is used to obtain test instructions, test configuration data and reference program interface images, and store the test instructions, the test configuration data and the reference program interface images in an instruction database, so that each of the test cloud real machines determines the test code according to the test instructions and the test configuration data, and runs the application to be tested according to the test code;

[0131] The determination unit 1103 is used to determine the target test result of the application to be tested based on the test results of the application to be tested returned by each of the test cloud real machines, and the test result is determined by the test cloud real machine based on the matching result of the test interface image and the reference program interface. The test interface image is obtained by the test cloud real machine taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code.

[0132] In one embodiment, when the selection unit 1101 selects multiple test cloud real machines from the multiple cloud real machines according to the cloud real machine status information of the multiple cloud real machines, it is specifically used to: obtain test requirement information, the test requirement information including quantity requirement information and status requirement information, the quantity requirement information is used to indicate the number of test cloud real machines required for application testing, and the status requirement information is used to indicate the status of the test cloud real machines required for application testing; select multiple to-be-selected cloud real machines from the multiple cloud real machines according to the status requirement information, the quantity requirement information and the cloud real machine status information of the multiple cloud real machines, and use the multiple to-be-selected cloud real machines as multiple test cloud real machines.

[0133] In one embodiment, the application to be tested is a multi-object interactive application, and the quantity requirement information is determined according to the quantity of interactive objects required by the test scenario of the application to be tested.

[0134] It can be understood that the functions of each functional unit of the application testing device in the embodiment of the present application can be specifically implemented according to the application testing method in the above method embodiment, and its specific implementation process can refer to the relevant description in the above application testing method embodiment, which will not be repeated here.

[0135] Based on the above embodiments, the beneficial effects of the present application are: the application testing method provided in the embodiments of the present application can complete the application testing of multi-object interaction scenarios only according to the input test instructions, test configuration data, reference program interface images and other data, which greatly improves the efficiency of application testing and reduces the manpower and material resources consumed in testing multi-object interaction scenarios; it can enable the cloud real machine to automatically generate corresponding test codes according to various test instructions and test configuration data, without the need for developers to write additional code, thereby improving the convenience of the present method; the same batch of cloud real machine devices can respond to test instructions sent by multiple clients at the same time, effectively improving the resource utilization of the cloud real machine; for different test scenarios, there is no need to write a large amount of complex execution code, and only a small amount of modification of configuration information is required to implement the testing process of the application program, effectively ensuring the universality of the application testing method provided by the present application.

[0136] See also Fig.12 , Fig.12 A block diagram of a computer device provided in an embodiment of the present application. The computer device described in the embodiment of the present application includes: a processor 1201, a communication interface 1202, and a memory 1203. The processor 1201, the communication interface 1202, and the memory 1203 may be connected via a bus or other means, and the embodiment of the present application takes the connection via a bus as an example.

[0137] Among them, the processor 1201 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which can parse various instructions in the computer device and process various data of the computer device. For example, the CPU can be used to parse the power on and off instructions sent by the user to the computer device, and control the computer device to perform power on and off operations; for another example, the CPU can transmit various interactive data between the internal structures of the computer device, and so on. The communication interface 1202 can optionally include a standard wired interface, a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which is controlled by the processor 1201 to send and receive data. The memory 1203 (Memory) is a memory device in the computer device for storing programs and data. It can be understood that the memory 1203 here can include both the built-in memory of the computer device and the extended memory supported by the computer device. The memory 1203 provides a storage space, which stores the operating system of the computer device, which may include but is not limited to: Android system, iOS system, Windows Phone system, etc., and this application does not limit this.

[0138] In a feasible embodiment, the computer device may correspond to the test cloud real machine described above. At this time, the processor 1201 performs the following operations by running the executable program code in the memory 1203:

[0139] Acquire test data for the application to be tested, wherein the test data includes test code and a reference program interface image;

[0140] Running the application to be tested according to the test code, wherein, during the running of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code;

[0141] Acquire a test interface image, wherein the test interface image is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code;

[0142] A matching process is performed based on the test interface image and the reference program interface image, and a test result of the application program to be tested is determined based on the matching result.

[0143] In one embodiment, when the processor 1201 obtains test data for the application to be tested, it is specifically used to: obtain test instructions, test configuration data and a reference program interface image for the application to be tested, the reference program interface image having a corresponding relationship with the test instructions, and the test configuration data including authentication data required to run the application to be tested, running data of the application to be tested and program interface matching data; generate test code according to the test instructions and the test configuration data; and use the test code and the reference program interface image as test data for the application to be tested.

[0144] In one embodiment, when the processor 1201 generates the test code according to the test instruction and the test configuration data, it is specifically used to: perform code search processing from a code database according to the test instruction to obtain a search result; if the search result indicates that there is code matching the test instruction, the code matching the test instruction and the test configuration data are combined to obtain the test code; if the search result indicates that there is no code matching the test instruction, an instruction implementation code is generated according to the test instruction, and the instruction implementation code is combined with the test configuration data to obtain the test code.

[0145] In one embodiment, the processor 1201 is also used to: obtain the running status of the application to be tested, and determine the cloud real machine status information according to the running status of the application to be tested; store the cloud real machine status information in a status database, and the cloud real machine status information is used to indicate whether the test cloud real machine is in a callable state.

[0146] In one embodiment, when the processor 1201 performs matching processing on the test interface image and the reference program interface image, and determines the test result of the application to be tested based on the matching result, it is specifically used to: perform feature extraction processing on the test interface image to obtain test interface image features; perform feature extraction processing on the reference program interface image to obtain reference program interface image features; perform feature matching processing on the test interface image features and the reference program interface image features to obtain image feature similarity; determine the test result of the application to be tested based on the image feature similarity, and the test result is used to indicate whether the application to be tested is running normally.

[0147] In a feasible embodiment, the computer device may correspond to the client described above. At this time, the processor 1201 executes the following operations by running the executable program code in the memory 1203:

[0148] Acquire cloud machine status information of multiple cloud machines, and select multiple test cloud machines from the multiple cloud machines according to the cloud machine status information of the multiple cloud machines, wherein the cloud machine status information is determined according to the running status of the application to be tested in the cloud machine;

[0149] Acquire test instructions, test configuration data, and a reference program interface image, and store the test instructions, the test configuration data, and the reference program interface image in an instruction database, so that each of the test cloud real machines determines a test code according to the test instructions and the test configuration data, and runs the application to be tested according to the test code;

[0150] According to the test results of the application to be tested returned by each of the test cloud real machines, the target test result of the application to be tested is determined, and the test result is determined by the test cloud real machine based on the matching result of the test interface image and the reference program interface. The test interface image is obtained by the test cloud real machine taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code.

[0151] In one embodiment, when the processor 1201 selects multiple test cloud real machines from the multiple cloud real machines based on the cloud real machine status information of the multiple cloud real machines, it is specifically used to: obtain test requirement information, the test requirement information including quantity requirement information and status requirement information, the quantity requirement information is used to indicate the number of test cloud real machines required for application testing, and the status requirement information is used to indicate the status of the test cloud real machines required for application testing; select multiple to-be-selected cloud real machines from the multiple cloud real machines based on the status requirement information, the quantity requirement information and the cloud real machine status information of the multiple cloud real machines, and use the multiple to-be-selected cloud real machines as multiple test cloud real machines.

[0152] In one embodiment, the application to be tested is a multi-object interactive application, and the quantity requirement information is determined according to the quantity of interactive objects required by the test scenario of the application to be tested.

[0153] In a specific implementation, the processor 1201, communication interface 1202 and memory 1203 described in the embodiment of the present application can execute the implementation method of the client or test cloud real machine described in an application testing method provided in an embodiment of the present application, and can also execute the implementation method described in an application testing device provided in an embodiment of the present application, which will not be repeated here.

[0154] Based on the above embodiments, the beneficial effects of the present application are: the application testing method provided in the embodiments of the present application can complete the application testing of multi-object interaction scenarios only according to the input test instructions, test configuration data, reference program interface images and other data, which greatly improves the efficiency of application testing and reduces the manpower and material resources consumed in testing multi-object interaction scenarios; it can enable the cloud real machine to automatically generate corresponding test codes according to various test instructions and test configuration data, without the need for developers to write additional code, thereby improving the convenience of the present method; the same batch of cloud real machine devices can respond to test instructions sent by multiple clients at the same time, effectively improving the resource utilization of the cloud real machine; for different test scenarios, there is no need to write a large amount of complex execution code, and only a small amount of modification of configuration information is required to implement the testing process of the application program, effectively ensuring the universality of the application testing method provided by the present application.

[0155] The present application also provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is run on a computer, the computer executes the application testing method as described in the present application. The specific implementation method can be referred to the above description, and will not be repeated here.

[0156] The embodiment of the present application also provides a computer program product, which includes a computer program or a computer instruction, and the computer program or the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device performs the application test method as described in the embodiment of the present application. The specific implementation method can be referred to the above description, which will not be repeated here.

[0157] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0158] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0159] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An application testing method, characterized in that: The method comprises: Acquire test data for the application to be tested, wherein the test data includes test code and a reference program interface image; Running the application to be tested according to the test code, wherein, during the running of the application to be tested, the interface display content of the application to be tested changes according to the operation indicated by the test code; Acquire a test interface image, wherein the test interface image is obtained by taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code; A matching process is performed based on the test interface image and the reference program interface image, and a test result of the application program to be tested is determined based on the matching result.

2. The method according to claim 1, characterized in that The obtaining of test data for the application to be tested includes: Acquire test instructions, test configuration data, and a reference program interface image for the application to be tested, wherein the reference program interface image has a corresponding relationship with the test instructions, and the test configuration data includes authentication data required to run the application to be tested, running data of the application to be tested, and program interface matching data; Generate test code according to the test instruction and the test configuration data; The test code and the reference program interface image are used as test data for the application program to be tested.

3. The method according to claim 2, characterized in that The generating of the test code according to the test instruction and the test configuration data comprises: Perform code search processing from a code database according to the test instruction to obtain a search result; If the search result indicates that there is a code matching the test instruction, the code matching the test instruction and the test configuration data are combined to obtain a test code; If the search result indicates that there is no code matching the test instruction, an instruction implementation code is generated according to the test instruction, and the instruction implementation code is combined with the test configuration data to obtain a test code.

4. The method according to claim 1, characterized in that: The method further comprises: Obtaining the running status of the application to be tested, and determining the cloud real machine status information according to the running status of the application to be tested; The cloud real machine state information is stored in a state database, and the cloud real machine state information is used to indicate whether the test cloud real machine is in a callable state.

5. The method according to any one of claims 1 to 4, characterized in that: The matching process is performed according to the test interface image and the reference program interface image, and the test result of the application to be tested is determined according to the matching result, including: Performing feature extraction processing on the test interface image to obtain test interface image features; Performing feature extraction processing on the reference program interface image to obtain reference program interface image features; Perform feature matching processing based on the test interface image features and the reference program interface image features to obtain image feature similarity; A test result of the application to be tested is determined according to the image feature similarity, and the test result is used to indicate whether the application to be tested is running normally.

6. An application testing method, characterized in that: The method comprises: Acquire cloud machine status information of multiple cloud machines, and select multiple test cloud machines from the multiple cloud machines according to the cloud machine status information of the multiple cloud machines, wherein the cloud machine status information is determined according to the running status of the application to be tested in the cloud machine; Acquire test instructions, test configuration data, and a reference program interface image, and store the test instructions, the test configuration data, and the reference program interface image in an instruction database, so that each of the test cloud real machines determines a test code according to the test instructions and the test configuration data, and runs the application to be tested according to the test code; According to the test results of the application to be tested returned by each of the test cloud real machines, the target test result of the application to be tested is determined, and the test result is determined by the test cloud real machine based on the matching result of the test interface image and the reference program interface. The test interface image is obtained by the test cloud real machine taking a screenshot of the program interface of the application to be tested when the target code is executed, and the target code is included in the test code.

7. The method according to claim 6, characterized in that The selecting a plurality of test cloud real machines from the plurality of cloud real machines according to the cloud real machine status information of the plurality of cloud real machines comprises: Acquire test requirement information, where the test requirement information includes quantity requirement information and status requirement information, where the quantity requirement information is used to indicate the quantity of test cloud real machines required for application testing, and the status requirement information is used to indicate the status of the test cloud real machines required for application testing; According to the state requirement information, the quantity requirement information and the cloud real machine state information of the multiple cloud real machines, multiple cloud real machines to be selected are selected from the multiple cloud real machines, and the multiple cloud real machines to be selected are used as multiple test cloud real machines.

8. The method according to claim 7, characterized in that The application to be tested is an application for multi-object interaction, and the quantity requirement information is determined according to the quantity of interactive objects required by the test scenario of the application to be tested.

9. An application testing device, characterized in that: The method comprises a unit for executing the application testing method according to any one of claims 1 to 5, or comprises a unit for executing the application testing method according to any one of claims 6 to 8.

10. A computer device, characterized in that: include: A processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores an executable program code, and the processor is used to call the executable program code to implement the application testing method as described in any one of claims 1 to 5, or to implement the application testing method as described in any one of claims 6 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when executed on a computer, enable the computer to implement the application testing method according to any one of claims 1 to 5, or the application testing method according to any one of claims 6 to 8.

12. A computer program product, characterized in that The computer program product includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the application testing method according to any one of claims 1 to 5 is implemented, or the application testing method according to any one of claims 6 to 8 is implemented.

Images