Application testing method, device and system

Abstract

The invention provides an application testing method, device and system, and relates to the technical field of cloud computing, in particular to the technical field of cloud mobile phones. A specific embodiment of the application test method comprises the following steps: receiving a test template corresponding to a to-be-tested application; installing and starting the to-be-tested application; executing a predefined operation sequence in the test template in the to-be-tested application; in response to monitoring that the state of the to-be-tested application meets a preset condition, collecting a current screenshot; and performing semantic understanding on the current screenshot to generate a test result of the to-be-tested application.

Description

Technical Field

[0001] The present disclosure relates to the field of cloud computing technology, specifically the field of cloud phone technology. Background Art

[0002] With the rapid development of the mobile application ecosystem, application quality assurance has become increasingly important. Currently, there are mainly the following types of solutions for mobile application testing and quality monitoring: First, manual testing and real-device labs: A real-device lab is set up, where testers manually execute test cases on various real devices. Testers follow the test cases step by step, operate the application, observe the results, and record any issues.

[0003] Second, scripted automated testing: To improve efficiency, the industry widely adopts automated testing frameworks such as Appium, Calabash, Robotium, and UIAutomator. Testers write or record scripts to simulate user clicks, input, and other operations. The framework then executes the scripts on the phone and checks the expected results.

[0004] Third, random testing: The installation platform provides a Monkey testing tool that performs a large number of random clicks, swipes, and other actions on the device to try to trigger crashes or exceptions. Many app stores perform Monkey testing before releasing apps to identify potential crashes. Summary of the Invention

[0005] The embodiments of the present disclosure provide an application testing method, apparatus, system, device, storage medium, and program product.

[0006] In the first aspect, an embodiment of the present disclosure proposes an application testing method, which is applied to a cloud phone, including: receiving a test template corresponding to the application to be tested; installing and starting the application to be tested; executing a predefined operation sequence in the test template in the application to be tested; in response to monitoring that the status of the application to be tested meets preset conditions, collecting a current screenshot; performing semantic understanding on the current screenshot to generate a test result for the application to be tested.

[0007] In the second aspect, an embodiment of the present disclosure proposes an application testing method, which is applied to a scheduling center, including: receiving a test request for an application to be tested, wherein the test request includes test information of the application to be tested; based on the test information, determining a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested; sending the test template to the cloud phone, so that the cloud phone tests the application to be tested based on the test template.

[0008] On the third aspect, an embodiment of the present disclosure proposes an application testing device, which is arranged in a cloud phone, including: a receiving module, configured to receive a test template corresponding to the application to be tested; an installation module, configured to install and start the application to be tested; an execution module, configured to execute a predefined operation sequence in the test template in the application to be tested; a collection module, configured to collect a current screenshot in response to monitoring that the status of the application to be tested meets preset conditions; an understanding module, configured to perform semantic understanding of the current screenshot and generate a test result for the application to be tested.

[0009] In the fourth aspect, an embodiment of the present disclosure proposes an application testing device, which is arranged in a scheduling center, including: a receiving module, configured to receive a test request for an application to be tested, wherein the test request includes test information of the application to be tested; a determination module, configured to determine a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested based on the test information; a sending module, configured to send the test template to the cloud phone, so that the cloud phone tests the application to be tested based on the test template.

[0010] In the fifth aspect, an embodiment of the present disclosure proposes an application testing system, including a cloud phone cluster and a scheduling center; and the scheduling center is configured to receive a test request for an application to be tested, wherein the test request includes test information of the application to be tested; based on the test information, determine a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested; send the test template to the cloud phone; the cloud phone is configured to install and start the application to be tested; execute a predefined operation sequence in the test template in the application to be tested; in response to monitoring that the status of the application to be tested meets a preset condition, collect a current screenshot; perform semantic understanding on the current screenshot, and generate a test result for the application to be tested.

[0011] In the sixth aspect, an embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect or the second aspect.

[0012] In a seventh aspect, an embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute the method described in the first aspect or the second aspect.

[0013] In an eighth aspect, an embodiment of the present disclosure proposes a computer program product, including a computer program, which implements the method described in the first aspect or the second aspect when executed by a processor.

[0014] The key or important features of the embodiments of the present disclosure are also not used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0015] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description when read in conjunction with the following drawings. The drawings are provided for the purpose of better understanding the present disclosure and are not to be construed as limiting the present disclosure. Among them: Figure 1 is a flowchart of an embodiment of an application testing method according to the present disclosure; Figure 2 is a flowchart of another embodiment of an application testing method according to the present disclosure; Figure 3 is an interaction timing diagram of an embodiment of an application testing system according to the present disclosure; Figure 4 is a business flowchart of application testing; Figure 5 is a background architecture diagram of application testing; Figure 6 is an application discovery diagram of application testing; Figure 7A is an implementation flowchart of a cloud mobile cluster; Figure 7B is an implementation flowchart of a multi-modal AI analysis service; Figure 7C is an implementation flowchart of result processing and regression; Figure 7D is an implementation flowchart of a dispatch center; Figure 8 is a structural schematic diagram of an embodiment of an application testing device according to the present disclosure; Figure 9 is a structural schematic diagram of another embodiment of an application testing device according to the present disclosure; Figure 10 is a block diagram of an electronic device for implementing an application testing method of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Thus, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of known functions and constructions are omitted in the following description for clarity and conciseness.

[0017] It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] Figure 1 Flow 100 of one embodiment of an application testing method according to the present disclosure is shown. The application testing method is applied to a cloud phone, including the following steps: Step 101, receiving a test template corresponding to the application to be tested.

[0019] In this embodiment, the cloud phone can receive a test template corresponding to the application to be tested.

[0020] The cloud phone cluster can be composed of multiple cloud phones, capable of running environments of different architectures (such as ARM (Advanced RISC Machine), x86), different system versions. Each cloud phone can be equivalent to a remote device, with real application running capability. In addition, the cloud phone cluster can support elastic expansion, on-demand creation and destruction of instances, thereby realizing large-scale concurrent testing.

[0021] The test template set can include a series of test templates preset by the system and customized by the user. Each test template can define the operation sequence or scenario that the application needs to perform (such as "install -> start -> click on each Tab of the main interface -> exit"), as well as the expected check content. Among them, the test template can be defined by script or no-code configuration. The general test template can be adapted to any application (such as "installation and startup test", "Monkey random operation for 5 minutes"), and the special test template can be written by the developer for a specific application. Through the test template set, the test process can be standardized and reused on different applications.

[0022] The scheduling center can distribute test tasks to cloud phones in the cloud phone cluster to perform the pre-defined operation sequence corresponding to the test template. When receiving a test request, the scheduling center can select a suitable test template from the test template set according to the application type and test requirements, and select a suitable cloud phone environment (such as ARM architecture Android 13, 1080p screen) from the cloud phone cluster according to the application characteristics to deploy the application for testing. In addition, the application can also be deployed on multiple cloud phones in different environments and run tests in parallel to cover the compatibility matrix. For example, for a game application, select the "install-start-Monkey random test for 5 minutes" template, and allocate one cloud phone instance for each of Android 10 (ARM) and Android 13 (ARM64) to test in parallel.

[0023] Step 102, installing and starting the application to be tested.

[0024] In this embodiment, the cloud phone can install and start the application to be tested.

[0025] After receiving the test task, the cloud phone cluster can first start the cloud phone configured accordingly, including but not limited to setting the CPU (Central Processing Unit) architecture, memory, screen resolution, and pre-installing basic services, etc. Then the application to be tested is installed on the cloud phone through ADB (Android Debug Bridge) or a cloud phone management interface. After installation, the RPA (Robotic Process Automation) execution engine can start the application to be tested on the cloud phone.

[0026] Step 103, executing the pre-defined operation sequence in the test template in the application to be tested.

[0027] In this embodiment, the cloud phone can execute the pre-defined operation sequence in the test template in the application to be tested.

[0028] After the application to be tested is started, the RPA execution engine can execute the pre-defined operation sequence in the application to be tested according to the test template. For example, the test template requires clicking the login button after starting, inputting the test account, entering the main interface, and then performing Monkey random operation for several minutes. The RPA execution engine can execute these operations step by step while monitoring the response of the application.

[0029] Among them, the RPA execution engine can build a robot process automation module in the system, which is used to simulate human user operations on the cloud phone. The PRA execution engine can perform clicking, sliding, text input, etc. on the application interface through ADB instructions or control interfaces provided by the cloud phone, and can obtain screenshots, UI (User Interface) current state, etc. The RPA execution engine can drive the application to run according to the steps of the test template, while collecting interface screenshots and recording logs after key steps.

[0030] Step 104, in response to monitoring that the state of the application to be tested meets the preset condition, collecting the current screenshot.

[0031] In this embodiment, the cloud phone can monitor the state of the application to be tested. When the state of the application to be tested meets the preset condition, the cloud phone can collect the current screenshot.

[0032] During operation, the cloud phone can monitor the application state in real time. When a key event occurs or a checkpoint is reached, the RPA execution engine can capture the current screenshot and logs. The key event can be an event that each step of operation is completed, or an event that the application shows abnormal signs (such as not responding for more than a set time, detecting that the application process crashes, etc.). The logs can include, but are not limited to, application running logs, system logcat, performance data, etc. The screenshot can be completed through the cloud phone screenshot interface.

[0033] At step 105, the current screenshot is semantically understood to generate a test result of the application to be tested.

[0034] In this embodiment, the cloud phone can perform semantic understanding on the current screenshot to generate a test result of the application to be tested.

[0035] Generally, the current screenshot can be understood by using a visual language model to generate a test result. For example, a prompt corresponding to the test scene of the application to be tested is obtained, the prompt and the current screenshot are input into the visual language model, and a test result is output.

[0036] The preloaded visual language model can be called to perform inference calculation on the screenshot. For quality inspection scenes, the visual language model can be prompted with words and light trained to enable it to output customized analysis results. The visual language model can perform semantic understanding on the screenshot content, such as “Is there an error prompt in the interface?” or “Is the application frozen?” If there is text in the screenshot, the visual language model can try to read and judge the meaning; if the interface element layout is abnormal, the visual language model can determine the abnormality according to the training knowledge. Finally, the visual language model can output an analysis result, such as “Error popup detected, content is ‘network request failed’” or “interface is normal, no abnormalities”.

[0037] A visual language model can be deployed as a standalone analysis service. The service can load well-trained large model weights and have the ability to understand images and text. With optimized inference code and high-performance computing power support (such as GPU (Graphics Processing Unit)), the service can analyze a screenshot in a few seconds to determine whether the application is running normally. The visual language model can combine the screenshot and the text UI elements in it to produce an analysis result. For example, it can identify the prompt content in the screenshot (if an error dialog box appears, the visual language model can read the text), identify the interface structure (determine whether the page is stuck on the loading screen or blank), and detect UI element reactions (such as the page should change after a button click, and if there is no change, the visual language model can perceive the repeated screen). Since the visual language model includes a large amount of screenshot data of mobile or web interfaces in training and learns the positioning and semantic understanding of graphical interface elements (such as by generating screenshot descriptions and UI element annotations to help the model understand the meaning of the interface), the visual language model can "understand" the application interface screenshot and determine whether there is an abnormal situation. This semantic-level judgment cannot be achieved by traditional visual comparison.

[0038] In some embodiments, if the test result is normal, the cloud phone can continue to perform the next operation in the predefined operation sequence; if the test result is abnormal, the cloud phone can send the running information of the application to be tested to the scheduling center for abnormal processing.

[0039] The test logic where the cloud phone is located receives the test result and can determine whether an abnormality has occurred according to the test result. If the test result is abnormal, this step is recorded as a failure, and an abnormal processing process is triggered; if the test result is normal, the next operation or test process is continued. For example, if the visual language model returns "application crash", the cloud phone test can identify that the current operation causes a crash exception and enter the processing branch; if the visual language model returns "everything is normal", the script continues the next operation. For unexpected situations, such as application unresponsiveness, the visual language model may output "suspected to be stuck", and the test logic also regards it as an abnormal situation.

[0040] When an abnormality is detected, the cloud phone can collect more detailed information, including but not limited to saving the current screenshot, exporting error logs and stack (if it is a crash, the crash stack backtracking is captured; if it is UI lag, the thread state at that time is recorded), and sending these information to the scheduling center to determine whether to perform subsequent operations on this cloud phone. In general, once a serious abnormality occurs (such as a crash), the current test process can be terminated to avoid wasting time; for non-fatal problems (such as UI disorder but the application is still running), it can be determined whether to continue the subsequent steps. Regardless, the cloud phone will end the test and release the resources afterwards.

[0041] In some embodiments, if all operations in the predefined operation sequence are executed, the cloud phone can release the resources, send the test results to the scheduling center, so that the scheduling center generates a test report based on the test results.

[0042] If the test sequence is executed normally, the cloud phone can be recycled and released. The scheduling center can aggregate the results of all cloud phones and enter the report and closed-loop processing stage. Specifically, the scheduling center can collect report data from each cloud phone and generate a test report. The test report can include the pass or fail status of the application in each environment, a list of discovered problems, screenshot evidence and logs, and performance data statistics (such as average CPU usage, memory peak, etc.). Then, the system checks whether there are any abnormal findings. If all tests pass and there are no abnormalities, the results are recorded and the report is marked as passed. If there are abnormalities, at least one of the following actions is performed: generating a problem fix package, sending the test report to a problem tracking system for problem tracking, and testing and verifying after problem fixing.

[0043] The relevant information is automatically packaged into a problem reproduction package. The problem fix package can be in XAPK format, including but not limited to the problematic application APK, possible resource files, and detailed logs, screenshots, device information specification files, etc. XAPK is an extended application packaging format commonly used to carry OBB data files. Here, all necessary files are encapsulated using XAPK to facilitate developers or testers to download and quickly reproduce the problem locally or in other environments.

[0044] The abnormal report is submitted to the problem tracking system to notify the developer to check and fix. The test report can indicate what abnormality occurred under what device configuration and what operation was performed, as well as the intelligent analysis conclusion of the model (such as "the application appears blank when loading the friend list, suspected interface return abnormality"). The conclusion of the visual language model can help developers quickly understand the problem scenario.

[0045] A retest function is provided. When the developer fixes the problem and submits a new version of the application, the scheduling center can trigger the same test template to run in the same environment again (regression test) when the update is detected, to verify whether the problem is solved, thereby forming an automated closed loop. As long as the problem is not solved, it can continue to be marked as failed.

[0046] In addition, all test process data is also stored for analysis. After a large number of test cases are accumulated, the visual language model can be further trained or fine-tuned to be more accurate in identifying new abnormal patterns, or the test template set can be adjusted to cover new scenarios. This enables the system to evolve itself, with test coverage and intelligent judgment capabilities improving over time. The test process data can include, but is not limited to, application basic information, test environment configuration, executed operation steps, intercepted screenshots, model analysis results, logs, and performance indicators. These data can be used to generate detailed test reports. For example, a timeline of a test on a certain device records each operation step and its result judgment, which steps pass and which steps have exceptions, with screenshots and explanations. If a crash or serious error is detected, it can be highlighted and the log stack can be provided for developers to view. The report can be displayed in the form of a dashboard and can be exported. The data can be fed back to the scheduling center to determine subsequent actions (such as whether retesting is needed).

[0047] With reference to the foregoing Figure 2 FIG. 20 shows a flow 200 of yet another embodiment of an application testing method according to the present disclosure. The application testing method is applied to a scheduling center, including the following steps: Step 201, receiving a test request for an application to be tested.

[0048] In this embodiment, the scheduling center can receive a test request for an application to be tested.

[0049] The scheduling center can be responsible for test task scheduling, so when there is an application to be tested that needs to be quality inspected, a test request can be sent to the scheduling center. The test request can include test information of the application to be tested, such as an application package, test configuration, etc.

[0050] Step 202, determining a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested based on the test information.

[0051] In this embodiment, the scheduling center can determine a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested based on the test information.

[0052] The cloud phone cluster can be composed of multiple cloud phones, which can run environments of different architectures and different system versions. Each cloud phone can be equivalent to a remote device and has the ability to run real applications. In addition, the cloud phone cluster can support elastic expansion, on-demand creation and destruction of instances, thereby realizing large-scale concurrent testing.

[0053] The test template set can include a series of test templates preset by the system and customized by the user. Each test template can define a sequence of operations or a scenario that the application needs to perform, and what is expected to be checked. Among them, the test template can be defined by a script or a code-free configuration. A general test template can be adapted to any application, and a special test template can be written by a developer for a specific application. Through the test template set, the test process can be standardized and reused on different applications.

[0054] The scheduling center can distribute test tasks to cloud phones in the cloud phone cluster to perform the predefined operation sequence corresponding to the test template. When receiving a test request, the scheduling center can select a suitable test template from the test template set according to the application type and test requirements, and select a suitable cloud phone environment from the cloud phone cluster according to the application characteristics to deploy the application for testing. In addition, the application can also be deployed on multiple cloud phones in different environments and run tests in parallel to cover the compatibility matrix.

[0055] In step 203, the test template is sent to the cloud phone to test the application to be tested based on the test template.

[0056] In this embodiment, the scheduling center can send the test template to the cloud phone. The cloud phone can test the application to be tested based on the test template.

[0057] It should be noted that the detailed implementation process of the cloud phone testing the application to be tested based on the test template can refer to the embodiments shown in Figure 1 and will not be described here.

[0058] In some embodiments, the scheduling center can receive running information of the application to be tested; and perform abnormality processing based on the running information. Generally, if it is determined based on the running information that the abnormality level is higher than a preset level, the scheduling center can send a stop testing instruction to the cloud phone; and if it is determined based on the running information that the abnormality level is not higher than the preset level, the scheduling center can send a continue testing instruction to the cloud phone.

[0059] The test logic where the cloud phone is located receives the test result, and can determine whether an abnormality occurs according to the test result. If the test result is abnormal, this step is recorded as a failure, and an abnormality processing process is triggered; if the test result is normal, the next operation or test process is continued.

[0060] When an anomaly is detected, the cloud phone can collect more detailed information, including but not limited to saving the current screenshot, exporting error logs and stack, and sending these information to the scheduling center to determine whether to perform subsequent operations on the cloud phone. Generally speaking, once a serious anomaly occurs, the current test process can be aborted to avoid wasting time; for non-fatal problems, it can be set whether to continue the subsequent steps. In any case, the cloud phone will end the test and release the resources later.

[0061] In some embodiments, the scheduling center can receive the test results sent by the cloud phone; generate a test report based on the test results. Wherein, if the test report is abnormal, at least one of the following can be performed: generating a problem fixing package, sending the test report to a problem tracking system for problem tracking, and testing and verifying after the problem is fixed.

[0062] If the test sequence is executed normally, the cloud phone can be recycled and released. The scheduling center can summarize the results of all cloud phones and enter the report and closed loop processing stage. Specifically, the scheduling center can collect report data from each cloud phone to generate a test report. Wherein, the test report can include the pass or fail status of the application in each environment, the list of problems found, screenshot evidence and logs, and performance data statistics. Then, the system checks whether there are any abnormal findings. If all tests are passed, there is no exception, the result is recorded, and the report is marked as passed. If there are exceptions, at least one of the following actions is performed: generating a problem fixing package, sending the test report to a problem tracking system for problem tracking, and testing and verifying after the problem is fixed.

[0063] The relevant information is automatically packaged into a problem reproduction package. This problem fixing package can be in XAPK format, including but not limited to the problematic application APK, possible resource files, and detailed logs, screenshots, device information specification files, etc. Wherein, XAPK is an extended application packaging format commonly used to carry OBB data files. Here, all necessary files are encapsulated using XAPK to facilitate developers or testers to download and quickly reproduce the problem locally or in other environments.

[0064] Submit the exception report to the problem tracking system and notify the developer to check and fix. The test report can indicate what exception occurred under what device configuration and what operation was performed, as well as the intelligent analysis conclusion given by the model. The conclusion of the visual language model can help developers quickly understand the problem scenario.

[0065] Provide retest function. When the developer fixes the problem and submits a new version of the application, the scheduling center can detect the update and trigger the same test template to run in the same environment again to verify whether the problem is solved, thereby forming an automated closed loop. As long as the problem is not solved, it can continue to be marked as failed.

[0066] In addition, all test process data is also stored for analysis. After a large number of test cases are accumulated, the visual language model can be further trained or fine-tuned to be more accurate in identifying new abnormal patterns, or the test template set can be adjusted to cover new scenarios. This enables the system to evolve itself, with test coverage and intelligent judgment capabilities improving over time. The test process data can include, but is not limited to, application basic information, test environment configuration, executed operation steps, intercepted screens, model analysis results, logs, and performance indicators. These data can be used to generate detailed test reports.

[0067] Further reference is made to Figure 3 which shows an interaction timing 300 of one embodiment of the application test system according to the present disclosure. The application test system can include a scheduling center and a cloud phone cluster, and the following interaction steps are performed: Step 301, the scheduling center receives a test request of an application to be tested.

[0068] In this embodiment, the scheduling center can receive a test request of an application to be tested.

[0069] The scheduling center can be responsible for test task scheduling, so when there is an application to be tested that needs to be tested, a test request can be sent to the scheduling center. The test request can include test information of the application to be tested, such as application package, test configuration, etc.

[0070] Step 302, the scheduling center determines the test template corresponding to the application to be tested and the cloud phone corresponding to the application to be tested based on the test information.

[0071] In this embodiment, the scheduling center can determine the test template corresponding to the application to be tested and the cloud phone corresponding to the application to be tested based on the test information.

[0072] The cloud phone cluster can be composed of multiple cloud phones, which can run environments of different architectures and different system versions. Each cloud phone can be equivalent to a remote device with real application running capability. In addition, the cloud phone cluster can support elastic expansion, on-demand creation and destruction of instances, thereby realizing large-scale concurrent testing.

[0073] The test template set can include a series of test templates preset by the system and customized by the user. Each test template can define the operation sequence or scenario that the application needs to perform, and the content that is expected to be checked. The test template can be defined by a script or a no-code configuration. The general test template can be adapted to any application, and the special test template can be written by the developer for a specific application. Through the test template set, the test process can be standardized and reused on different applications.

[0074] The scheduling center can distribute the test task to the cloud phone in the cloud phone cluster to execute the predefined operation sequence corresponding to the test template. When receiving the test request, the scheduling center can select a suitable test template from the test template set according to the application type and test requirement, and select a suitable cloud phone environment from the cloud phone cluster according to the application characteristics to deploy the application for testing. In addition, the application can also be deployed on multiple cloud phones in different environments and run tests in parallel to cover the compatibility matrix.

[0075] In step 303, the scheduling center sends the test template to the cloud phone.

[0076] In this embodiment, the scheduling center can send the test template to the cloud phone.

[0077] In step 304, the cloud phone installs and starts the application to be tested.

[0078] In this embodiment, the cloud phone can install and start the application to be tested.

[0079] After receiving the test task, the cloud phone cluster can first start the cloud phone configured accordingly, including but not limited to setting CPU architecture, memory, screen resolution, and pre-installing basic services, etc. Then install the application to be tested to the cloud phone through ADB or cloud phone management interface. After installation, the RPA execution engine can start the application to be tested on the cloud phone.

[0080] In step 305, the cloud phone executes the predefined operation sequence in the test template in the application to be tested.

[0081] In this embodiment, the cloud phone can execute the predefined operation sequence in the test template in the application to be tested.

[0082] After starting the application to be tested, the RPA execution engine can execute the predefined operation sequence in the test template in the application to be tested, while monitoring the response of the application.

[0083] Among them, the RPA execution engine can build a robot process automation module in the system, which is used to simulate human user operation on the cloud phone. The PRA execution engine can execute click, swipe, text input, etc. on the application interface through ADB instruction or control interface provided by the cloud phone, and can obtain screenshot, UI current state, etc. The RPA execution engine can drive the application to run according to the steps of the test template, while collecting interface screenshots and recording logs after key steps.

[0084] In step 306, the cloud phone monitors the state of the application to be tested.

[0085] In this embodiment, the cloud phone can monitor the state of the application to be tested.

[0086] Step 307, in response to the preset condition being met, the cloud phone captures the current screenshot.

[0087] In this embodiment, the cloud phone can monitor the state of the application to be tested. When the state of the application to be tested meets the preset condition, the cloud phone can capture the current screenshot.

[0088] In the operation process, the cloud phone can monitor the application state in real time. When a key event occurs or a checkpoint is reached, the RPA execution engine can capture the current screenshot and log. The log can include, but is not limited to, application running log, system logcat, performance data, etc. The screenshot can be completed through the cloud phone screenshot interface.

[0089] Step 308, the cloud phone performs semantic understanding on the current screenshot to generate a test result of the application to be tested.

[0090] In this embodiment, the cloud phone can perform semantic understanding on the current screenshot to generate a test result of the application to be tested.

[0091] Generally, the current screenshot can be understood by using a visual language model to generate a test result. For example, a prompt word corresponding to the test scene of the application to be tested is obtained, the prompt word and the current screenshot are input into the visual language model, and the test result is output.

[0092] A preloaded visual language model can be called to perform inference calculation on the screenshot. For quality inspection scenes, the visual language model can be prompted with words and lightly trained to enable it to output customized analysis results. The visual language model can perform semantic understanding on the content of the screenshot. If there is text in the screenshot, the visual language model can try to read and determine the meaning; if the layout of the interface element is abnormal, the visual language model can determine the abnormality according to the training knowledge. Finally, the visual language model can output an analysis result.

[0093] Deploying a visual language model can serve as an independent analysis service. The service can load well-trained large model weights and have the ability to understand images and text. Through optimized inference code and high-performance computing power support, the service can analyze the screenshot in seconds to determine whether the application is running normally. The visual language model can combine the screenshot and the text UI elements in it to output an analysis result. Since the visual language model contains a large amount of screenshot data of mobile or web interfaces in training and learns the positioning and semantic understanding of graphical interface elements, the visual language model can "understand" the application interface screenshot and determine whether there is an abnormal situation. This semantic level judgment cannot be achieved by traditional visual comparison.

[0094] In some embodiments, if the test result is normal, the cloud phone can continue to perform the next operation in the predefined operation sequence; if the test result is abnormal, the cloud phone can send the running information of the application to be tested to the scheduling center for abnormal processing. Generally, if it is determined based on the running information that the abnormality level is higher than the preset level, the scheduling center can send a stop testing instruction to the cloud phone; if it is determined based on the running information that the abnormality level is not higher than the preset level, the scheduling center can send a continue testing instruction to the cloud phone.

[0095] The test logic where the cloud phone is located receives the test result, and can determine whether an abnormality occurs according to the test result. If the test result is abnormal, this step is recorded as a failure, and an abnormality processing flow is triggered; if the test result is normal, the next operation or test flow is continued.

[0096] When an abnormality is detected, the cloud phone can collect more detailed information, including but not limited to saving the current screen snapshot, exporting error logs and stack, and sending the information to the scheduling center to determine whether to perform subsequent operations on the cloud phone. Generally, once a serious abnormality occurs, the current test flow can be aborted to avoid wasting time; for non-fatal problems, it can be determined whether to continue the subsequent steps. Regardless, the cloud phone will end the test and release the resources subsequently.

[0097] In some embodiments, if all operations in the predefined operation sequence are performed, the cloud phone can release the resources, and send the test result to the scheduling center, so that the scheduling center generates a test report based on the test result. If the test report is abnormal, at least one of the following can be performed: generating a problem fixing package, sending the test report to a problem tracking system for problem tracking, and performing test verification after the problem is fixed.

[0098] If the test sequence is all normally performed, the cloud phone can be recycled and released. The scheduling center can summarize the results of all cloud phones, and enter a report and closed loop processing phase. Specifically, the scheduling center can collect report data from each cloud phone, and generate a test report. The test report can include the pass or fail status of the application in each environment, a list of problems found, screenshot evidence and logs, and performance data statistics. Then, the system checks whether there are any abnormal findings. If all tests are passed and there is no abnormality, the result is recorded, and the report is marked as passed. If there is an abnormality, at least one of the following actions is performed: generating a problem fixing package, sending the test report to a problem tracking system for problem tracking, and performing test verification after the problem is fixed.

[0099] The relevant information is automatically packaged into a problem reproduction package. This problem fix package can be in the XAPK format, including but not limited to the problematic application APK, possible resource files, and detailed logs, screenshots, device information specification files, etc. Among them, XAPK is an extended application packaging format commonly used to carry OBB data files. Here, all necessary files are encapsulated using XAPK, making it easy for developers or testers to download and quickly reproduce the problem locally or in other environments.

[0100] The exception report is submitted to the problem tracking system, notifying the developer to review the fix. The test report can indicate what exception occurred under what device configuration and what operation was performed, as well as the intelligent analysis conclusion given by the model. The conclusion of the visual language model can help developers quickly understand the problem scenario.

[0101] A retest function is provided. When the developer fixes the problem and submits a new version of the application, the scheduling center can trigger the same test template to run in the same environment again when the update is detected, verifying whether the problem is solved, thus forming an automated closed loop. As long as the problem is not solved, it can continue to be marked as failed.

[0102] In addition, all test process data is also stored for analysis. After a large number of test cases are accumulated, the visual language model can be further trained or fine-tuned to make it more accurate in identifying new abnormal patterns, or the test template set can be adjusted to cover new scenarios. This enables the system to evolve itself, with test coverage and intelligent judgment capabilities improving over time. Among them, the test process data can include but is not limited to application basic information, test environment configuration, executed operation steps, intercepted screenshots, model analysis results, logs and performance indicators, etc. These data can be used to generate detailed test reports.

[0103] Through the above process, the system realizes a quality assurance closed loop of "finding problems -> locating problems -> promoting fixes -> verifying fixes" without human intervention. It is worth emphasizing that the introduction of multi-modal large models greatly expands the boundaries of automated testing capabilities, enabling the system to understand the content of screenshots and make judgments close to humans. For example, when the application pops up a "login failure, please log in again" prompt, the traditional script may ignore it if it does not explicitly check this text, but AI analysis will identify it as an exception (the user is logged out), thus capturing the problem in a timely manner.

[0104] The disclosed embodiments provide a mobile application testing system based on a cloud phone cluster and a multimodal large model. By combining the virtualization capabilities of the cloud phone cluster with the intelligent analysis capabilities of a visual language model, this system enables fully automated quality testing of mobile applications. This fundamentally improves the coverage, accuracy, and efficiency of mobile application testing, meeting the high-quality delivery requirements of rapid iterations.

[0105] Compared with the prior art, the main innovations of the application testing system provided by the embodiments of the present disclosure are as follows: First, a multi-platform compatible testing architecture driven by cloud phones: This solution applies cloud phone virtualization technology to large-scale application testing, supporting a consistent testing platform for different system architectures and versions. On the one hand, by utilizing ARM architecture system array servers to build cloud phone clusters, applications can run directly on ARM hardware, achieving performance and compatibility close to that of real machines. On the other hand, x86 virtualization can also be combined to cover more architectures. The scheduling system can uniformly deploy cloud phones of different architectures, achieving "one set of test cases, multiple execution environments." This architecture solves the problem of unified testing of multiple architectures and systems for the first time. The containerized deployment of cloud phones also brings scalability, capable of creating and destroying test environments in seconds, enabling on-demand allocation and high resource utilization. Furthermore, this solution addresses the issue of cloud phones not being able to directly access UI elements. By combining this with the AI ​​visual analysis described later, it enables automated testing without an auxiliary debugging interface.

[0106] Second, intelligent screenshot recognition using a multimodal large model: This solution introduces visual language models into the application quality inspection field, developing a new method for identifying screenshot anomalies. Based on deep learning, the visual language model understands the natural language and visual semantics in screenshots. This eliminates the need to define matching templates or rules for each application, allowing it to be universally applicable to various interfaces. The language and vision model can be trained to read interface text, identify the status of UI components, and even infer whether an application has crashed based on visual cues. This presents a new perspective on automated testing: enabling AI to act as a "virtual tester" to review interfaces. For example, the visual language model can identify "app crashes" or "page loading animation remains unchanged for more than 30 seconds" and then output a corresponding judgment. This intelligent recognition method offers far greater accuracy and generalizability than hard-coded rules, catching many previously difficult-to-detect anomalies. Deploying the large model as a service and embedding it into the testing process is also an industry first. Through the appropriate prompt design, the visual language model output is structured, making it easier for programmers to interpret and make decisions. This innovation significantly reduces the workload of manual inspection and scripted assertions, ushering quality inspection into the AI-driven era.

[0107] Third, test process automation and regression closed loop: This solution breaks down the full process automation from testing to feedback of the application. In particular, an application quality inspection RPA pipeline is designed, covering the whole process of application discovery, automatic installation, automatic operation, exception detection, result collection, XAPK packaging, problem tracking and retesting. Through the task scheduling center and the result processing module, the system can automatically generate problem reproduction packages and notify relevant personnel, and automatically trigger regression testing verification after repair. This highly automated closed loop greatly shortens the problem life cycle and reduces human intervention. Especially the problem snapshot of XAPK packaging facilitates cross-environment sharing and reproduction, ensuring that the problem is "visible, portable, and reproducible". The above-mentioned closed loop automation process not only improves test efficiency, but also forms a feedback data loop, making it possible for the visual language model to continue learning, making the system more intelligent.

[0108] Fourth, configurable test templates and intelligent scheduling: In order to balance generality and customization, this solution proposes a test template configurable and automatic dispatch mechanism. By abstracting common test scenarios into templates and providing parameterization capabilities, test tasks can be generated in batches. The scheduling center can adaptively select test templates and cloud phone resources based on the properties of the application (such as type, historical problem data, etc.). For example, newly listed games can automatically apply the "installation + Monkey + resource loading check" test template, and financial applications can automatically apply the "login + main function traversal" test template. The scheduling algorithm can optimize the scheduling strategy based on real-time resources and historical execution, such as prioritizing the use of the last configuration found to verify problems, or increasing the number of tests for high-risk scenarios. This intelligent scheduling and templating makes testing more efficient and comprehensive, reducing the need for manual test plan development.

[0109] Fifth, performance data combined with AI evaluation: This solution not only relies on large models to analyze vision, but also integrates performance monitoring into the quality inspection process. During test execution, cloud phones can record CPU, memory, GPU, network and other indicators. These data are combined with AI evaluation. For example, if the visual language model finds a suspected interface freeze, it can also check if the CPU is soaring or the memory is abnormally increasing at that time, to distinguish between real application unresponsiveness and network slow loading. In the future, large models can be further trained to input multi-modal information of screenshots and performance values for comprehensive judgment. Establish performance baseline automatic comparison, after each version test is completed, the system automatically compares the performance indicators with the historical version, if significant degradation is found, it will be marked as a prompt. This makes quality inspection not only focus on "right or wrong", but also on "good or bad" (optimization / retrogression). This quality and performance combined detection approach provides more comprehensive protection for product experience.

[0110] The above innovations complement each other. The cloud phone multi-architecture testing solves the compatibility environment problem; the large model screenshot analysis solves the intelligent judgment problem; the automatic closed loop solves the efficiency and continuous integration problem.

[0111] Compared with the prior art, the main technical effects of the application test system provided by the embodiments of the present disclosure are as follows: First, the compatibility coverage is comprehensive, and the test is more reliable: by supporting different chip architectures and system versions through the cloud phone cluster, the scheme realizes unified execution of a set of test processes in multiple device environments. It can simulate a large number of real devices running tests concurrently in the cloud, covering fragmented ecosystems. This wide coverage capability allows the application to discover specific problems on various devices before release, reducing online accidents. In addition, the cloud phone environment has good isolation, each test starts from a clean initial state, the result reproducibility is high, and is not affected by the residual of the last test, improving the test reliability.

[0112] Second, highly automated, efficiency greatly improved: the entire quality inspection process realizes end-to-end automation without human intervention. Batch application testing, 7x24 hour uninterrupted operation is possible. With the help of cloud phone parallelism and automatic scheduling, the test efficiency is improved. Especially after the analysis, recording, reporting, packaging, notification, regression and a series of actions are all automatically completed, the problem turnaround time is reduced. Thus, faster version iteration and lower test labor cost are realized.

[0113] Third, intelligent abnormality identification, reducing missed detection and misjudgment: the multi-modal large model gives the system "sharp eyes", which can understand the text and visual clues on the interface to judge the application state. For example, when the application crashes, the model not only captures the application exit, but also identifies the error information in the crash dialog box; when the application is stuck, the model can notice that the interface has been stuck in the loading animation or the button click is invalid. AI acts as an observer in the test process, greatly reducing the risk of missed detection and reducing false positives (for example, UI subtle changes will not be mistaken as faults, because the model focuses on semantics rather than pixels). AI has been used in script maintenance and visual comparison in the industry, but the scheme goes further, using a large model to make quality judgments at the semantic level, making the accuracy and comprehensiveness of abnormality detection reach a new height.

[0114] Fourth, reduce script writing and adaptability: With the help of AI and general test templates, the system can quickly configure and deploy tests for new applications and scenarios without the need to write detailed scripts for each function. The general visual understanding ability of the AI model means that even if the application interface style is different, it can analyze and determine abnormalities without relying on hard-coded logic. Therefore, when the application is upgraded or the UI skin is replaced, the quality inspection system is still applicable and does not require extensive maintenance. AI judgment does not require maintenance rules and has stronger adaptability. For some unforeseen new errors, the system also has a higher probability of capturing them because the large model has the ability to generalize knowledge, even if it has never seen a specific text, it can infer its meaning (for example, seeing the "server exception" word can determine that it belongs to an error state).

[0115] Fifth, closed-loop quality feedback to improve product iteration efficiency: The automatic closed loop of this solution ensures that each problem found is recorded and verified in a timely manner. The report received by the developer not only has a description of the problem phenomenon, but also an AI cause analysis and complete reproduction environment, greatly accelerating debugging and positioning. After fixing, the system automatically returns to testing, making it clear whether the problem has been solved. This closed loop allows testing and development to form a positive interaction, avoiding possible omissions or repeated communication in traditional processes. The quality inspection system can also accumulate problem data over a long period of time to analyze which types of problems are most common, thereby guiding the development of code quality or testing priorities to continuously improve product quality.

[0116] Sixth, performance and function together for quality assurance: This system not only focuses on function correctness, but also pays attention to performance indicators. In the same test process, both functional abnormalities and performance abnormalities (such as memory spikes or frame rate drops) are checked. The report compares performance data with historical baselines to help identify performance degradation. Because performance data collection and anomaly detection are integrated, developers can have a more comprehensive understanding of the impact of version changes. In today's user experience-centric world, this can effectively ensure that the application is not only functionally correct but also smooth and efficient, improving user satisfaction.

[0117] Seventh, technical versatility and scalability: Although this solution is primarily focused on Android application quality inspection, its ideas are versatile and can be easily extended to other platforms or fields. For example, cloud phones can be extended to iOS cloud real machines or HarmonyOS virtual machines, and large model analysis can be generalized to desktop application screenshots, web pages, and other quality inspections. The entire architecture is modular, and the test template set can continuously add new scenarios, and the AI model can be replaced or upgraded. Therefore, this system has long-term evolution capabilities and will not become obsolete soon, providing sustainable value.

[0118] Eighth, reduce costs and protect privacy: using cloud phones instead of a large number of physical real machines can reduce hardware investment and maintenance costs. At the same time, the test is carried out in the cloud, and the test data is stored centrally, which is convenient for management and auditing. For application tests that require security, a private cloud phone cluster can also be deployed to protect data privacy, which is more controllable than testing applications on third-party real machine platforms. The AI analysis process in the system can also be completed locally (the model can be deployed offline), avoiding uploading application screenshots to a public cloud AI interface, thereby protecting sensitive information during the test process.

[0119] In summary, the system combines the advantages of virtualization and artificial intelligence, achieving compatibility coverage of multiple devices in breadth, intelligent problem insight in depth, high automation and closed-loop feedback in efficiency, and user experience indicators in performance. It is a next-generation mobile application testing solution. It can significantly improve the testing capacity and quality level of mobile applications, reduce user complaints and reputation loss caused by application defects, and has great practical value and promotion prospects for the mobile Internet industry.

[0120] Figure 4 A business flowchart of application testing is shown. As shown in Figure 4 , the business flow of application testing can include intelligence collection 401, full model compatibility detection 402, result presentation and verification 403, and data application 404. Among them, intelligence collection 401 can include discovering new applications, monitoring inventory applications, and aggregating application information. Full model compatibility detection 402 can include installation-running-updating, detecting crashes or stalls, screenshots, and identifying screenshot states with the help of AI. Result presentation and verification 403 can include multi-model detection results, application original screenshots, and manual review. Data application 404 can include RXAPK packaging, application market listing, and modifying device allocation rules.

[0121] Figure 5 A background architecture diagram of application testing is shown. As shown in Figure 5 , the background architecture of application testing can include RPA basic services 501, scheduling center 502, PRA background management 503, and unified data table 504. RPA basic services 501 can include executor machine room X86 and cloud phone cluster. The scheduling center 502 can create tasks and put them into the queue. Among them, the task queue stores all pending tasks according to priority inventory XAPK update, discovers new games, compatibility detection, and packs XAPK. The scheduling center 502 can also notify callbacks and other processing. PRA background management 503 can include PRA-application discovery, game management, PRA-compatibility testing, PRA-XAPK packaging, and PRA-XAPK update detection.

[0122] For PRA base service 501, the cloud phone cluster can be composed of multiple cloud phones, which can run different architectures and different system versions of the environment. Each cloud phone can be equivalent to a remote device with real application running capability. In addition, the cloud phone cluster can support elastic expansion, on-demand creation and destruction of instances, thereby realizing large-scale concurrent testing.

[0123] For the scheduling center 502, it can be responsible for receiving test requirements and distributing tasks to devices in the cloud phone cluster for execution. The scheduling center selects appropriate cloud phone environments according to the application characteristics to deploy the application for testing, and can simultaneously run tests in parallel on multiple different configuration devices to cover the compatibility matrix.

[0124] For RPA background management, the PRA execution engine can be managed. The RPA execution engine can have a robot process automation module built-in the system, which is used to simulate human user operations on the cloud phone. The PRA execution engine can perform clicking, swiping, text input, etc. on the application interface through ADB instructions or control interfaces provided by the cloud phone, and can obtain screenshots, UI current state, etc. The RPA execution engine can drive the application to run according to the steps of the test template, while capturing interface screenshots and recording logs after key steps.

[0125] For the unified data summary table 504, it can be used to collect and store various data in the testing process, including application basic information, test environment configuration, executed operation steps, intercepted screens, results obtained by model analysis, logs and performance indicators, etc. The system will generate detailed test reports based on these data.

[0126] The entire system can be integrated with the CI / CD (Continuous Integration / Continuous Delivery) process: when a new application version is uploaded or triggered at a fixed time, the scheduling center automatically starts testing, and after the analysis is completed, the results are fed back to the developer or management background, realizing continuous quality monitoring.

[0127] Figure 6 An application discovery diagram for application testing is shown. As shown in Figure 6 The application discovery steps for application testing are as follows: Step 601, allocate cloud machine to restore factory settings.

[0128] Step 602, mount the corresponding template.

[0129] Step 603, execute the script to crawl new games.

[0130] Step 604, whether the new game is detected.

[0131] Step 605, notify the server that the task is completed.

[0132] Step 606, the server reissues a task of discovering a new game.

[0133] Step 607, release the cloud machine.

[0134] Step 608, create a new template.

[0135] Step 609, execute a new game download task.

[0136] Step 610, present data to the background.

[0137] Step 611, send to compatibility detection on the application market.

[0138] In some embodiments, the implementation process of application testing can include the implementation process of the cloud phone cluster, the implementation process of the multi-modal AI analysis service, the implementation process of result processing and regression, and the implementation process of the scheduling center.

[0139] Figure 7A An implementation flowchart of the cloud phone cluster is shown. As shown in Figure 7A , the implementation process of the cloud phone cluster can include the following steps: Step 701, start a virtual phone instance.

[0140] Step 702, install the application to be tested.

[0141] Step 703, start the application and perform predefined operations.

[0142] Step 704, collect running logs and screenshots.

[0143] Step 705, send screenshots and logs to the AI analysis service.

[0144] Step 706, wait for analysis results.

[0145] Step 707, determine whether it is abnormal.

[0146] Step 708, collect abnormal information.

[0147] Step 709, continue to the next step or end the test.

[0148] Step 710, end the test and release the cloud phone instance.

[0149] Figure 7B An implementation flowchart of the multi-modal AI analysis service is shown. As shown in Figure 7B , the implementation process of the multi-modal AI analysis service can include the following steps: Step 711, receive a screenshot analysis request.

[0150] Step 712, load multi-modal model inference.

[0151] Step 713, identify interface state.

[0152] Step 714, return result to dispatch center.

[0153] Figure 7C An implementation flowchart of result processing and regression is shown. As shown, the implementation flow of result processing and regression can include the following steps: Figure 7C Step 715, summarize test results and report.

[0154] Step 716, whether there is an exception.

[0155] Step 717, generate XAPK problem package.

[0156] Step 718, notify development and record problem.

[0157] Step 719, record pass result.

[0158] Step 720, wait for repair version.

[0159] Step 721, repair version release triggers automatic regression test.

[0160] Figure 7D An implementation flowchart of the dispatch center is shown. As shown, the implementation flow of the dispatch center can include the following steps: Figure 7D Step 722, receive new application test request.

[0161] Step 723, select test template.

[0162] Step 724, issue test task to cloud phone.

[0163] Through the above flow, a quality assurance closed loop of "finding problems -> locating problems -> promoting repair -> verifying repair" is realized. In particular, the introduction of multi-modal large models greatly expands the ability boundary of automated testing, enabling the system to understand the content of screenshots and make judgments close to humans.

[0164] Further referring to Figure 8 , as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an application testing device, which corresponds to the method embodiment shown in Figure 1 , and the device can be applied in various electronic devices.

[0165] As Figure 8 ​​As shown, the application testing apparatus 800 in this embodiment is arranged in the cloud phone, and can include a receiving module 801, an installing module 802, an executing module 803, a collecting module 804, and an understanding module 805. The receiving module 801 is configured to receive a test template corresponding to an application to be tested. The installing module 802 is configured to install and start the application to be tested. The executing module 803 is configured to execute a predefined operation sequence in the test template in the application to be tested. The collecting module 804 is configured to collect a current screenshot in response to monitoring that a state of the application to be tested meets a preset condition. The understanding module 805 is configured to perform semantic understanding on the current screenshot to generate a test result of the application to be tested.

[0166] In this embodiment, the specific processes of the receiving module 801, the installing module 802, the executing module 803, the collecting module 804, and the understanding module 805 in the application testing apparatus 800 and the technical effects brought by the specific processes can be respectively referred to the specific processes of the receiving module 801, the installing module 802, the executing module 803, the collecting module 804, and the understanding module 805 and the technical effects brought by the specific processes in the method embodiments of the present disclosure. Figure 1 The related descriptions of steps 101-105 in the corresponding embodiments will not be repeated here.

[0167] In some optional implementation manners of this embodiment, the understanding module 805 is further configured to: perform understanding on the current screenshot by using a visual language model to generate the test result.

[0168] In some optional implementation manners of this embodiment, the understanding module 805 is further configured to: obtain a prompt word corresponding to a test scenario of the application to be tested; input the prompt word and the current screenshot into the visual language model to output the test result.

[0169] In some optional implementation manners of this embodiment, the application testing apparatus 800 further includes a test processing module configured to, if the test result is normal, continue to execute a next operation in the predefined operation sequence; and if the test result is abnormal, send running information of the application to be tested to a dispatch center for abnormal processing.

[0170] In some optional implementation manners of this embodiment, the application testing apparatus 800 further includes a test sending module configured to, if all operations in the predefined operation sequence are executed, release resources; and send the test result to the dispatch center, so that the dispatch center generates a test report based on the test result.

[0171] Further reference Figure 9 As an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an application testing apparatus, which corresponds to the method embodiment shown in Figure 2 The apparatus can be applied in various electronic devices.

[0172] As Figure 9As shown, the application test device 900 in this embodiment is arranged in a dispatch center, and can include a receiving module 901, a determining module 902, and a sending module 903. The receiving module 901 is configured to receive a test request of an application to be tested, wherein the test request includes test information of the application to be tested. The determining module 902 is configured to determine a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested based on the test information. The sending module 903 is configured to send the test template to the cloud phone, so that the cloud phone tests the application to be tested based on the test template.

[0173] In this embodiment, the specific processing of the receiving module 901, the determining module 902, and the sending module 903 in the application test device 900 and the technical effects brought by the specific processing can be respectively referred to Figure 2 The related description of steps 201-203 in the corresponding embodiment will not be repeated here.

[0174] In some optional implementation manners of this embodiment, the determining module 902 is further configured to select the test template corresponding to the application to be tested from a test template set and select the cloud phone corresponding to the application to be tested from a cloud phone set based on the test information.

[0175] In some optional implementation manners of this embodiment, the application test device 900 further includes an exception processing module configured to receive running information of the application to be tested and perform exception processing based on the running information.

[0176] In some optional implementation manners of this embodiment, the exception processing module is further configured to send a stop test instruction to the cloud phone if it is determined based on the running information that an exception level is higher than a preset level, and send a continue test instruction to the cloud phone if it is determined based on the running information that the exception level is not higher than the preset level.

[0177] In some optional implementation manners of this embodiment, the application test device 900 further includes a report generation module configured to receive a test result sent by the cloud phone and generate a test report based on the test result.

[0178] In some optional implementation manners of this embodiment, the application test device 900 further includes a problem processing module configured to perform at least one of the following if the test report is abnormal: generate a problem fixing package, send the test report to a problem tracking system for problem tracking, and perform test verification after problem fixing.

[0179] In the technical solution of the present disclosure, the acquisition, storage, and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0180] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0181] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0182] As shown in Figure 10 The device 1000 includes a computing unit 1001 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0183] Various components in the device 1000 are connected to the I / O interface 1005, including an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; the storage unit 1008, such as a magnetic disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0184] The computing unit 1001 can be various general purpose and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs various methods and processes described above, such as the application testing method. For example, in some embodiments, the application testing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded onto the RAM 1003 and executed by the computing unit 1001, one or more steps of the application testing method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 can be configured to perform the application testing method by any other suitable means, such as by means of firmware.

[0185] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0186] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0187] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0188] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0189] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0190] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0191] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved. This is not limited herein.

[0192] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. An application testing method, applied to cloud phones, comprising: Receive the test template corresponding to the application to be tested; Install and start the application to be tested; Executing a predefined operation sequence in the test template in the application to be tested; In response to monitoring that the state of the application to be tested meets a preset condition, capturing a current screenshot; Perform semantic understanding on the current screenshot to generate a test result of the application to be tested.

2. The method according to claim 1, wherein The performing semantic understanding on the current screenshot to generate a test result of the application to be tested includes: The current screenshot is understood using a visual language model to generate the test result.

3. The method according to claim 2, wherein: The utilizing a visual language model to understand the current screenshot and generate the test result includes: Obtaining prompt words corresponding to the test scenario of the application to be tested; The prompt word and the current screenshot are input into the visual language model, and the test result is output.

4. The method according to any one of claims 1 to 3, wherein The method further comprises: If the test result is normal, continue to execute the next operation in the predefined operation sequence; If the test result is abnormal, the running information of the application to be tested is sent to the dispatch center for abnormal processing.

5. The method according to claim 4, wherein The method further comprises: If all operations in the predefined operation sequence are completed, releasing resources; The test results are sent to the dispatch center, so that the dispatch center generates a test report based on the test results.

6. An application testing method, applied to a dispatch center, comprising: receiving a test request of an application to be tested, wherein the test request includes test information of the application to be tested; Based on the test information, determining a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested; The test template is sent to the cloud phone, so that the cloud phone tests the application to be tested based on the test template.

7. The method according to claim 6, wherein: The determining, based on the test information, a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested includes: Based on the test information, a test template corresponding to the application to be tested is selected from a test template set, and a cloud phone corresponding to the application to be tested is selected from a cloud phone set.

8. The method according to claim 6 or 7, wherein: The method further comprises: Receiving operation information of the application to be tested; Exception handling is performed based on the operation information.

9. The method according to claim 8, wherein The performing exception handling based on the operation information includes: If the abnormality level is determined to be higher than a preset level based on the operation information, a stop test instruction is sent to the cloud phone; If it is determined based on the operation information that the abnormality level is not higher than the preset level, a continue test instruction is sent to the cloud phone.

10. The method according to claim 6 or 7, wherein: The method further comprises: Receiving the test result sent by the cloud phone; A test report is generated based on the test results.

11. The method according to claim 10, wherein: The method further comprises: If the test report is abnormal, perform at least one of the following: generate a problem repair package, send the test report to a problem tracking system for problem tracking, and perform test verification after the problem is repaired.

12. An application testing device, provided in a cloud phone, comprising: A receiving module is configured to receive a test template corresponding to the application to be tested; An installation module, configured to install and start the application to be tested; an execution module, configured to execute a predefined operation sequence in the test template in the application to be tested; a collection module configured to collect a current screenshot in response to monitoring that the state of the application to be tested meets a preset condition; The understanding module is configured to perform semantic understanding on the current screenshot and generate a test result of the application to be tested.

13. An application testing device, located in a dispatching center, comprising: A receiving module is configured to receive a test request of an application to be tested, wherein the test request includes test information of the application to be tested; a determination module configured to determine, based on the test information, a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested; The sending module is configured to send the test template to the cloud phone, so that the cloud phone tests the application to be tested based on the test template.

14. An application testing system, comprising a cloud phone cluster and a dispatch center; and The scheduling center is configured to receive a test request of an application to be tested, wherein: The test request includes test information of the application to be tested; Based on the test information, determining a test template corresponding to the application to be tested and a cloud phone corresponding to the application to be tested; Sending the test template to the cloud phone; The cloud phone is configured to install and start the application to be tested; Executing a predefined operation sequence in the test template in the application to be tested; In response to monitoring that the state of the application to be tested meets a preset condition, capturing a current screenshot; Perform semantic understanding on the current screenshot to generate a test result of the application to be tested.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5 or 6-11.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5 or 6 to 11.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1-5 or 6-11.

Images