Interface automation test method and device, electronic equipment and medium

Abstract

The application relates to the technical field of computers and discloses an interface automatic testing method and device, electronic equipment and a medium, which receive a test case for a user interface, initialize a data record table in an execution process and run the test case, wherein the data record table at least comprises a completed step list; a current step of the test case is determined, and the current step is run; in response to the end of the current step running, a current screenshot page of a user interface corresponding to the current step is determined; target operation text of a next step is generated based on the current screenshot page; the target operation text is compared with each historical operation text in the completed step list, and the running result of the current step is determined according to the comparison result; the running result can be dynamically determined by using the historical text accumulated in the execution process, accurate judgment of the operation result is realized, and the automation degree of the test and the error detection efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to methods, apparatus, electronic devices, and media for automated interface testing. Background Technology

[0002] Graphical user interface automation refers to the use of programs to simulate human behavior on a device's graphical interface. By automatically controlling actions such as clicking, inputting, swiping, and asserting through scripts, it replaces manual labor in completing repetitive interface testing tasks. Its core value lies in freeing up manpower, improving efficiency, and reducing human error.

[0003] Currently, in GUI automation testing methods based on large models, the visual language model generates operation steps based on the interface screenshots and is executed by the terminal device. However, in actual execution, there are cases where the terminal device's execution engine reports the operation as successful, but the operation does not actually produce the expected effect on the interface. In related technologies, the simple fixed-number retry strategy is usually used, but it is still impossible to accurately judge the accuracy of the operation result. Summary of the Invention

[0004] This invention provides an automated interface testing method, apparatus, electronic device, and medium to solve the problem of being unable to accurately determine the accuracy of operation results.

[0005] In a first aspect, the present invention provides an automated interface testing method, the method comprising: Receive test cases for the user interface, initialize the data record table during execution and run the test cases. The data record table includes at least a list of completed steps, which is used to store the historical operation text corresponding to each completed step. Determine the current step of the test case and run the current step; In response to the completion of the current step, determine the current screenshot page of the user interface corresponding to the current step; Based on the current screenshot, generate the target operation text for the next step; The target operation text is compared with the historical operation texts in the list of completed steps, and the result of the current step is determined based on the comparison results.

[0006] This invention receives test cases and initializes a data record table, which at least contains a list of completed steps to store the historical operation text corresponding to each completed step. Then, it runs the test cases and determines the current step. After execution, it obtains the current screenshot of the user interface corresponding to the current step, generates the target operation text for the next step based on the screenshot, and finally compares the target operation text with the historical operation text in the list of completed steps. The result of the comparison is then determined. The beneficial effects are: by comparing the next operation text generated from the screenshot with the historical operation text, it can automatically detect whether the current step causes the interface state to revert or repeat existing operations, thereby effectively identifying problems such as abnormal loops, operation failures, or the interface not progressing as expected. Furthermore, it does not rely on preset expected results or manual verification; it can dynamically determine the running result using only the historical text accumulated during execution. By identifying operation steps that are reported as successful by the device but have not actually taken effect, combined with a degradation strategy layered by intent type and a circuit breaker protection mechanism, it achieves self-repair and self-adaptation in the automated test script generation process for graphical user interfaces, improving the robustness and success rate of script generation.

[0007] In one optional implementation, in response to the completion of the current step, determining the current screenshot page of the user interface corresponding to the current step includes: In response to the completion of the current step, the number of failures of the current step is accumulated, and if the number of failures of the current step is less than the preset number, the current screenshot page of the user interface corresponding to the current step is captured after a preset delay. If the number of failures in the current step is greater than or equal to the preset number, the current step is determined to have failed, and the test case is stopped.

[0008] This invention accumulates the number of failures for the current step upon completion. If this number is less than a preset number, a preset delay is applied before capturing a screenshot of the user interface corresponding to the current step. If the number of failures is greater than or equal to the preset number, the current step is determined to have failed, and the test case is stopped. Subsequently, the target operation text for the next step is generated based on the captured screenshot, and this text is compared with the historical operation texts stored in the list of completed steps. The result of the current step is determined based on the comparison. By introducing a failure count accumulation and delayed screenshot mechanism, misjudgments caused by brief interface flickering or response delays can be effectively avoided, improving the stability and accuracy of screenshot acquisition. At the same time, the test is terminated in a timely manner when the number of failures reaches a preset threshold, avoiding invalid loops or long-term suspensions and saving execution resources. Finally, by comparing historical operation texts, it is possible to automatically identify whether a step has rolled back or been repeated without manually setting expected values, thereby enhancing the robustness of the test and the ability to automatically identify errors.

[0009] In one optional implementation, based on the current screenshot page, the target operation text for the next step is generated, including: Based on the current screenshot page, update the list of completed steps and historical operation records in the data record table; Based on the updated list of completed steps and historical operation records, construct decision prompts for the current screenshot page. The decision prompts include at least one of the following: the test objective corresponding to the operation text of the current step, the historical execution record, and the failure warning prompt for the current step. Based on the decision prompts and the current screenshot, generate the target operation text for the next step.

[0010] When generating the target operation text for the next step, this invention first updates the list of completed steps and historical operation records in the data record table based on the current screenshot page. Then, it uses this updated information to construct a decision prompt for the current screenshot page. This prompt includes at least one or more of the following: the test objective corresponding to the operation text of the current step, the historical execution record, and the failure warning prompt for the current step. Finally, the target operation text for the next step is generated based on the decision prompt and the current screenshot page. By dynamically updating the historical execution record and integrating the test objective, historical trajectory, and failure warning into the decision prompt, the generated target operation text has sufficient contextual basis, effectively avoiding duplicate or invalid operations, and can adjust the execution strategy in advance based on the warning information. This improves the intelligence and adaptability of the test, while reducing the reliance on manual intervention and enhancing the reliability and execution efficiency of automated testing.

[0011] In one optional implementation, the target operation text is compared with the historical operation texts in the list of completed steps, and the result of the current step is determined based on the comparison result, including: The target operation text is compared with each first historical operation text in the list of completed steps. The first historical operation text is the historical operation text obtained by scanning forward a preset number of steps from the end of the list of completed steps. When the target operation text is determined to be consistent with any first historical operation text, the current step is determined to be a false success step, and the result of the current step is determined to be a false success. If the target operation text is found to be inconsistent with all first historical operation texts, the result of the current execution step is determined to be completed.

[0012] This invention compares the generated target operation text with each first historical operation text in the list of completed steps. The first historical operation text is obtained by scanning backwards from the end of the list of completed steps within a preset number of steps. If the target operation text matches any first historical operation text, the current step is determined to be a false success step, and its execution result is determined to be false success. If the operation text does not match any of the first historical operation texts, the execution result of the current step is determined to be completed. By comparing only the recent historical operations within the preset number of steps at the end of the list, rather than all the history, it can more efficiently and accurately capture the rollback or loop repetition of interface operations, avoid false alarms caused by matching historical data with a long span, and accurately distinguish between "false success" (i.e., the interface seems to advance but actually returns to a recently executed step) and true completion, thereby improving the accuracy and timeliness of test result determination and reducing unnecessary test interruptions or misjudgments.

[0013] In one alternative implementation, the method further includes: When the result of the current step is determined to be completed, the operation text of the current step is updated to the list of completed steps, and the next step is executed.

[0014] When the present invention determines that the current step has been completed according to the aforementioned comparison rules (i.e., the target operation text is inconsistent with all first historical operation texts), indicating that the current step has progressed effectively without regression, the operation text of the current step is updated to the list of completed steps (appended to the end), and the next step is executed. The beneficial effect of this process is that by promptly including verified and valid new operation texts in the list of completed steps, real-time synchronization between the list and the execution progress is ensured. This allows subsequent steps to perform a "scanning a preset number of steps backward from the end" comparison based on the latest and most complete recent historical trajectory, thereby continuously improving the accuracy of false success detection and ensuring that the testing process can proceed in an orderly and coherent manner. It avoids repeated execution, infinite loops, or state loss due to outdated lists, further enhancing the stability and adaptability of automated testing.

[0015] In an optional implementation, the method further includes: the data recording table further includes invalid operation records, which are used to record the cumulative invalid operation data of each false success step; If the result of the current step is determined to be a false success, update the invalid operation record and determine the operation type and operation intent of the current step; Based on the operation type and operation intent, when it is determined that there is a preset parameter degradation strategy for the current step, the parameters of the device operation corresponding to the current step are downgraded, and the current step is rerun based on the device operation after parameter downgrading.

[0016] When the current step is determined to be a false success, this invention further performs the following steps: updating the invalid operation record in the data record table (used to accumulate invalid operation data for each false success step), and determining the operation type and operation intent of the current step; then, based on the operation type and operation intent, determining whether there is a preset parameter degradation strategy for the current step; if so, downgrading the parameters of the device operation corresponding to the current step, and rerunning the current step based on the downgraded device operation; by attributing false success to operation type and intent, and introducing a parameter degradation retry mechanism, the testing system can adaptively adjust the operation intensity or method, avoiding repeated misjudgments of false success due to fixed parameters, thereby improving the real effectiveness of step execution; at the same time, the continuous updating of invalid operation records can provide data support for subsequent warnings or strategy optimization, reduce the waste of resources caused by blind retries, and enhance the fault tolerance and self-healing ability of automated testing in the face of dynamic interface changes.

[0017] In one optional implementation, based on the operation type and operation intent, when it is determined that a preset parameter degradation strategy exists for the current step, parameter degradation is performed on the device operation of the current step, including: Based on the operational intent, determine the operation type of the current step; When the operation type of the current step is determined to be an input type, the input method of the current step is downgraded based on the parameter downgrade strategy; When the operation type of the current step is determined to be click type, the number of consecutive false successes of click operation steps with the same operation intent is determined, and if the number of consecutive failures does not reach the preset number of false successes, the process returns to the step of determining the current screenshot page of the user interface corresponding to the current step.

[0018] This invention first determines the operation type of the current step based on the operation intent. If the type is determined to be an input type, the input method of the current step is downgraded according to the parameter downgrade strategy (e.g., downgraded from precise input to fuzzy or delayed input). If the type is determined to be a click type, the number of consecutive false successes of click operation steps with the same operation intent is counted. If the number of consecutive failures does not reach the preset number of false successes, the downgrade is not performed directly, but the process returns to the step of "determining the current screenshot page of the user interface corresponding to the current step" to retry the screenshot and subsequent comparison. Differentiated downgrade processing is adopted for input type and click type. Input type can directly adjust the input method to avoid abnormal interface response, while click type avoids erroneous downgrade due to occasional interface jitter or short delay by accumulating the number of consecutive false successes and allowing re-screening and retry. This ensures the effectiveness of the downgrade strategy, reduces unnecessary changes in operation parameters, improves the stability and retry efficiency of test execution, and enhances the adaptive processing capability for dynamic interface changes.

[0019] Secondly, the present invention provides an automated interface testing device, the device comprising: The receiving module is used to receive test cases for the user interface, initialize the data recording table during the execution process, and run the test cases. The data recording table includes at least a list of completed steps, which is used to store the historical operation text corresponding to each completed step. The step determination module is used to determine the current step of a test case and run the current step. The page determination module is used to determine the current screenshot page of the user interface corresponding to the current step in response to the completion of the current step. The generation module is used to generate the target operation text for the next step based on the current screenshot page; The result determination module is used to compare the target operation text with the historical operation texts in the list of completed steps, and determine the result of the current step based on the comparison results.

[0020] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first type of interface automated testing method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a second type of interface automated testing method according to an embodiment of the present invention; Figure 4 This is a structural block diagram of an automated interface testing device according to an embodiment of the present invention; Figure 5This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] As an optional application scenario of this invention, such as Figure 1 As shown, application 101 is installed in terminal device 110, and user 130 can interact with application 101 through terminal device 110 and / or access device of terminal device 110.

[0028] For example, application 101 can be any application that provides question-and-answer related services. For instance, application 101 could be a question-and-answer interactive application, such as a text-to-text application, an image-to-text application, etc. Figure 1 In the application scenario shown, if application 101 is active, the terminal device 110 can display the interface 102 of application 101. The interface 102 may include various pages that application 101 can provide, such as interactive pages, settings pages, query pages, etc.

[0029] In some embodiments, terminal device 110 is communicatively connected to server 120 to provide services to application 101. Terminal device 110 may be a mobile terminal, fixed terminal, or portable terminal, etc., including but not limited to mobile phones, desktop computers, laptop computers, multimedia tablets, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 may also support any type of interface, and server 120 may be various types of computing systems or servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0030] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this invention.

[0031] The embodiments of the present invention will now be described with reference to the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and various page designs are possible in practice. The various graphic elements on the page may have different arrangements and different visual representations; one or more elements may be omitted or replaced, and one or more other elements may also be present, without any limitation in the embodiments of the present invention. Furthermore, the embodiments described below primarily pertain to terminal device 110. It should be understood that the actions described relative to terminal device 110 can be performed by application 101 on terminal device 110, or can be performed by application 101 in conjunction with its server (e.g., server 120).

[0032] GUI (Graphical User Interface) automation refers to the use of programs to simulate human behavior on a device's (phone, tablet, PC, browser, etc.) graphical interface. By automatically controlling actions such as clicking, inputting, swiping, and asserting through scripts, it replaces manual labor in completing repetitive interface testing tasks. Its core value lies in freeing up manpower, improving efficiency, and reducing human error.

[0033] In existing GUI automation testing methods based on large model-based (LLM / VLM) approaches, the visual language model generates operation steps based on screenshots of the interface, which are then executed by the terminal device. However, in actual execution, there is a type of problem that is difficult to detect through conventional means: "false success." False success refers to a situation where the terminal device's execution engine reports the operation as successful (returning success=true), but the operation does not actually produce the expected effect on the interface. For example, when a button is clicked, the device locates the element and performs the click action, but due to reasons such as the element being obscured, the page not being fully loaded, or the element responding to the event but not triggering the expected page jump, the interface state does not actually change.

[0034] This "false success" can lead to serious chain reactions: when the large model sees no change on the page in the next round of decision-making, it will generate the same operation instruction again, forming an infinite loop of "operation, report success, page unchanged, operation again", wasting a lot of computing resources and execution time, and ultimately causing the script generation to fail.

[0035] Furthermore, when an operation does fail (the device returns success=false), existing solutions typically employ a simple fixed-number retry strategy, lacking differentiated handling for different operation types. For example, the root causes of click operation failures and input operation failures are often different, requiring different remedial measures rather than uniformly retrying the same instruction.

[0036] Therefore, a GUI automated testing method based on false success detection and adaptive multi-level degradation is proposed. By identifying operation steps that are reported as successful by the device but do not actually take effect, and combining a degradation strategy layered by intent type and a circuit breaker protection mechanism, the method achieves self-repair and self-adaptation in the GUI automated test script generation process, thereby improving the robustness and success rate of script generation.

[0037] According to an embodiment of the present invention, an embodiment of an automated interface testing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] This embodiment provides an automated interface testing method, which can be used on the aforementioned mobile terminals, such as mobile phones and tablets. Figure 2 This is a flowchart of an automated interface testing method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Receive test cases for the user interface, initialize the data record table during execution and run the test cases. The data record table includes at least a list of completed steps, which is used to store the historical operation text corresponding to each completed step.

[0039] It should be noted that the user interface refers to the graphical user interface. A test case is a pre-designed set of operation sequences, input data, and expected results for a graphical user interface, used to verify whether the interface function or interaction flow meets expectations. Among them, the test case is the starting input for automated execution, which contains multiple steps, each corresponding to a specific interface operation.

[0040] The user interface refers to the visual graphical interface in a software application that interacts with the user, including elements such as windows, buttons, input boxes, and menus. Test cases are performed on this interface, and the changes in the interface state are monitored through screenshots and operation text.

[0041] The completed steps list is the core substructure of the data record table, used to sequentially store the operation text corresponding to all steps that have been successfully run or determined to be completed. Historical operation text refers to the descriptive text generated or recorded after each executed step has finished running, such as clicking the login button or entering information in an input box; the specific input information is related to the test case.

[0042] Step S202: Determine the current step of the test case and run the current step.

[0043] It's important to clarify that "running the current step" refers to the process during the execution of automated graphical user interface (GUI) testing, where the device or browser simulates user interaction behavior to trigger changes in the user interface state for the currently executing operation node in the test case. Specifically, this action involves locating target elements on the interface, such as buttons or input fields, performing simulated operations like clicking or text input, and waiting for the interface to provide corresponding feedback. The current step is the automated script actually operating the GUI, not merely recording or planning a specific operation.

[0044] Step S203: In response to the end of the current step, determine the current screenshot page of the user interface corresponding to the current step.

[0045] It should be noted that the completion of the current step refers to running test cases to simulate user interaction behavior and complete various operations for the graphical user interface. These operations include at least clicking on the interface and entering text on the interface.

[0046] The process of determining the current screenshot page corresponding to the current step refers to the system instantly capturing the user interface operated in that step—the active window of the software under test—at that moment after the current test step has finished running. This captures the complete static visual image of the user interface at that instant by calling the screenshot interface of the operating system or browser. This action is not unconditionally executed but must be combined with the preceding logic. For example, if the number of failed runs is less than a preset number, the capture will be delayed for a preset time to ensure that the captured interface state has stabilized from the transition animation or brief refresh, avoiding invalid screenshots due to incomplete interface rendering. The captured screenshot page serves as the core basis for the entire subsequent processing flow. Its purpose is not for manual viewing but as visual input provided to the generative model or image recognition algorithm to dynamically generate the target operation text for the next step. It is also used for comparison with historical operation texts to support the automated determination of the running results.

[0047] Step S204: Based on the current screenshot page, generate the target operation text for the next step.

[0048] It should be noted that, driven by a visual language model, the current screenshot is first used as visual input and concatenated with pre-built decision prompts. These prompts include at least the test cases for the current step, completed historical execution records, and failure warnings for the current step. Based on the visual elements of the current interface and the task objective, the system infers and outputs the natural language operation command to be executed next. This target operation text can be interpretable instructions such as clicking the submit button in the upper right corner or entering a password in a password input box.

[0049] It should be noted that the operations between the current step and the next step are related in terms of their operational intent. That is, the next step's action planning is based on the feedback from the current step, ensuring that automated testing can dynamically adapt to real changes in the interface.

[0050] Step S205: Compare the target operation text with the historical operation texts in the list of completed steps, and determine the result of the current step based on the comparison results.

[0051] It's important to note that the "Completed Steps" list stores the historical operation text corresponding to all steps judged as completed in execution order, recording the actual progress of the test. For example, if the current step passes the technical test without errors during execution, but comparison reveals that the next step generated by it is a duplicate of the current step or a previously completed historical operation, this means the interface has actually regressed or entered a loop. The current step did not truly advance the test, and is therefore judged as a false success. "Completed" means that after the current step finishes execution, the generated next step is not a duplicate of the current step or a previously completed historical operation, proving that the interface state has entered a new or expected stage, and the current step has truly and effectively completed its task.

[0052] The interface automation testing method provided in this embodiment receives test cases and initializes a data recording table, which at least contains a list of completed steps to store the historical operation text corresponding to each completed step. Then, the test cases are run to determine the current step. After the run, a screenshot of the user interface corresponding to the current step is obtained, and the target operation text for the next step is generated based on this screenshot. Finally, the target operation text is compared with the historical operation texts in the list of completed steps, and the result of the current step is determined based on the comparison result. The beneficial effects are: by comparing the next operation text generated based on the screenshot with the historical operation text, it can automatically detect whether the current step causes the interface state to revert or repeat existing operations, thereby effectively identifying problems such as abnormal loops, operation failures, or the interface not progressing as expected. At the same time, it does not rely on preset expected results or manual verification; it can dynamically determine the running result using only the historical text accumulated during execution. By identifying operation steps that are reported as successful by the device but have not actually taken effect, combined with a degradation strategy layered by intent type and a circuit breaker protection mechanism, it achieves self-repair and self-adaptation in the process of generating automated test scripts for graphical user interfaces, improving the robustness and success rate of script generation.

[0053] This embodiment provides an automated interface testing method, which can be used on the aforementioned mobile terminals, such as mobile phones and tablets. Figure 3 This is a flowchart of an automated interface testing method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Receive test cases for the user interface, initialize the data recording table during execution, and run the test cases. The data recording table includes at least a list of completed steps, which stores the historical operation text corresponding to each completed step. For details, please refer to [link to details]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0054] Step S302: Determine the current step of the test case and run the current step. See details below. Figure 2 Step S202 of the illustrated embodiment will not be described again here.

[0055] Step S303: In response to the end of the current step, determine the current screenshot page of the user interface corresponding to the current step.

[0056] Specifically, step S303 includes: Step S3031: In response to the end of the current step, the number of failures of the current step is accumulated, and if the number of failures of the current step is less than the preset number, the current screenshot page of the user interface corresponding to the current step is captured after a preset delay.

[0057] It should be noted that if the number of failed runs recorded by the consecutive failure counter is less than the preset number, the system waits for the interface to stabilize (with a 2-second delay) before calling the terminal device screenshot interface to obtain a complete screenshot of the current interface as visual input for the GUI model. The preset number of failures can be between 2 and 5, but can be 3 for example.

[0058] Step S3032: When the number of failures in the current step is greater than or equal to the preset number, the current step is determined to have failed, and the test case is stopped.

[0059] It should be noted that at the beginning of each loop, the number of consecutive failures is checked first. If the number of failures recorded by the consecutive failure counter is greater than or equal to the preset number, the script generation is stopped immediately and the error message "N consecutive failures, circuit breaker stopped" is returned.

[0060] The interface automation testing method provided in this embodiment accumulates the number of failures for the current step when it ends. If the number is less than a preset number, a preset delay is made before capturing the current screenshot of the user interface corresponding to the current step. If the number of failures is greater than or equal to the preset number, the current step is directly determined to have failed and the test case is stopped. Subsequently, the target operation text for the next step is generated based on the captured screenshot and compared with the historical operation texts stored in the list of completed steps. The result of the current step is determined based on the comparison. By introducing a failure count accumulation and delayed screenshot mechanism, misjudgments caused by brief interface flickering or response delay can be effectively avoided, improving the stability and accuracy of screenshot acquisition. At the same time, the test is terminated in time when the number of failures reaches a preset threshold to avoid invalid loops or long-term suspension, saving execution resources. Finally, by comparing historical operation texts, it is possible to automatically identify whether a step has rolled back or been repeated without manually setting expected values, thereby enhancing the robustness and automated judgment capability of the test.

[0061] Step S304: Based on the current screenshot page, generate the target operation text for the next step.

[0062] Specifically, step S304 includes: Step S3041: Based on the current screenshot page, update the list of completed steps and historical operation records in the data record table.

[0063] It's important to clarify that updating the completed steps list and historical operation records in the data record table involves two aspects: first, appending the operation text of the current step to the end of the completed steps list, allowing the list to dynamically expand as execution progresses; second, synchronously updating the historical operation records. These records can include more granular execution status, timestamps, or screenshot features, ensuring consistency between the two types of storage in the data record table and reflecting the latest execution trajectory. The purpose of this update is to enable decision-making based on the complete historical trajectory, including the current step, when generating the target operation text for the next step. Whether the current step is ultimately completed or a false success will be determined after the target text for the next step is generated, by comparing it with the updated list.

[0064] Step S3042: Based on the updated list of completed steps and historical operation records, construct decision prompt words for the current screenshot page. The decision prompt words include at least one of the following: the test target corresponding to the operation text of the current step, the historical execution record, and the failure warning prompt of the current step.

[0065] It should be noted that the decision prompts for building the GUI model in the system include the following contextual information: test case target text; list of completed steps (last 10 steps); historical operation records (including failure records and reasons for failure); execution result of the previous step; list of declared variables.

[0066] The system filters operations with a cumulative invalid operation count greater than or equal to a preset number from the invalid operation cumulative table. The preset number can be 2 to 5 times, with 2 times being an example. A warning message is generated and a prompt is injected. The warning format is: "Important Warning: The following operation replay report is successful but it did not actually take effect. It must be replaced with 'Operation Text' (invalidated N times)." This warning enables the large model to proactively avoid known invalid operation paths when making decisions.

[0067] Step S3043: Based on the decision prompt words and the current screenshot page, generate the target operation text for the next step.

[0068] It should be noted that the pre-constructed text-based decision prompts, which integrate the current step's test objective, the latest historical execution records, and failure warning information, are used together with the currently captured real-world interface visual image (i.e., the current screenshot page) as joint inputs to the multimodal large model. Based on understanding the layout, state, and text context constraints of interface elements, the model uses reasoning to transform the abstract test task into a concrete, structured natural language action description, namely the "target operation text." This text indicates the specific interactive behavior to be performed in the next round (such as clicking a specific coordinate or inputting specific content), thus completing the crucial intelligent transformation from "observation" to "action decision." This ensures that the generated next operation not only fits the actual situation of the current interface but also conforms to the evolution direction of the test task, and provides parsable textual evidence for subsequent comparison with historical operations.

[0069] For example, the prompts for large model decisions refer to the complete input text dynamically constructed by the system for the GUI model in each iteration to drive its decision on the next operation. This includes the test case objective, a list of completed steps, historical operation records (including reasons for failure), the result of the previous step, and context such as declared variables. When the cumulative number of invalid operations (uniquely identified by its operation text) reaches a threshold (≥2 times), the system injects a warning message for that operation into the prompts for subsequent large model decisions, explicitly informing the model that "this operation has been invalidated multiple times and an alternative solution must be used," thereby guiding the model to proactively avoid the failed operation path.

[0070] The interface automation testing method provided in this embodiment first updates the list of completed steps and historical operation records in the data record table based on the current screenshot page when generating the target operation text for the next step. Then, the updated information is used to construct the decision prompt words for the current screenshot page. These prompt words contain at least one or more of the following: the test target corresponding to the operation text of the current step, the historical execution record, and the failure warning prompt for the current step. Finally, the target operation text for the next step is generated based on the decision prompt words and the current screenshot page. By dynamically updating the historical execution record and integrating the test target, historical trajectory, and failure warning into the decision prompt words, the generated target operation text has sufficient contextual basis, which can effectively avoid duplicate or invalid operations and adjust the execution strategy in advance based on the warning information. This improves the intelligence and adaptability of the test, while reducing the reliance on manual intervention and enhancing the reliability and execution efficiency of automated testing.

[0071] Step S305: Compare the target operation text with the historical operation texts in the list of completed steps, and determine the result of the current step based on the comparison results.

[0072] Specifically, step S305 includes: Step S3051: Compare the target operation text with each first historical operation text in the list of completed steps. The first historical operation text is the historical operation text obtained by scanning forward a preset number of steps from the end of the list of completed steps.

[0073] The first historical operation text contains the operation text of the current step. The preset number of steps can be two.

[0074] Since most GUI operations are not idempotent (the interface changes or jumps after it actually takes effect), the system re-captures the latest screenshot of the current interface before each round of decision-making, using it as the sole visual input for the large model. If the previous operation truly takes effect, the interface will inevitably change, and the large model will generate different next operations accordingly. Conversely, if the large model generates an instruction based on the latest screenshot that is completely identical to the text of the most recent step (backwards the last two steps) in the completed steps list, and the operation type is AI click or AI input, etc., which are AI-based location operations, the system uses this as the machine criterion for "the interface has not changed as expected," and infers that the previous operation with the same name was a "false success," meaning that although the device reported success, it did not actually take effect. The system automatically removes the false success step and its associated waiting steps from the completed steps list and records the operation text in the invalid operation accumulation table.

[0075] Step S3052: When it is determined that the target operation text is consistent with any first historical operation text, the current step is determined to be a false success step, and the result of the current step is determined to be a false success.

[0076] It should be noted that the system scans backward from the end of the list of completed steps, taking a preset number of steps and comparing them one by one. Once it is determined that the target text is completely consistent with any one of the recent history, the system considers that although the current step was executed without an error, the actual interface state it caused did not produce any substantial new progress. Instead, it brought the interface back to a state that had been recently accessed, causing the next step to repeat the operation that had been previously performed. This is considered a false success.

[0077] Step S3053: When it is determined that the target operation text is inconsistent with all first historical operation texts, the result of the current execution step is determined to be completed.

[0078] It should be noted that if the operation text of the next step is inconsistent with all the first historical operation texts, it means that the interface has entered a new state after the current step is executed, confirming that the step has not fallen into a loop or rolled back. Therefore, the result of the current execution step is determined to be completed.

[0079] The interface automation testing method provided in this embodiment compares the generated target operation text with each first historical operation text in the list of completed steps. The first historical operation text is obtained by scanning backwards from the end of the list of completed steps within a preset number of steps. If the operation text of the target step is found to be consistent with any first historical operation text, the current step is determined to be a false success step, and its execution result is determined to be false success. If the operation text is determined to be inconsistent with all first historical operation texts, the execution result of the current execution step is determined to be completed. By comparing only the recent historical operations within the preset number of steps at the end of the list, rather than all the history, it can more efficiently and accurately capture the rollback or loop repetition of interface operations, avoid false alarms caused by historical matching with too long a span, and accurately distinguish between "false success" (i.e. the interface seems to advance but actually returns to the recently executed steps) and real completion, thereby improving the accuracy and timeliness of test result determination and reducing unnecessary test interruptions or misjudgments.

[0080] Step S306: When it is determined that the result of the current step is completed, update the operation text of the current step to the list of completed steps, and execute the next step.

[0081] It should be noted that the next step should only be performed after confirming that the current step has been executed successfully. The above method can be used to verify the execution result of the next step again.

[0082] The interface automation testing method provided in this embodiment determines that the current step is complete according to the aforementioned comparison rules, i.e., the target operation text is inconsistent with all first historical operation texts, indicating that the current step has progressed effectively without regression. At this time, the operation text of the current step is updated to the list of completed steps (appended to the end), and the next step is executed. The beneficial effect of this process is that by timely including the new operation text that has been verified to be valid into the list of completed steps, the real-time synchronization between the list and the execution progress is ensured. This allows subsequent steps to perform a "scanning a preset number of steps backward from the end" comparison based on the latest and complete recent historical trajectory, thereby continuously improving the accuracy of false success detection and ensuring that the testing process can proceed in an orderly and coherent manner. It avoids repeated execution, infinite loops, or state loss caused by the list not being updated, further enhancing the stability and adaptability of automated testing.

[0083] The data logging table also includes invalid operation records, which are used to record the cumulative invalid operation data for each false success step; Step S307: When it is determined that the result of the current step is a false success, update the invalid operation record and determine the operation type and operation intent of the current step.

[0084] The operation type includes at least input operations and click operations; the operation intent corresponds to the operation type, that is, it includes at least input intent and click intent.

[0085] Step S308: Based on the operation type and operation intent, when it is determined that there is a preset parameter downgrade strategy for the current step, the parameters of the device operation corresponding to the current step are downgraded, and the current step is rerun based on the device operation after parameter downgrade.

[0086] For example, input operations (i.e., writing text to controls such as input boxes and text fields) will be downgraded from AI-based input (which requires semantic positioning to locate the input box before writing text) to direct input (which bypasses the element positioning process and allows the execution engine to directly write text to the currently focused control. This process does not rely on screen coordinates and does not require manual intervention).

[0087] Direct input refers to typing text using the keyboard, which does not involve clicking or triggering any navigation. In this embodiment, the focus for input operations is usually already on the input box, resulting in a high success rate for text input directly into the input box. Even if a mistake is made during direct input, false success detection can be achieved by taking a screenshot in the next round, allowing for automatic correction or triggering a shutdown.

[0088] The interface automation testing method provided in this embodiment, when determining that the result of the current step is a false success, further executes the following steps: updating the invalid operation records in the data record table (used to accumulate invalid operation data for each false success step), and determining the operation type and operation intent of the current step; then, based on the operation type and operation intent, determining whether there is a preset parameter degradation strategy for the current step; if so, downgrading the parameters of the device operation corresponding to the current step, and rerunning the current step based on the device operation after parameter downgrading; by attributing false success to operation type and intent, and introducing a parameter degradation retry mechanism, the testing system can adaptively adjust the operation intensity or method, avoiding repeated misjudgments of false success due to fixed parameters, thereby improving the real effectiveness of step execution; at the same time, the continuous updating of invalid operation records can provide data support for subsequent warnings or strategy optimization, reduce the waste of resources caused by blind retries, and enhance the fault tolerance and self-healing ability of automated testing in the face of dynamic interface changes.

[0089] Specifically, step S308 includes: Step a1: Based on the operation intent, determine the operation type of the current step.

[0090] When the current operation type is input, its intent is the corresponding input type intent; when the current operation type is click, its intent is the corresponding click type intent.

[0091] Step a2: When it is determined that the operation type of the current step is an input type, the input method of the current step is downgraded based on the parameter downgrade strategy.

[0092] In this process, AI operations other than click-based operations (such as AI input and AI assertions) are downgraded to intelligent input tasks, with the original operation description treated as a natural language task to be completed autonomously by the AI. The actions before and after downgrade belong to parallel branches at the same downgrade level, with no hierarchy between them; they are simply routed according to the original operation's intent type. Because the AI ​​autonomously executes and verifies the completion status, the probability of false success is significantly reduced compared to the ordinary AI step of "reporting success upon location."

[0093] Step a3: When the operation type of the current step is determined to be click type, determine the number of consecutive false successes of click operation steps with the same operation intent, and when the number of consecutive failures does not reach the preset number of false successes, return to the step of determining the current screenshot page of the user interface corresponding to the current step.

[0094] Among them, click-based operations are downgraded to intelligent clicks, where AI autonomously locates and clicks the target.

[0095] Among them, failure counting isolation based on intent type: failures of any intent type are counted, but the consecutive failure counter only increments when the same intent type fails again. Failures of different intent types reset the counter to 1 and do not accumulate, thus ensuring that only repeated failures of the same type of operation will trigger degradation and circuit breaking, avoiding occasional and scattered failures in normal diversified operation sequences from falsely triggering the protection mechanism.

[0096] The interface automation testing method provided in this embodiment first determines the operation type of the current step based on the operation intent. If the type is determined to be an input type, the input method of the current step is downgraded according to the parameter downgrade strategy (e.g., downgraded from precise input to fuzzy or delayed input). If the type is determined to be a click type, the number of consecutive false successes of click operation steps with the same operation intent is counted. If the number of consecutive failures does not reach the preset number of false successes, the downgrade is not performed directly, but the process returns to the step of "determining the current screenshot page of the user interface corresponding to the current step" to retry the screenshot and subsequent comparison. Differentiated downgrade processing is adopted for input type and click type. Input type can directly adjust the input method to avoid abnormal interface response, while click type avoids erroneous downgrade due to occasional interface jitter or short delay by accumulating the number of consecutive false successes and allowing re-screening and retry. This ensures the effectiveness of the downgrade strategy, reduces unnecessary changes in operation parameters, improves the stability and retry efficiency of test execution, and enhances the adaptive processing capability for dynamic interface changes.

[0097] An application embodiment of an automated interface testing method is provided, including: Step S1: System initialization and main loop startup: The system receives natural language test cases submitted by the user (such as "Open Settings, go to Display & Brightness, turn off Auto Brightness") and initializes the following core data structures: a list of completed steps, used to record all successfully executed operation steps; a cumulative table of invalid operations, a dictionary structure containing the operation text and the cumulative number of times the operation was judged as a false success; a consecutive failure counter, recording the current number of consecutive failures; and the previous failure intent, recording the type of the operation intent of the last failure. The system enters the main loop, executing one operation step in each loop iteration, with a maximum number of loop iterations of a preset limit (e.g., 30 steps).

[0098] Step S2: Screenshot Acquisition and Circuit Breaker Check: At the start of each loop, the circuit breaker condition is checked first: if the consecutive failure counter is ≥3, the circuit breaker protection is triggered, script generation is immediately stopped, and the error message "N consecutive failures, circuit breaker stopped" is returned. If the circuit breaker is not triggered, the system waits for the interface to stabilize (delayed for 2 seconds) and then calls the terminal device screenshot interface to obtain a complete screenshot of the current interface as the visual input for the GUI model.

[0099] Step S3: Construct the prompt word and inject an invalid operation warning: The system constructs decision prompts for the GUI model, including the following contextual information: test case target text; a list of completed steps (the most recent 10 steps); historical operation records (including failure records and reasons for failure); the result of the previous step; and a list of declared variables. Key innovation: The system filters operations with a cumulative count ≥2 from the invalid operation cumulative table and generates warning messages to inject as prompts. The warning format is: "Important Warning: The following operation replay report shows success but did not actually take effect." This warning enables the large model to proactively avoid known failed operation paths during decision-making.

[0100] Step S4: GUI Model Decision Making and False Success Detection The GUI visual language model receives the latest screenshot and prompt words of the current interface and outputs the next operation instruction (including fields such as intent type, target element, and value). Since each round of decision-making is based on the latest screenshot, if the previous operation has actually taken effect, the interface has changed, and the model will not output the same operation again. The system takes advantage of this by performing false success detection after obtaining the operation text (ai_text) output by the model: scanning backward from the end of the list of completed steps (the scanning range is the two most recent steps) to check if there are any completed steps that are identical to the current operation text. The judgment logic is: if there is a completed step that is completely identical to the text field of the current operation to be executed, and the action of that step is an AI operation type such as AI click or AI input, then the completed step is judged as a "false success" (i.e. the device reported success but the interface did not actually change as expected).

[0101] If a false success is detected: remove the false success step from the list of completed steps; check if a waiting step follows the false success step, and remove it if so; increment the count of the operation text in the invalid operation accumulation table; and log a warning.

[0102] Step S5: Operation execution and parameter-level degradation: The system maps operational intentions to specific device operations, calls the device's interface to construct step expressions, and then executes them through the interface. If execution fails and a parameter-level degradation strategy exists: for input-type operations, the system attempts alternative input methods. For example, if AI-based input positioning fails, it degrades to direct input, bypassing the element positioning process to directly input text at the current focus position. Each degradation strategy is tried one by one, and the process stops once any one succeeds.

[0103] Step S6: Intent-level downgrade determination and execution: If parameter-level downgrading fails to resolve the issue (or is not applicable), the system checks the intent-level downgrading conditions: Condition 1, the current operation belongs to an AI operation type (clickAI, inputAI, assertAI, longClickAI, doubleClickAI, rightClickAI, hoveringAboveAI, etc.); Condition 2, the consecutive failure counter is ≥2. When both conditions are met, the system performs targeted downgrading based on the intent type of the operation: Click-type intents (clickAI, longClickAI, doubleClickAI, rightClickAI, hoveringAboveAI) are downgraded to xAgentClick intelligent click, which uses an AI Agent to autonomously locate and perform the click target, with an execution timeout of 600 seconds; other intents (inputAI, assertAI, etc.) are downgraded to xAgentTask intelligent tasks, which treat the operation description as a natural language task and delegate it to the AI ​​Agent to complete autonomously, with an execution timeout of 600 seconds. When performing a downgraded operation, the system uses the description text of the original operation as the query parameter of the intelligent task to ensure that the semantics of the downgraded operation are consistent with the original operation.

[0104] Step S7: Execution result processing and failure count management: After the operation is completed, the system updates the status based on the execution result. If successful: the step is added to the completed steps list and assigned an incrementing step number; if the step is a click-type operation, a "wait 2 seconds" step is automatically added to ensure the interface stabilizes before proceeding to the next screenshot; the consecutive failure counter is reset to 0; and the previous failed intent record is cleared. If the operation fails (including failures after downgrading): the system compares the intent type of the current operation with the last failed intent (last_failed_intent); if they are the same (i.e., consecutive failures of the same intent type), the consecutive failure counter is incremented by 1; if they are different (i.e., failures of different intent types), the consecutive failure counter is reset to 1, and the last failed intent is updated to the current intent. This mechanism ensures that only consecutive failures of the same intent type will trigger downgrading and circuit breaking, preventing accidental failures of unrelated operations from mistakenly triggering the protection mechanism.

[0105] Step S8: Continue or terminate the loop: After completing the current step, the system returns to step S2 to continue the next loop. Loop termination conditions include: the GUI model outputs intent="done", indicating the test objective has been completed; circuit breaker protection is triggered (≥3 consecutive failures); the maximum number of steps is reached; or the user actively cancels the task. After the loop ends, the system assembles the list of completed steps into a savable automated test script.

[0106] The interface automation testing method provided in this embodiment has the following beneficial effects: It solves the deadlock problem caused by "false success": by detecting the repeated generation of the same operation by the detection model, it can be inferred that the previous operation did not really take effect, thus breaking the deadlock of "operation → report success → page remains unchanged → operate again".

[0107] It achieves precise feedback at the operational level: invalid operation warnings are directly injected into the decision prompts of the large model, enabling the model to avoid failure paths at the decision level, rather than passively retrying at the execution level.

[0108] Differentiated degradation based on intent type: Click-type operations and non-click-type operations use different degradation targets (xAgentClick vs xAgentTask), matching their respective failure characteristics to improve the degradation success rate.

[0109] Failure counting isolation avoids false triggers: the count is incremented only when the same intent type fails consecutively, and failures of different intent types do not interfere with each other, preventing normal and diverse operation sequences from being erroneously circuit-broken.

[0110] The three-layer progressive protection process—parameter-level degradation → intent-level degradation → global circuit breaking—forms a progressive protection from light to heavy, maximizing the success rate of script generation while preventing resource waste.

[0111] This embodiment also provides an automated interface testing device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0112] This embodiment provides an automated interface testing device, such as... Figure 4 As shown, it includes: The receiving module 401 is used to receive test cases for the user interface, initialize the data record table during the execution process and run the test cases. The data record table includes at least a list of completed steps, which is used to store at least the historical operation text corresponding to each completed step. The step determination module 402 is used to determine the current step of the test case and run the current step; Page determination module 403 is used to determine the current screenshot page of the user interface corresponding to the current step in response to the end of the current step. Module 404 is used to generate the target operation text for the next step based on the current screenshot page. The result determination module 405 is used to compare the target operation text with the historical operation texts in the list of completed steps, and determine the running result of the current step based on the comparison result.

[0113] In some alternative implementations, the page determination module 403 includes: The first running unit is used to respond to the end of the current step, accumulate the number of failures of the current step, and capture the current screenshot page of the user interface corresponding to the current step after a preset delay when the number of failures of the current step is less than a preset number.

[0114] The second execution unit is used to determine that the current step has failed and stop running the test case when the number of failures in the current step is greater than or equal to a preset number.

[0115] In some alternative implementations, the generation module 404 includes: The first generation unit is used to update the list of completed steps and historical operation records in the data record table based on the current screenshot page.

[0116] The second generation unit is used to construct decision prompts for the current screenshot page based on the updated list of completed steps and historical operation records. The decision prompts include at least one of the following: the test target corresponding to the operation text of the current step, the historical execution record, and the failure warning prompt of the current step.

[0117] The third generation unit is used to generate the target operation text for the next step based on the decision prompt words and the current screenshot page.

[0118] In some alternative implementations, the result determination module 405 includes: The first comparison unit is used to compare the target operation text with each first historical operation text in the list of completed steps, wherein the first historical operation text is the historical operation text obtained by scanning forward a preset number of steps from the end of the list of completed steps.

[0119] The second comparison unit is used to determine that the current step is a false success step and the result of the current step is a false success when the target operation text is consistent with any first historical operation text.

[0120] The third comparison unit is used to determine that the result of the current execution step is completed when the target operation text is inconsistent with all the first historical operation texts.

[0121] In some alternative embodiments, the device further includes: The execution module is used to update the operation text of the current step to the list of completed steps and execute the next step when it is determined that the result of the current step is completed.

[0122] The update module is used to update the invalid operation record when the result of the current step is determined to be a false success, and to determine the operation type and operation intent of the current step. The degradation module is used to downgrade the parameters of the device operation corresponding to the current step based on the operation type and operation intent, when it is determined that there is a preset parameter degradation strategy for the current step, and then rerun the current step based on the device operation with downgraded parameters.

[0123] In some alternative implementations, the degradation module includes: The first degradation unit is used to determine the operation type of the current step based on the operation intent; The second degradation unit is used to downgrade the input method of the current step based on the parameter degradation strategy when it is determined that the operation type of the current step is an input type. The third degradation unit is used to determine the number of consecutive false successes of click operation steps with the same operation intent when the operation type of the current step is determined to be click type, and to return to the step of determining the current screenshot page of the user interface corresponding to the current step when the number of consecutive failures has not reached the preset number of false successes.

[0124] The interface automated testing apparatus provided in this embodiment of the invention can execute the interface automated testing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0125] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0126] The following is a detailed reference. Figure 5This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0127] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0128] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the interface automated testing method of the embodiments of the present invention.

[0129] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0130] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the interface automation testing method shown in the above embodiments is implemented.

[0131] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0132] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for automated interface testing, characterized in that, The method includes: Receive test cases for the user interface, initialize a data record table during execution and run the test cases, wherein the data record table includes at least a list of completed steps, and the list of completed steps is used to store at least the historical operation text corresponding to each completed step; Determine the current step of the test case and run the current step; In response to the completion of the current step, determine the current screenshot page of the user interface corresponding to the current step; Based on the current screenshot page, generate the target operation text for the next step; The target operation text is compared with each historical operation text in the list of completed steps, and the result of the current step is determined based on the comparison result.

2. The method according to claim 1, characterized in that, The response to the completion of the current step, determining the current screenshot page of the user interface corresponding to the current step, includes: In response to the completion of the current step, the number of failures of the current step is accumulated, and when the number of failures of the current step is less than a preset number, the current screenshot page of the user interface corresponding to the current step is captured after a preset delay. If the number of failures in the current step is greater than or equal to the preset number, the current step is determined to have failed, and the test case is stopped.

3. The method according to claim 1, characterized in that, The step of generating the target operation text for the next step based on the current screenshot page includes: Based on the current screenshot page, update the list of completed steps and historical operation records in the data record table; Based on the updated list of completed steps and historical operation records, a decision prompt word for the current screenshot page is constructed. The decision prompt word includes at least one of the following: the test target corresponding to the operation text of the current step, the historical execution record, and the failure warning prompt of the current step. Based on the decision prompt and the current screenshot page, generate the target operation text for the next step.

4. The method according to claim 3, characterized in that, The step of comparing the target operation text with each historical operation text in the list of completed steps, and determining the execution result of the current step based on the comparison result, includes: The target operation text is compared with each first historical operation text in the list of completed steps, wherein the first historical operation text is the historical operation text obtained by scanning forward a preset number of steps from the end of the list of completed steps. When it is determined that the target operation text is consistent with any first historical operation text, the current step is determined to be a false success step, and the result of the current step is determined to be a false success. When it is determined that the target operation text is inconsistent with all first historical operation texts, the result of the current execution step is determined to be completed.

5. The method according to claim 1, characterized in that, The method further includes: When it is determined that the current step has been completed, the operation text of the current step is updated to the list of completed steps, and the next step is executed.

6. The method according to claim 1, characterized in that, The method further includes: the data recording table further includes invalid operation records, which are used to record the cumulative invalid operation data of each false success step; When it is determined that the result of the current step is a false success, the invalid operation record is updated, and the operation type and operation intent of the current step are determined. Based on the operation type and the operation intent, when it is determined that there is a preset parameter downgrade strategy for the current step, the parameters of the device operation corresponding to the current step are downgraded, and the current step is rerun based on the device operation after parameter downgrade.

7. The method according to claim 6, characterized in that, Based on the operation type and the operation intent, when it is determined that a preset parameter degradation strategy exists for the current step, parameter degradation is performed on the device operation of the current step, including: Based on the stated operational intent, determine the operation type of the current step; When the operation type of the current step is determined to be an input type, the input method of the current step is downgraded based on the parameter downgrade strategy; When the operation type of the current step is determined to be a click type, the number of consecutive false successes of click operation steps with the same operation intent is determined, and if the number of consecutive failures does not reach the preset number of false successes, the process returns to the step of determining the current screenshot page of the user interface corresponding to the current step.

8. An automated interface testing device, characterized in that, The device includes: A receiving module is used to receive test cases for the user interface, initialize a data recording table during execution, and run the test cases. The data recording table includes at least a list of completed steps, which stores at least the historical operation text corresponding to each completed step. The step determination module is used to determine the current step of the test case and run the current step; The page determination module is used to determine the current screenshot page of the user interface corresponding to the current step in response to the completion of the current step. The generation module is used to generate the target operation text for the next step based on the current screenshot page; The result determination module is used to compare the target operation text with each historical operation text in the list of completed steps, and determine the running result of the current step based on the comparison result.

9. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 7.

Images