A UI test script visual arrangement method and system based on reinforcement learning

Abstract

The application discloses a UI test script visual arrangement method and system based on reinforcement learning, relates to the technical field of user interface testing, and discloses a UI test script visual arrangement method based on reinforcement learning, which comprises the following steps: receiving an initial test path constructed by a user through a preset first visual arrangement interface; performing path optimization on the initial test path to obtain an optimized test path; displaying comparison information of the optimized test path and the initial test path on a preset second visual arrangement interface, and generating a UI test script in response to a user interaction operation received on the second visual arrangement interface. The application can efficiently and at low cost create and maintain a UI automatic test with clear business intention and stable execution.

Description

Technical Field

[0001] This application relates to the field of user interface testing technology, and in particular to a method and system for visualizing and orchestrating UI test scripts based on reinforcement learning. Background Technology

[0002] UI (User Interface) testing is an indispensable part of the modern software development lifecycle, aiming to improve testing efficiency, coverage, and product quality.

[0003] Currently, UI testing is generally done by testers writing scripts. However, this method relies heavily on the testers' experience, and the scripts often contain redundant steps, suboptimal navigation, and unnecessary waiting. They reflect the testers' exploration process rather than the optimal path required to verify a specific function. Furthermore, this method results in a high workload for testers and a long project testing cycle.

[0004] In summary, how to efficiently and cost-effectively create and maintain robust UI automated tests with clear business intent has become a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0005] The main purpose of this application is to provide a reinforcement learning-based visual orchestration method and system for UI test scripts, which aims to create and maintain UI automated tests with clear business intent and robust execution at a high efficiency and low cost.

[0006] To achieve the above objectives, this application proposes a reinforcement learning-based method for visualizing and orchestrating UI test scripts. This reinforcement learning-based method for visualizing and orchestrating UI test scripts includes:

[0007] The system receives the initial test path built by the user through a pre-defined first visual orchestration interface.

[0008] The initial test path is optimized to obtain the optimized test path;

[0009] The optimized test path and the initial test path are displayed in a preset second visual orchestration interface, and a UI test script is generated in response to user interaction operations received in the second visual orchestration interface.

[0010] In one embodiment, before the step of optimizing the initial test path to obtain an optimized test path, the method further includes:

[0011] Obtain an initial deep Q-network, wherein the state space of the initial deep Q-network is set to store the environment state when the test steps are executed in the preset application environment under test, and the action space of the initial deep Q-network is set to store various test path optimization operations;

[0012] Obtain training samples, wherein the training samples include multiple sets of training and testing paths;

[0013] The initial deep Q-network is trained based on the training samples to obtain a trained deep Q-network, which is used to optimize the initial test path.

[0014] In one embodiment, the step of training the initial deep Q-network based on the training samples to obtain a trained deep Q-network includes:

[0015] For each training and testing path in the training samples, the application environment under test is interacted with according to the training and testing path, and each experience tuple in the interaction process is obtained according to the preset reinforcement learning strategy. The experience tuple includes the current state, the action performed, the state at the next moment, and the composite reward at each moment.

[0016] Calculate the priority of each of the experience tuples, and store each of the experience tuples and its corresponding priority into a preset experience replay pool;

[0017] Based on the priority replay mechanism, sampled experience tuples are selected from the experience replay pool to iteratively train the initial deep Q network, resulting in a trained deep Q network.

[0018] In one embodiment, the step of optimizing the initial test path to obtain an optimized test path includes:

[0019] Based on the environmental state when the initial test path is executed in the preset application environment under test, determine the state information of each test step in the initial test path;

[0020] For each of the aforementioned test steps, the state information of the test step is analyzed using a deep Q-network to determine the optimal action after the test step.

[0021] The initial test path is iteratively updated based on each of the optimal actions to obtain an optimized test path.

[0022] In one embodiment, the step of determining the state information of each test step in the initial test path based on the environment state when the initial test path is executed in a preset application environment under test includes:

[0023] For each test step of the initial test path executed in the preset application environment under test, the visual state is extracted from the current user interface screenshot when each test step is executed.

[0024] The document object model or view hierarchy of the current user interface during the execution of each test step is vectorized to obtain the structural state;

[0025] The path status is obtained based on the position information of each test step in the initial test path;

[0026] Extract the preceding action embedding representation of each test step to obtain the historical action state;

[0027] Extract the state-action pairs of each test step to obtain the tail sequence state.

[0028] In one embodiment, the deep Q-network includes a convolutional neural network branch, a multilayer perceptron branch, a long short-term memory network branch, and a fully connected layer;

[0029] The step of analyzing the state information of each test step using a deep Q-network to determine the optimal action after the test step includes:

[0030] For each of the test steps, the visual state corresponding to the test step is used as the input of the convolutional neural network branch, the structural state and path state corresponding to the test step are used as the input of the multilayer perceptron branch, and the historical action state and tail sequence state corresponding to the test step are used as the input of the long short-term memory network branch.

[0031] The outputs of the convolutional neural network branch, the multilayer perceptron branch, and the long short-term memory network branch are concatenated to obtain a fused feature vector.

[0032] The fused feature vector is input into the fully connected layer to obtain the optimal action after the test step output by the fully connected layer.

[0033] In one embodiment, the first visual orchestration interface includes natural language command blocks, and the step of receiving the initial test path constructed by the user through the preset first visual orchestration interface includes:

[0034] In response to the user's drag-and-drop operation and parameter configuration operation on the natural language instruction block in the first visual orchestration interface, an initial test path is generated.

[0035] In one embodiment, the step of displaying comparison information between the optimized test path and the initial test path on a preset second visual orchestration interface, and generating a UI test script in response to user interaction received on the second visual orchestration interface, includes:

[0036] In the preset second visual arrangement interface, the optimized test path and the initial test path are distinguished by different colors or styles;

[0037] The steps that were optimized out in the initial test path are visually grayed out, and the reasons for the optimization are displayed in response to the first user interaction operation.

[0038] In response to a second user action, the optimized test path and / or the initial test path are modified to generate a target test path;

[0039] A UI test script is generated based on the target test path.

[0040] Furthermore, to achieve the above objectives, this application also proposes a reinforcement learning-based UI test script visualization orchestration system, which includes:

[0041] The visual orchestration module is used to receive the initial test path built by the user through a preset first visual orchestration interface;

[0042] A reinforcement learning optimization engine is used to optimize the initial test path to obtain an optimized test path.

[0043] The visualization orchestration module is also used to display the comparison information between the optimized test path and the initial test path on a preset second visualization orchestration interface, and to generate a UI test script in response to user interaction operations received on the second visualization orchestration interface.

[0044] In one embodiment, the reinforcement learning optimization engine includes:

[0045] The status determination unit is used to determine the status information of each test step in the initial test path based on the environmental status when the initial test path is executed in the preset application environment under test.

[0046] An action determination unit is used to analyze the state information of each test step using a deep Q-network to determine the optimal action after the test step.

[0047] The path optimization unit is used to iteratively update the initial test path according to each of the optimal actions to obtain an optimized test path.

[0048] This application proposes a reinforcement learning-based method for visualizing and orchestrating UI test scripts. The method receives an initial test path constructed by the user through a preset first visual orchestration interface; optimizes the initial test path to obtain an optimized test path; displays a comparison between the optimized test path and the initial test path on a preset second visual orchestration interface; and generates a UI test script in response to user interaction received on the second visual orchestration interface.

[0049] In summary, this application, by utilizing a first visual orchestration interface, reduces the reliance on testers' programming skills and scripting experience, allowing testers to intuitively construct initial test paths. It then uses reinforcement learning algorithms to automatically optimize these initial test paths, removing redundant steps, optimizing navigation, and reducing unnecessary waiting times, generating optimized test paths to verify specific functions and improve testing efficiency. Finally, a second visual orchestration interface provides a clear comparison between the initial and optimized test paths, enabling users to clearly understand the effectiveness of path optimization. Furthermore, based on user confirmation or further adjustment instructions, the final UI test script is generated. Therefore, this application enables the efficient and cost-effective creation and maintenance of robust UI automated tests with clear business intent. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating an embodiment of the reinforcement learning-based UI test script visualization orchestration method of this application.

[0053] Figure 2 This is a schematic diagram of an example scenario provided in Embodiment 1 of the UI test script visualization orchestration method based on reinforcement learning in this application;

[0054] Figure 3 This is a flowchart illustrating Embodiment 2 of the UI test script visualization orchestration method based on reinforcement learning provided in this application;

[0055] Figure 4 This is a flowchart illustrating Embodiment 3 of the UI test script visualization orchestration method based on reinforcement learning provided in this application;

[0056] Figure 5 This is a schematic diagram of the module structure of the UI test script visualization orchestration system based on reinforcement learning, as described in an embodiment of this application.

[0057] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0059] Currently, UI testing is generally done by testers writing scripts. However, this method relies heavily on the testers' experience, and the scripts often contain redundant steps, suboptimal navigation, and unnecessary waiting. They reflect the testers' exploration process rather than the optimal path required to verify a specific function. Furthermore, this method results in a high workload for testers and a long project testing cycle.

[0060] In addition, it's worth mentioning that while some other UI testing solutions have been proposed, such as recording and playback (which records user actions on the interface and automatically generates repeatable scripts to quickly create simple test cases); no-code / low-code platforms (which provide graphical user interfaces that allow users to orchestrate test processes by dragging and dropping preset instruction blocks or using natural language without writing code, enabling non-technical personnel to participate in automated testing); AI-based autonomous test generation (which uses AI (Artificial Intelligence) algorithms to allow intelligent agents to autonomously explore the application under test and generate test cases, guiding the agents to learn strategies for discovering defects or meeting specific test objectives); and visual testing (which focuses on verifying the correctness of the UI's visual presentation by capturing application screenshots and using computer vision and AI technologies to perform pixel-level comparisons with pre-approved baseline images to detect unexpected changes in layout, color, font, etc.).

[0061] However, each of them has significant drawbacks, which together constitute a series of technical problems that urgently need to be solved:

[0062] The prevalent high maintenance costs and test vulnerabilities: Manually written and recorded scripts are extremely fragile. They heavily rely on the UI's DOM (Document Object Model) structure and element locators; even minor UI changes can cause numerous test cases to fail. This results in enormous maintenance costs, often offsetting the benefits of automation, and is widely recognized as the biggest pain point in the industry.

[0063] The inherent inefficiency of test paths: Whether manually written, recorded and replayed, or orchestrated through no-code platforms, test paths are essentially direct reproductions of human operational logic. These paths often contain redundant steps, suboptimal navigation, and unnecessary waits, reflecting the tester's exploration process rather than the optimal path required to verify a specific function. Existing tools generally lack the ability to perform logical-level global optimization of created paths.

[0064] Limitations of No-Code / Low-Code Platforms: While these platforms simplify the test creation process, they do not address the efficiency issues of test paths. Inefficient paths orchestrated by the user are executed directly. Their "self-healing" features are merely passive, localized locator fixes, failing to fundamentally optimize the logic of the test process.

[0065] Challenges of AI-Generated Test Cases: Relying entirely on AI to generate tests from scratch presents significant challenges. First, reinforcement learning encounters state space explosion when exploring unconstrained UI environments, making it difficult to scale to complex applications. Second, generated test cases often lack clear business intent and interpretability, making them difficult for humans to understand and maintain. Furthermore, code generated based on large language models may contain "test smells" that do not conform to engineering best practices.

[0066] In summary, how to efficiently and cost-effectively create and maintain robust UI automated tests with clear business intent has become a pressing technical problem that needs to be solved in this field.

[0067] This application provides a solution that receives an initial test path constructed by a user through a preset first visual orchestration interface; optimizes the initial test path to obtain an optimized test path; displays a comparison between the optimized test path and the initial test path on a preset second visual orchestration interface; and generates a UI test script in response to user interaction operations received on the second visual orchestration interface.

[0068] In summary, this application, through its first visual orchestration interface, reduces the reliance on testers' programming skills and scripting experience, allowing testers to intuitively construct initial test paths. Reinforcement learning algorithms are then used to automatically optimize these initial test paths, removing redundant steps, optimizing navigation, and reducing unnecessary waiting times, generating optimized test paths to verify specific functions and improve testing efficiency. Finally, a second visual orchestration interface provides a clear comparison between the initial and optimized test paths, enabling users to clearly understand the effectiveness of path optimization. Furthermore, based on user confirmation or further adjustment instructions, the final UI test script is generated. Therefore, this application enables the efficient and cost-effective creation and maintenance of robust UI automated tests with clear business intent.

[0069] It should be noted that the executing entity in this embodiment can be an electronic device with data processing, network communication and program execution functions, such as a computer, tablet computer, server, etc., or an electronic device capable of realizing the above functions.

[0070] Based on this, embodiments of this application provide a UI test script visualization orchestration method based on reinforcement learning, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the reinforcement learning-based UI test script visualization orchestration method of this application.

[0071] In this embodiment, the UI test script visualization orchestration method based on reinforcement learning includes steps S10~S30:

[0072] Step S10: Receive the initial test path constructed by the user through the preset first visual orchestration interface;

[0073] It should be noted that the reinforcement learning-based UI test script visualization orchestration method in this embodiment can be applied to a reinforcement learning-based UI test script visualization orchestration system. This system includes a visualization orchestration module and a reinforcement learning optimization engine. The visualization orchestration module is used to orchestrate, receive, parse, display and manage UI test paths; the reinforcement learning optimization engine is responsible for analyzing UI test paths and proposing optimization suggestions.

[0074] The visual orchestration module includes a first visual orchestration interface, which is used to interact with the user and receive the initial test path constructed by the user. The initial test path can be rendered into a visual flowchart in real time, thereby transforming the complex test script creation process into an intuitive, easy-to-understand, and code-free visual operation.

[0075] In one feasible embodiment, the first visual orchestration interface includes natural language instruction blocks, and step S10 may include step S101:

[0076] Step S101: In response to the user's drag-and-drop operation and parameter configuration operation on the natural language instruction block in the first visual orchestration interface, an initial test path is generated.

[0077] It should be noted that in this embodiment, the visual orchestration module provides a natural language command library that encapsulates UI operations. Users do not need to concern themselves with the underlying code implementation; they only need to select commands from the library that match their intent, such as "enter text," "click the button," or "verify element existence." This method utilizes Natural Language Processing (NLP) technology to map human-readable commands to executable test actions, significantly lowering the barrier to test automation and enabling non-technical personnel such as product managers and business analysts to participate in test case creation.

[0078] The first visual orchestration interface provides a dynamic, interactive canvas where users can organize and arrange the aforementioned natural language instruction blocks through simple drag-and-drop operations, thereby constructing a complete test flow. In this interface, each node represents a specific UI operation (e.g., "Enter 'AA' in the 'Username' input box"), and allows users to configure relevant parameters. This visualization enables the internal logic of the test script to be intuitively constructed and understood.

[0079] Step S20: Optimize the initial test path to obtain the optimized test path;

[0080] By using reinforcement learning techniques, the initial test path constructed by the user is optimized. The optimization aims to reduce redundant and inefficient steps in the initial test path, resulting in an optimized test path that makes the final generated UI test script execute faster and consumes less computing resources.

[0081] Step S30: Display the comparison information between the optimized test path and the initial test path on the preset second visual orchestration interface, and generate a UI test script in response to the user interaction received on the second visual orchestration interface.

[0082] The second visual orchestration interface displays a comparison between the optimized and initial test paths. This presentation method prioritizes readability and ease of understanding, allowing users to quickly grasp the differences between the two. Furthermore, this interface incorporates a robust interactive response mechanism. When it receives user-triggered interactive actions (such as clicking "confirm" or modifying parameters), it rapidly invokes internal processing modules to generate UI test scripts according to predetermined algorithms and processes.

[0083] The layout of the second visual arrangement interface can be the same as that of the first visual arrangement interface; this embodiment does not impose any specific limitations on this.

[0084] In one feasible embodiment, step S30 may include steps S301 to S304:

[0085] Step S301: In the preset second visual arrangement interface, use different colors or styles to distinguish between the optimized test path and the initial test path;

[0086] Step S302: Visually gray out the steps that were optimized out in the initial test path, and display the reason for optimization in response to the first user interaction operation;

[0087] Step S303: In response to the second user operation, modify the optimized test path and / or the initial test path to generate the target test path;

[0088] Step S304: Generate UI test scripts based on the target test path.

[0089] In this embodiment, after the user completes path orchestration and requests optimization, the optimized test path generated by the reinforcement learning engine will be displayed as a suggestion, side-by-side or overlaid with the user's original test path on the second visual orchestration interface. In this interface, the initial test path and the optimized test path will be distinguished by different colors or styles, making the differences immediately apparent; optimized steps (e.g., redundant waits or unnecessary page jumps) will be visually "grayed out" in the initial test path; when the user hovers the mouse over an optimization point, the system will display a tooltip explaining the reason for the optimization in natural language, such as: "This step is redundant because the UI element 'username input box' is already in the target state" or "A faster navigation path has been found, skipping the three intermediate steps, with an estimated 50% reduction in execution time"; the user has complete decision-making power regarding which version of the path to adopt. The second visual orchestration interface provides clear controls, allowing the user to accept the optimized path or retain its original orchestration version, ultimately confirming the production of the UI test script.

[0090] For example, such as Figure 2 As shown, in the first visual orchestration interface, the user constructs a test case by dragging and dropping a series of UI operation instructions encapsulated in natural language command blocks (e.g., "click the 'Login' button"). This orchestration process is rendered as a visual flowchart in real time. After the user completes the orchestration of the test path, the system passes the orchestrated initial test path as input to the reinforcement learning optimization engine. The optimization engine explores the user-defined initial test path in the interactive environment with the application under test (AUT). By learning optimization strategies, the engine identifies and eliminates redundant steps, merges multiple operations, or replaces them with more efficient action sequences, while ensuring that the functional goals and core coverage of the original test case are not changed.

[0091] All optimization suggestions will be presented to users in a visual manner, specifically through a second visual orchestration interface that displays a comparison between the optimized test path and the initial test path. Users can review each suggestion and decide whether to accept or reject it, thus ensuring the user's ultimate control. The system's final output is a brand-new UI test script that is optimized, has higher execution efficiency, and is more maintainable.

[0092] Thus, in this embodiment, the reliance on testers' programming skills and scripting experience is reduced by utilizing a first visual orchestration interface, allowing testers to intuitively construct initial test paths. Reinforcement learning algorithms are then used to automatically optimize these initial test paths, removing redundant steps, optimizing navigation, and reducing unnecessary waiting times, generating optimized test paths to verify specific functions and improve testing efficiency. Finally, a second visual orchestration interface provides a clear comparison between the initial and optimized test paths, enabling users to clearly understand the effectiveness of path optimization. Furthermore, based on user confirmation or further adjustment instructions, the final UI test script is generated. Therefore, this application enables the efficient and low-cost creation and maintenance of robust UI automated tests with clear business intent.

[0093] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. On this basis, as Figure 3 As shown, steps A10 to A30 may be included before step S20:

[0094] Step A10: Obtain the initial deep Q-network, wherein the state space of the initial deep Q-network is set to the environment state when the test steps are executed in the preset test application environment, and the action space of the initial deep Q-network is set to store various test path optimization operations.

[0095] First, an initial deep Q-network is obtained, and its key parameters are set. The state space of the initial deep Q-network is defined as the environment state presented when the test steps are executed in the preset application under test environment. This environment state covers all possible situations of the application under test during the test process, providing the network with comprehensive environmental awareness information. At the same time, the action space of the initial deep Q-network is set to store various test path optimization operations. These operations are the basic action set for subsequent optimization and adjustment of the test path.

[0096] It should be noted that, in this embodiment, the states in the state space may include visual states, structural states, path states, historical action states, and tail sequence states. Specifically, the visual state (Vt) is a feature vector extracted from the current application screenshot by a convolutional neural network. This approach can capture the visual layout information of the UI without relying on the fragile DOM structure, and is a method used in advanced visual testing techniques.

[0097] Structural State (Dt): A vectorized representation of the document object model or view hierarchy of the current page, providing a structured context that might be overlooked by purely visual information.

[0098] Path state (Pt): A vector encoding the current position (t) in the original N-step test path, and may contain a look-through window for the next k planned steps. This is crucial for the agent to understand the original test intent.

[0099] Historical Action State (Ht): An embedded representation of the m most recent actions performed by the agent, used to provide temporal context and prevent the agent from getting stuck in a loop.

[0100] Tails: To address the common problem of state confusion in UIs (i.e., two visually and structurally identical interfaces requiring different actions due to differing previous operation histories), state representation can be further enhanced into "tails." A tail is a sequence of the l most recent (state-action) pairs. This provides the agent with richer historical trajectory information, enabling it to distinguish between similar states with different contexts.

[0101] Specifically, test path optimization operations in the action space may include:

[0102] Execute_Original_Step: Executes the operation specified at step t in the user-programmed path. This is the default baseline action.

[0103] Skip_Step_As_Redundant: If the state analysis indicates that the goal of step t has been achieved (e.g., an input box already contains the correct text), the agent can choose this action.

[0104] Propose_Alternative_Action(a'): An agent can choose an alternative action a' from a predefined library of general UI interactions if it predicts that the action will more efficiently reach a state at a future step.

[0105] Assert_State_And_Terminate: If the agent determines that the final goal of the test case has been achieved ahead of schedule, it can choose to terminate the test early.

[0106] Step A20: Obtain training samples, wherein the training samples include multiple sets of training and testing paths;

[0107] The training samples are obtained, which contain multiple sets of training and testing paths. Each set of training and testing paths simulates the path situations that may occur in actual testing scenarios, providing the network with rich and diverse learning materials to ensure that the trained network can adapt to various complex testing path optimization needs.

[0108] Step A30: Train the initial deep Q network based on the training samples to obtain the trained deep Q network. The deep Q network is used to optimize the initial test path.

[0109] The initial deep Q-network is trained based on the acquired training samples to obtain a well-trained deep Q-network that can be used to optimize the initial test path.

[0110] In one feasible embodiment, step A30 may include steps A301 to A303:

[0111] Step A301: For each training and testing path in the training samples, interact with the application environment under test according to the training and testing path, and obtain each experience tuple in the interaction process according to the preset reinforcement learning strategy. The experience tuple includes the current state, the action executed, the state at the next moment, and the compound reward at each time step.

[0112] For each training-test path in the training samples, the training-test path interacts with a pre-defined application environment under test. During the interaction, it operates according to a pre-defined reinforcement learning strategy, thereby obtaining various experience tuples during the interaction process. Each experience tuple contains: the current state at each time step, i.e., the specific state of the application environment under test at a certain time step; the action performed, i.e., the test path optimization operation taken in the current state; the state of the next time step, i.e., the state that the application environment under test will transition to after performing the action; and a composite reward, which is a comprehensive evaluation index used to measure the benefits obtained after performing the action, taking into account multiple factors such as the effectiveness and efficiency of path optimization.

[0113] It is worth mentioning that, in this embodiment, the reward function (i.e., the calculation function of the compound reward) can specifically be: R = α•R efficiency +β•R coverage -γ•C deviation -δ•C costThe weighting coefficients α, β, γ, and δ can be dynamically adjusted to suit the focus of different optimization stages. R efficiency R represents efficiency rewards. coverage Indicates coverage bonus, C deviation Indicates deviation from punishment, C cost This indicates a cost penalty.

[0114] In one feasible implementation, R efficiency The calculation method is 1 / num_steps, where num_steps represents the number of steps, or is proportional to the number of steps saved. The purpose is to reward the agent for completing the task in fewer steps, and it is the main driving force for optimization; R coverage The calculation method involves awarding a reward when a critical checkpoint is reached, with the aim of ensuring that the optimization process does not sacrifice the core coverage objective of the original test; C deviation The calculation method is -1×state_distance_metric, where state_distance_metric represents the state distance metric. Its purpose is to penalize paths that lead to a final state inconsistent with the original script, ensuring the correctness of the optimization; C cost The calculation method is -0.01×execution_time_sec, where execution_time_sec represents the execution time. The purpose is to penalize computationally expensive operations (such as long waiting times) and encourage streamlined execution.

[0115] In this embodiment, a simple reward function is insufficient for complex optimization tasks, while this composite function embodies a profound understanding of the various trade-offs in test automation. It demonstrates that the agent seeks not merely the "shortest path," but a path found according to a more refined definition of "optimal" that integrates speed, correctness, and cost.

[0116] It is also worth mentioning that the preset reinforcement learning strategy in this embodiment can be an epsilon-greedy (exploration-exploitation) strategy. During training, "exploration" does not involve randomly trying arbitrary UI operations (such as clicking on irrelevant advertisements), but rather attempting to optimize actions within the step logic of a predefined path. For example, in the "enter quantity on the product page" step, the exploration action might be "skip the input (if the quantity is already correct by default)" or "directly enter the target value (replacing the original '+1' click 3 times)". This "targeted exploration" ensures that the agent's trial and error always revolves around "improving path efficiency", avoiding the problem of "invalid action proliferation" caused by excessive exploration scope in traditional RL UI testing (such as clicking on elements outside the path causing test interruption).

[0117] Step A302: Calculate the priority of each experience tuple and store each experience tuple and its corresponding priority into a preset experience replay pool.

[0118] The priority of each experience tuple is calculated. The calculation method can be set according to specific needs and actual conditions, such as determining it based on factors like the magnitude of the composite reward and the importance of state transitions. After calculating the priorities, each experience tuple and its corresponding priority are stored in a pre-defined experience replay pool. The experience replay pool serves to store and manage experience tuples, providing data support for subsequent network training.

[0119] Step A303: Based on the priority replay mechanism, sampled experience tuples are selected from the experience replay pool to iteratively train the initial deep Q network, resulting in a trained deep Q network.

[0120] A priority replay mechanism selects sampled experience tuples from the experience replay pool. This mechanism selectively samples experience tuples based on their priority, prioritizing higher-priority tuples to ensure the network learns more valuable information. The sampled experience tuples are then used to iteratively train the initial deep Q-network. In each iteration, the network adjusts its parameters based on the information in the experience tuples, gradually optimizing its performance. After multiple iterations, a well-trained deep Q-network is obtained, capable of optimizing the initial test path.

[0121] It is worth mentioning that the weight of TD-error (empirical importance) in the priority replay mechanism in this embodiment is tilted towards the following scenarios:

[0122] State confusion scenarios: such as "differences in the results of actions under two UI states with the same visual / structural characteristics but different histories". This kind of experience can help the agent quickly learn to distinguish similar states and reduce subsequent decision-making errors.

[0123] Optimize effective scenarios: such as "successfully reaching the next critical checkpoint after skipping steps" - this type of experience is the core of efficient paths, and prioritizing learning can accelerate the formation of optimization strategies;

[0124] Deviating from risky scenarios, such as "alternative actions leading to a final state inconsistent with the original path," can strengthen the agent's understanding of "optimization boundaries" and prevent over-optimization from compromising test correctness. By prioritizing the sampling of these experiences, the agent's understanding of "what to optimize and what to avoid" in UI testing is improved by 2-3 times under the same number of training steps, significantly shortening the convergence period.

[0125] For example, in one feasible implementation, the training process of a deep Q-network is performed in rounds, each round designed to learn how to complete a given UI test path more efficiently. The following are the specific steps of the agent in a complete training cycle:

[0126] (1) Initialize the round:

[0127] Anchoring Path and Goal: At the start of each round, the system loads a user-defined N-step test path (e.g., "Login → Homepage → Product Page → Checkout Page"). This sets clear macro-level goals and exploration boundaries for the agent.

[0128] Set initial state: Set the initial state s0 to the UI state of the path starting point, including visual screenshot V0, DOM structure D0, path context P0 (such as "current position 0 / N") and historical action H0 (empty).

[0129] Network and Environment Preparation: Initialize the parameters of the evaluation network and the target network (usually set the parameters of both to be the same), and prepare the UI testing environment.

[0130] (2) Exploration and interaction within the path framework:

[0131] At each step t of the round:

[0132] Select action at based on the current state st:

[0133] The agent inputs the current state st (including Vt, Dt, Pt, Ht) into the evaluation network to obtain the Q-values ​​of all possible optimized actions ("execute the original step", "skip", "alternative action", etc.).

[0134] An epsilon-greedy strategy is adopted: a directed exploration action is randomly selected with probability ε (such as trying to skip a seemingly optional step), and the action with the highest Q value is selected with probability 1-ε. This ensures that the exploration always revolves around the core task of "improving path efficiency".

[0135] Interact with the UI environment and receive feedback:

[0136] The agent performs the action at.

[0137] The UI testing environment returns the next state s{t+1} (new screenshots V{t+1} and DOMD{t+1}), along with a comprehensive reward rt. This reward consists of the following parts:

[0138] Efficiency Reward: A positive reward is given if an action (such as "skip") successfully shortens the steps and the status is met.

[0139] Coverage Reward: If the agent's actions enable the test cases to cover previously untouched interface elements, functional modules, or specific business process branches (e.g., successfully triggering a hidden function entry point or traversing an order processing branch process that has not been visited before), a corresponding positive reward will be given based on the difficulty and importance of the coverage, thereby incentivizing the agent to explore more diverse scenarios and improve the comprehensiveness of the test.

[0140] Deviation penalty: If an action causes the UI state to deviate from the expected path (such as skipping required fields), a negative penalty is applied.

[0141] Cost penalty: Record the actual time spent on an operation (such as page loading) as a negative reward to prevent the agent from choosing actions that are "seemingly efficient but actually time-consuming".

[0142] (3) Store and prioritize key experiences:

[0143] Encapsulating Experience: After performing the action, we obtain a complete experience tuple: (st, at, rt, s{t+1}). This tuple is the raw material for the agent's learning.

[0144] Prioritization (TD-error): To store this new experience in the replay pool, we first calculate its "unexpectedness," or TD-error. This value is roughly equal to the difference between "the actual reward received (rt) + the best expectation for the future" and "the initial expectation." The higher the TD-error, the more unexpected the experience is, and the greater its learning value.

[0145] Store in Prioritized ExperienceReplay (PER): Store the experience tuple (st,at,rt,s{t+1}) along with its TD-error as priority in the experience replay pool.

[0146] (4) Sample experience and update the network:

[0147] Priority Sampling: Once enough experience has accumulated in the experience replay pool, we no longer use the newly generated experience. Instead, we sample from the pool according to priority, extracting a batch of experience. High-priority experiences (such as those key scenarios) have a higher probability of being selected. This step is the core application of the PER mechanism in step (4), which ensures that training resources are concentrated on the most worthwhile samples to learn from.

[0148] Calculate the loss function: For each sampled experience (sj,aj,rj,s{j+1}):

[0149] Target Q-value: The target network is used to calculate the maximum Q-value that can be obtained in state s{j+1}, and this value is combined with the reward rj to form the learning objective. The parameters of the target network are fixed, which provides a stable objective for learning and is specifically designed to handle dynamic changes such as temporary pop-ups in the UI and loading delays, ensuring decision stability.

[0150] Predicted Q-value: The evaluation network is used to calculate the Q-value of performing action aj in state sj.

[0151] Loss: The loss function calculates the difference between the "target Q-value" and the "predicted Q-value". Simultaneously, path intent constraints are also added to the loss function as a penalty: if action aj causes state s{j+1} to deviate significantly from the original path plan, the loss value is increased.

[0152] Update the evaluation network: Through backpropagation, the parameters of the evaluation network are updated using the gradient descent algorithm, so that its predicted Q value gets closer and closer to the more accurate target Q value.

[0153] Synchronize the target network: Every fixed number of steps, the parameters of the evaluation network are completely copied to the target network to achieve parameter synchronization.

[0154] (5) Termination and Iteration:

[0155] Check for round termination: After each interaction, check if the round termination condition is met.

[0156] a. The final state of the test path has been reached.

[0157] b. Core coverage objectives were achieved ahead of schedule (e.g., key checkpoints have been covered).

[0158] c. Exceeding the preset maximum number of steps.

[0159] Start a new cycle:

[0160] If the round does not end, let st = s{t+1} and return to step (2) to continue the interaction.

[0161] If the round ends, record the "step saving rate" and "final state consistency" metrics for this round. Then return to step (1) and start a brand new round.

[0162] Training convergence determination: When all indicators (such as step saving rate ≥ 20%, state consistency ≥ 95%) in multiple consecutive rounds are stably met to the preset standards, the model is determined to have converged and training ends.

[0163] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that of Embodiments 1 and 2 described above can be referred to the above description and will not be repeated hereafter. On this basis, as Figure 4 As shown, step S20 may include steps S201 to S203:

[0164] Step S201: Determine the status information of each test step in the initial test path based on the environment status when the initial test path is executed in the preset application environment under test.

[0165] It should be noted that, since the direct input of the deep Q network is not the original initial test path, but rather preprocessed state information representing the current UI interface, the state information of each test step in the initial test path is determined first in this embodiment.

[0166] Based on the environmental state presented when the initial test path is actually executed in the preset application environment under test, each test step in the initial test path is analyzed in detail to accurately determine the state information corresponding to each test step. This state information comprehensively reflects the specific context of the test step in the application environment under test.

[0167] In one feasible embodiment, step S201 may include steps S2011-S2015:

[0168] Step S2011: For each test step executed in the preset test application environment of the initial test path, extract the visual state from the current user interface screenshot when each test step is executed.

[0169] For each test step executed in the preset test application environment of the initial test path, a screenshot of the current user interface when the test step is executed is taken, and then a preprocessing module (usually a pre-trained image model) is used to convert it into a fixed-dimensional visual feature vector, which is the visual state.

[0170] Step S2012: Vectorize the document object model or view hierarchy of the current user interface during the execution of each test step to obtain the structural state;

[0171] For each test step executed in the preset test application environment of the initial test path, the DOM tree or view hierarchy of the current user interface at the time of the test step execution is parsed, the attributes of key elements (such as whether the button is clickable, the content of the text box, etc.) are extracted, and then converted into a structural feature vector by an encoder, that is, the structural vector is obtained.

[0172] Step S2013: Obtain the path status based on the position information of each test step in the initial test path;

[0173] Based on the current step's position within the entire "test path," a path context vector is generated. For example, if the total path has N steps, and the current step is i, this vector could be [i / N] or an encoding representing the intent of future steps, thus obtaining the path state.

[0174] Step S2014: Extract the preceding action embedding representation of each test step to obtain the historical action state;

[0175] Record the most recent series of actions for each test step to form a sequence, which is used to capture temporal dependencies, i.e., to obtain the historical action state.

[0176] Step S2015: Extract the state-action pairs of each test step to obtain the tail sequence state.

[0177] To address the common state confusion problem in UIs (i.e., two visually and structurally identical interfaces requiring different actions due to differing previous operation histories), state representation can be further enhanced into "Tails." A tail is a sequence of the l most recent (state-action) pairs, i.e., Tt = ((st-l, at-l), ..., (st-1, at-1), st). This provides the agent with richer historical trajectory information, enabling it to distinguish similar states with different contexts, thus obtaining the tail sequence state.

[0178] Step S202: For each test step, the state information of the test step is analyzed by a deep Q-network to determine the optimal action after the test step.

[0179] For each test step with defined state information, a pre-trained deep Q-network is used to conduct in-depth analysis of its state information. Leveraging its powerful learning and decision-making capabilities, the deep Q-network can search and determine the optimal action after the test step within a pre-defined action space based on the input state information. This optimal action is calculated using a reinforcement learning algorithm, taking into account various factors such as test efficiency, coverage, and path rationality, and can guide the test path towards a more optimal direction.

[0180] It's worth noting that traditional reinforcement learning applications in the testing domain typically treat the entire application under test as the environment, resulting in an extremely large and even unmanageable state-action space. This embodiment redefines this problem: the "environment" is no longer the entire application, but rather the traversal process along a test path predefined by the user through a first visual orchestration interface. The goal of the reinforcement learning agent is not to discover a valid test case from scratch, but to find the most efficient way to execute a known valid test case. This problem transformation greatly narrows the scope of the RL (Reinforcement Learning) problem, making network training computationally feasible and efficient. The agent is no longer blindly exploring, but finding shortcuts guided by a "map" (i.e., the original path created by the user).

[0181] In one feasible embodiment, the deep Q-network includes a convolutional neural network branch, a multilayer perceptron branch, a long short-term memory network branch, and a fully connected layer; step S202 may include steps S2021-S2023:

[0182] Step S2021: For each test step, the visual state corresponding to the test step is used as the input of the convolutional neural network branch, the structural state and path state corresponding to the test step are used as the input of the multilayer perceptron branch, and the historical action state and tail sequence state corresponding to the test step are used as the input of the long short-term memory network branch.

[0183] To accurately process multi-dimensional state information in UI testing, the Deep Q-Network employs a multi-branch fusion architecture. Each branch is specifically adapted to the specific information type of the UI scenario, ensuring that the parsing of UI states is both comprehensive and accurate. Specifically, this Deep Q-Network includes:

[0184] The Convolutional Neural Network (CNN) branch takes the visual state of the current UI screenshot as input. In UI testing, visual information is the direct feedback of user actions (such as whether a button is clicked or a form is filled out), but the position and style of elements in the screenshot may change dynamically (such as pop-up loading or list scrolling). The CNN branch extracts invariant features (such as the visual outline of the "input box") and variable features (such as the pixel distribution of the "text in the input box") through convolutional layers. It can capture the essential structure of the UI layout and distinguish the differences between key states such as "not filled" and "filled," providing a basis for "skipping redundant steps" (such as not needing to re-enter if the input box is filled).

[0185] Multilayer Perceptron (MLP) branch: The input consists of the structural and path states at the DOM / view level. In UI testing, the DOM structure contains logical relationships that are difficult to capture visually (such as the disabled / enabled state of the "submit button" or the option list of the "drop-down menu"), while the path context (such as "currently in step 3, the next step is to enter the payment page") anchors the agent's optimization direction—avoiding deviations from the user's predefined test intent. The MLP branch encodes structured information and path intent (such as "the next 2 steps are to complete the payment") into vectors through fully connected layers, providing logical support for "suggesting alternative actions" (such as directly clicking the enabled submit button instead of clicking the blank space first in the original steps).

[0186] The Long Short-Term Memory (LSTM) branch takes historical action states and tail sequence states as input. "State confusion" is a common problem in UI testing. For example, a "product details page" might require different subsequent actions depending on whether it's accessed from the homepage or from a search result (the former requires returning to the homepage, the latter to the search page). The LSTM branch, through its temporal modeling capabilities, transforms the most recent (state-action) pairs into context vectors, accurately distinguishing UI states that are visually and structurally identical but have different histories, thus preventing the agent from getting stuck in loops (such as repeatedly jumping between two identical UI pages).

[0187] Step S2022: The outputs of the convolutional neural network branch, the multilayer perceptron branch, and the long short-term memory network branch are concatenated to obtain the fused feature vector;

[0188] The feature vectors output by the three branches of CNN, MLP, and LSTM are concatenated end to end to form a longer fused feature vector that contains all the information.

[0189] Step S2023: Input the fused feature vector into the fully connected layer to obtain the optimal action after the test step output by the fully connected layer.

[0190] This fused long vector is input into one or more fully connected layers (also called output layers). The number of neurons in this output layer is equal to the number of all possible actions (such as "execute the original step", "skip", "alternate action A", "alternate action B", etc.). The output value of each neuron is the Q-value of the corresponding action in the current state, and the action with the highest Q-value is selected as the optimal action.

[0191] Step S203: Iteratively update the initial test path based on each optimal action to obtain the optimized test path.

[0192] After determining the optimal action for each test step, the initial test path is iteratively updated based on these optimal actions. In each iteration, the path is adjusted accordingly based on the optimal action for the current test step, generating a new test path, which is then used as the initial path for the next iteration. Through continuous iterative updates, the individual test steps and overall structure of the initial test path are gradually optimized, ultimately resulting in an optimized test path. Compared to the initial test path, this optimized test path has higher testing efficiency, broader test coverage, and a more reasonable test process.

[0193] Therefore, in this embodiment, a no-code, natural language visual orchestration interface is first used to allow users to easily define an initial test path with high business value, preserving human intent. Then, reinforcement learning techniques are applied to a constrained optimization problem (i.e., optimizing this initial test path) rather than an unconstrained generation problem, thus avoiding the problem of state space explosion. In this way, this embodiment aims to produce automated test cases that are both efficient and robust, and fully conform to the original business objectives, thereby significantly reducing maintenance costs and improving the return on investment of automated testing.

[0194] Compared to traditional no-code visual orchestration testing platforms, this embodiment achieves a leap from "simplified creation" to "intelligent optimization": the core value of existing no-code platforms lies in simplifying the test case creation process. However, they faithfully execute the user's orchestration path, even if that path contains redundant and inefficient steps. This embodiment, on top of simplified creation, adds a completely new dimension of intelligent optimization, which is not available in existing technologies. It not only makes test creation easier, but also makes the final test cases themselves more efficient and concise.

[0195] Significantly improves test execution efficiency and reduces costs: By using a reinforcement learning engine to discover and eliminate unnecessary steps (such as redundant verification clicks and invalid waits), the final test script generated in this embodiment executes faster and consumes less computing resources.

[0196] Compared to AI-driven autonomous test generation technology, this embodiment effectively avoids the state space explosion problem: fully autonomous RL test generation technology suffers from the "curse of dimensionality" when faced with complex modern applications due to its excessively large state-action space, making it difficult to converge or expand. This embodiment solves this core problem technically by limiting the application of RL to optimizing a predefined manual path, greatly constraining the search space and making the application of AI truly feasible.

[0197] Perfectly Preserving Business Intent and Interpretability: Self-generated test cases often lack clear business context, making their behavior difficult for humans to understand and maintain, severely hindering their adoption in enterprises. This implementation starts from a user-created path with clear business logic, ensuring that the core purpose and business value of the test are always preserved. AI acts as an "optimizer" rather than a "creator," thus its outputs are highly interpretable and business-relevant.

[0198] Providing a controllable and reliable human-machine collaboration mechanism: The AI ​​autonomous generation process is often like a "black box," making it difficult for users to control its behavior. This embodiment designs a transparent human-machine interaction closed loop: all optimization suggestions proposed by the AI ​​must be reviewed and approved by the user. This "human-in-the-loop" design ensures that the final test script fully meets the user's expectations, thereby establishing user trust in the AI's capabilities and promoting the practical application of the technology.

[0199] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the reinforcement learning-based UI test script visualization orchestration method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0200] This application also provides a UI test script visualization orchestration system based on reinforcement learning. Please refer to... Figure 5 The reinforcement learning-based UI test script visualization orchestration system includes:

[0201] The visual orchestration module 10 is used to receive the initial test path constructed by the user through a preset first visual orchestration interface;

[0202] The reinforcement learning optimization engine 20 is used to optimize the initial test path to obtain an optimized test path.

[0203] The visualization orchestration module 10 is also used to display comparison information between the optimized test path and the initial test path on a preset second visualization orchestration interface, and to generate UI test scripts in response to user interaction operations received on the second visualization orchestration interface.

[0204] Optionally, the reinforcement learning optimization engine 20 includes:

[0205] The status determination unit is used to determine the status information of each test step in the initial test path based on the environmental status when the initial test path is executed in the preset application environment under test.

[0206] The action determination unit is used to analyze the state information of each test step through a deep Q-network and determine the optimal action after the test step.

[0207] The path optimization unit is used to iteratively update the initial test path based on each optimal action to obtain an optimized test path.

[0208] The reinforcement learning-based UI test script visualization orchestration system provided in this application adopts the reinforcement learning-based UI test script visualization orchestration method in the above embodiments, enabling efficient and low-cost creation and maintenance of UI automated tests with clear business intent and robust execution. Compared with the prior art, the beneficial effects of the reinforcement learning-based UI test script visualization orchestration system provided in this application are the same as those of the reinforcement learning-based UI test script visualization orchestration method provided in the above embodiments, and other technical features of the reinforcement learning-based UI test script visualization orchestration system are the same as those disclosed in the reinforcement learning-based UI test script visualization orchestration method in the above embodiments, and will not be repeated here.

[0209] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A UI test script visualization orchestration method based on reinforcement learning, characterized in that, The reinforcement learning-based UI test script visualization orchestration method includes: The system receives the initial test path built by the user through a pre-defined first visual orchestration interface. The initial test path is optimized to obtain the optimized test path; The optimized test path and the initial test path are displayed in a preset second visual orchestration interface, and a UI test script is generated in response to user interaction operations received in the second visual orchestration interface. The step of optimizing the initial test path to obtain an optimized test path includes: Based on the environmental state when the initial test path is executed in the preset application environment under test, the state information of each test step in the initial test path is determined, wherein the state information includes visual state, structural state, path state, historical action state and tail sequence state. For each of the test steps, the state information of the test step is analyzed by a deep Q-network to determine the optimal action after the test step. The action space of the deep Q-network is set to store test path optimization operations, which include: executing the original step, skipping redundant steps, proposing alternative actions, asserting the state and terminating. The initial test path is iteratively updated based on each of the optimal actions to obtain an optimized test path; The deep Q-network includes a convolutional neural network branch, a multilayer perceptron branch, a long short-term memory network branch, and a fully connected layer; The step of analyzing the state information of each test step using a deep Q-network to determine the optimal action after the test step includes: For each of the test steps, the visual state corresponding to the test step is used as the input of the convolutional neural network branch, the structural state and path state corresponding to the test step are used as the input of the multilayer perceptron branch, and the historical action state and tail sequence state corresponding to the test step are used as the input of the long short-term memory network branch. The outputs of the convolutional neural network branch, the multilayer perceptron branch, and the long short-term memory network branch are concatenated to obtain a fused feature vector. The fused feature vector is input into the fully connected layer to obtain the optimal action after the test step output by the fully connected layer.

2. The UI test script visualization orchestration method based on reinforcement learning as described in claim 1, characterized in that, Before the step of optimizing the initial test path to obtain the optimized test path, the method further includes: Obtain an initial deep Q-network, wherein the state space of the initial deep Q-network is set to store the environment state when the test steps are executed in the preset application environment under test, and the action space of the initial deep Q-network is set to store various test path optimization operations; Obtain training samples, wherein the training samples include multiple sets of training and testing paths; The initial deep Q-network is trained based on the training samples to obtain a trained deep Q-network, which is used to optimize the initial test path.

3. The UI test script visualization orchestration method based on reinforcement learning as described in claim 2, characterized in that, The step of training the initial deep Q-network based on the training samples to obtain a trained deep Q-network includes: For each training and testing path in the training samples, the application environment under test is interacted with according to the training and testing path, and each experience tuple in the interaction process is obtained according to the preset reinforcement learning strategy. The experience tuple includes the current state, the action performed, the state at the next moment, and the composite reward at each moment. Calculate the priority of each of the experience tuples, and store each of the experience tuples and its corresponding priority into a preset experience replay pool; Based on the priority replay mechanism, sampled experience tuples are selected from the experience replay pool to iteratively train the initial deep Q network, resulting in a trained deep Q network.

4. The UI test script visualization orchestration method based on reinforcement learning as described in claim 1, characterized in that, The step of determining the status information of each test step in the initial test path based on the environment state when the initial test path is executed in a preset application environment under test includes: For each test step of the initial test path executed in the preset application environment under test, the visual state is extracted from the current user interface screenshot when each test step is executed. The document object model or view hierarchy of the current user interface during the execution of each test step is vectorized to obtain the structural state; The path status is obtained based on the position information of each test step in the initial test path; Extract the preceding action embedding representation of each test step to obtain the historical action state; Extract the state-action pairs of each test step to obtain the tail sequence state.

5. The UI test script visualization orchestration method based on reinforcement learning as described in claim 1, characterized in that, The first visual orchestration interface includes natural language command blocks. The step of receiving the initial test path constructed by the user through the preset first visual orchestration interface includes: In response to the user's drag-and-drop operation and parameter configuration operation on the natural language instruction block in the first visual orchestration interface, an initial test path is generated.

6. The UI test script visualization orchestration method based on reinforcement learning as described in claim 1, characterized in that, The step of displaying the comparison information between the optimized test path and the initial test path on a preset second visual orchestration interface, and generating a UI test script in response to user interaction operations received on the second visual orchestration interface, includes: In the preset second visual arrangement interface, the optimized test path and the initial test path are distinguished by different colors or styles; The steps that were optimized out in the initial test path are visually grayed out, and the reasons for the optimization are displayed in response to the first user interaction operation. In response to a second user action, the optimized test path and / or the initial test path are modified to generate a target test path; A UI test script is generated based on the target test path.

7. A UI test script visualization orchestration system based on reinforcement learning, characterized in that, The reinforcement learning-based UI test script visualization orchestration system includes: The visual orchestration module is used to receive the initial test path built by the user through a preset first visual orchestration interface; A reinforcement learning optimization engine is used to optimize the initial test path to obtain an optimized test path. The visualization orchestration module is also used to display the comparison information between the optimized test path and the initial test path on a preset second visualization orchestration interface, and to generate a UI test script in response to user interaction operations received on the second visualization orchestration interface. The reinforcement learning optimization engine includes: The state determination unit is used to determine the state information of each test step in the initial test path based on the environmental state when the initial test path is executed in the preset application environment under test. The state information includes visual state, structural state, path state, historical action state, and tail sequence state. An action determination unit is used to analyze the state information of each test step through a deep Q-network to determine the optimal action after the test step. The action space of the deep Q-network is set to store test path optimization operations, which include: executing the original step, skipping redundant steps, proposing alternative actions, asserting the state, and terminating. The path optimization unit is used to iteratively update the initial test path according to each of the optimal actions to obtain an optimized test path; The deep Q-network includes a convolutional neural network branch, a multilayer perceptron branch, a long short-term memory network branch, and a fully connected layer; the action determination unit is further used for: For each of the test steps, the visual state corresponding to the test step is used as the input of the convolutional neural network branch, the structural state and path state corresponding to the test step are used as the input of the multilayer perceptron branch, and the historical action state and tail sequence state corresponding to the test step are used as the input of the long short-term memory network branch. The outputs of the convolutional neural network branch, the multilayer perceptron branch, and the long short-term memory network branch are concatenated to obtain a fused feature vector. The fused feature vector is input into the fully connected layer to obtain the optimal action after the test step output by the fully connected layer.

Images