A method and system for automatically generating web application test cases based on reinforcement learning

Abstract

The application relates to a Web application test case automatic generation method and system based on reinforcement learning, which extracts a state from a webpage and outputs the extracted state; identifies an interactive element in the webpage, generates an action of accessing the interactive element, and outputs the generated action; identifies whether the state is a known state, constructs a state diagram with the state as a node and the transition between the states as an edge; calculates a reward according to the state and the action; trains an action selection strategy by using a reinforcement learning algorithm, selects an action according to the action selection strategy, executes the action on a Web application, makes the Web application jump to another webpage, and saves a sequence of actions executed on the Web application as a test case. The application can automatically provide a test case for a target Web application, greatly improves the coverage rate of an existing test case, and discovers potential defects as early as possible. In the test case generation process, the automation degree of the Web application test is improved, and the usability of the Web application is further improved.

Description

TECHNICAL FIELD

[0001] The application relates to a Web application test case automatic generation method and system based on reinforcement learning, and belongs to the field of software testing. BACKGROUND

[0002] In recent years, Web applications have increased dramatically. A recent survey shows that as of July 2022, there are more than 1 billion Web applications in the world. On average, users spend 7 hours a day using web applications. Manual testing and automated testing are important means to ensure the quality of web applications.

[0003] However, manual testing is very time-consuming. Moreover, there are a large number of feasible sequences in Web applications. Manual test cases can only cover a small part of them. Tools such as Selenium and Playwright can generate clicks, inputs and other actions that simulate manual operations to interact with target applications according to test scripts written by testers. However, these tools still require testers to have professional knowledge to write test cases. A Web application program automatic testing tool and method CN201710023922.8 further supports T language to write test cases, and testers only need to understand web testing scenarios and simple basic programming knowledge to write test scripts for testing. Although T language simplifies Java and C language, CN201710023922.8 still requires testers to have research and development skills and scenario knowledge.

[0004] In summary, although the prior art simplifies the test case, it still requires manual intervention and is difficult to automatically generate effective action sequences. SUMMARY

[0005] The application solves the problem: overcome the shortcomings of the prior art, provide a Web application test case automatic generation method and tool based on reinforcement learning, which can train effective action selection strategy to generate test cases using executed action sequences, support automatic extraction of application state, and select actions on the state, according to the action execution result, use reinforcement learning to train the action selection strategy, so as to no longer manually generate test cases.

[0006] The application solves the problem: overcome the shortcomings of the prior art, provide a Web application test case automatic generation method and tool based on reinforcement learning, which can train effective action selection strategy to generate test cases using executed action sequences, support automatic extraction of application state, and select actions on the state, according to the action execution result, use reinforcement learning to train the action selection strategy, so as to no longer manually generate test cases.

[0007] In a first aspect, a Web application test case automatic generation method based on reinforcement learning is provided, comprising the following steps:

[0008] Step 1: Extracting states from the webpage: obtaining the node tree of the webpage and initializing the state with the node tree; then traversing the state to delete the elements and related edges of the redundant node tree which have no influence on the visualization of the page, simplifying the number of elements of the state; the node tree after simplification is called the rendering tree, and the elements on the rendering tree are traversed in a breadth-first manner, if two elements are similar, it is considered that the two elements have similar functions, for an element, if there is another element similar to the one element, it is considered that the two elements are elements with similar functions, elements with similar functions are identified as the same state, thereby extracting states from the webpage;

[0009] Step 2: identifying accessible elements in the extracted states and generating actions of the accessible elements: identifying actions through elements of the node tree, traversing elements on the original node tree, if an element meets a pre-defined condition, it is considered that there is an accessible element in the action, and the action of the accessible element is generated;

[0010] Step 3: constructing a state diagram with states as nodes and actions between states as edges to reflect the transition relationship between states, the state diagram includes a state set, an action set, an action that can trigger state transition and an initial state; and when the current state and the current action are added to the state diagram, it is necessary to judge whether the current state already exists in the state diagram, identify whether the current state is a known state, to avoid redundant states in the state diagram; for the current action, traverse the action set of the state diagram, judge whether there is a known action in the action set that is the same as the current action, to avoid redundant actions in the state diagram;

[0011] Step 4: based on the states and actions in the state diagram, evaluating the contribution of the action executed at each step to the state execution, designing a reward model to calculate the reward for the action; whenever the current action is executed, the state is transferred from the previous state to the current state, and the reward for the current action is calculated according to the reward model;

[0012] Step 5: training an action selection policy using a reinforcement learning algorithm according to the state of step 1, the action output of step 2 and the reward of step 4, and selecting an action according to the action selection policy, and outputting the selected action;

[0013] Step 6: executing the action output by step 5 on the web application, so that the web application jumps to another webpage;

[0014] Step 7: continue to execute steps 1-6, and save the sequence of actions executed on the web application as a test case.

[0015] Further, the step 1 is specifically implemented as follows:

[0016] (11) obtaining the DOM tree of the webpage and initializing the state with the DOM tree, i.e. initializing the node set and the edge set of the state with the node set and the edge set of the DOM tree respectively;

[0017] (12) traversing the state to delete redundant elements, if an element has only one child element, deleting the element from the node set of the state, deleting the edge associated with the element from the edge set of the state, and adding an edge from the parent element of the element to the child element of the element to the edge set of the state;

[0018] (13) performing a breadth-first traversal on the rendering tree, for an element, if there is another element similar to the element, considering the element and the other element as the same service function and using the element to represent the service function; then further traversing the child elements of the two elements to find similar elements of the child elements of the element in the child elements of the other element; conversely, if the element does not have similar elements, stopping searching for similar elements of the child elements of the element.

[0019] Further, in the step (13), the judgment that the other element is similar to the element includes that the other element has a similar structure to the element or the other element has a similar style to the element, wherein:

[0020] (131) the other element has a similar structure to the element: using the ambiguity of the path language of the element to identify whether the elements have similar structures, i.e. finding similar elements by ignoring the index of the ancestor element in the path language of the element;

[0021] (132) the other element has a similar style to the element: defining the style similarity of the other element to the element as the weighted average of the edit distances of the class names, positions, sizes, hyperlinks, external file references and identifiers of the two elements; if the style similarity of the other element to the element is greater than or equal to a pre-set threshold, considering that the other element has a similar style to the element.

[0022] Further, in the step 2, the pre-defined condition is one of the following three:

[0023] (21) if the tag name of an element matches the default setting, considering that the element is operated and generating an action accessing the element;

[0024] (22) if the tag name of an element is an input control or a multi-line plain text editing control and the type of the element is one of a radio button and a check box, considering that the element is a clickable element and generating an action accessing the element with the type of clickable for the element;

[0025] (23) If one element tag name and type match the user configuration, it is considered that the one element can be operated, and an action of accessing the one element is generated.

[0026] Further, in the step 3, when the state and the action are added to the state diagram, it is identified whether the specific state and the action already exist in the state diagram, and the specific judgment is as follows:

[0027] (31) The current state and the existing state are compared one by one to see whether they are the same, if the current state is the same as the existing state, the current state is set as the existing state; otherwise, if the current state is different from the existing state, it indicates that the current state does not exist in the state set of the state diagram, and the current state is added to the state set of the state diagram; then, a transition from the previous state to the current state is added to the state diagram;

[0028] (32) At the current state, each action of the current state is also compared with the existing action in the state diagram, if the current action is the same as the existing action, the current action is not added to the action set; otherwise, if the current action is different from the existing action, the current action is added to the action set; for the action, the action set of the state diagram is traversed to check whether there is an existing action that is the same as the current action, if the types of the two actions are the same, and the elements accessed by them are similar, it is considered that the two actions are the same.

[0029] Further, in the step (1), if the current state is the same as the existing state, a state index tree is constructed, and similarity is used for judgment, if the similarity between the current state and the existing state in the state set is higher than a pre-set threshold, it is considered that the current state is the same as the existing state;

[0030] The similarity judgment is as follows:

[0031] Given the current state, a state index tree is constructed, and the similarity is used to compare the state and the state index tree to determine whether there is an existing state in the state diagram that is the same as the current state;

[0032] The specific implementation is as follows: when the state index tree is constructed, first, all elements on the current state s- are labeled to indicate which state the element belongs to; then, the state index tree composed of the existing states is obtained, if the state index tree has not been constructed, the state index tree is set as the current state, and there is no state that is the same as the current state;

[0033] If the state index tree exists, the number of similar elements between the current state and the existing state is calculated by traversing the state index tree; at the same time, the elements of the current state are also merged into the state index tree, and the number of similar elements between the current state and the existing state is calculated;

[0034] After obtaining the number of similar elements between the current state and the existing state, the similarity between the current state and the existing state is calculated, and the similarity between the current state and the existing state is defined as:

[0035]

[0036] Wherein, #similarNum is the number of similar elements between the state s and the existing state s', #s and #s' represent the number of elements of the state s and the existing state s' respectively; min represents the minimum;

[0037] It is determined that the state has the maximum similarity with the existing state, if the maximum similarity is greater than the pre-set threshold value, it is considered that the state and the existing state are the same state; otherwise, the state and the existing state are not the same state.

[0038] Further, in step 4, the reward model is as follows:

[0039]

[0040] r i For calculating the reward, if the current state is an external web page extraction, or the current state is the same as the previous state, a negative value reward is given to the current action, i.e. penalty < 0; for other cases, a positive value reward r' is calculated i :

[0041] r′ i =w loc *r loc +w cur *r curiosity +w freq *r freq +w explore *r explore

[0042] Wherein, w loc , w cur , w freq and w explore are the corresponding weights of each reward index, all greater than 0;

[0043] The r loc is a reward index based on the current action position: the agent selects an action of accessing an element e i and the adjacent element e i of the element e i-1 , and the formula of r loc is:

[0044]

[0045] Where h(e) and w(e) each represent an element e i Height and width, dist(e i-1 e i ) indicates that it is related to the element e. i Adjacent element e i-1 The Leibniz distance between them;

[0046] The r curiosity The reward metric is based on curiosity about the current action. Elements that exist in the current state but not in the previous state are considered changing elements. curiosity The calculation formula is:

[0047]

[0048] That is, if the element accessed by the current action is a mutable element, the reward value assigned to the current action is 1 / mutants(s). i ),mutants(s i ) indicates the current state s i The number of changing elements;

[0049] The r freq This is a reward metric based on the frequency of action transitions. It refers to the reward metric *r* based on the frequency of action execution, where the more times an action transitions from the previous state to the current state, the higher the reward metric *r*. freq The smaller r is freq Defined as:

[0050]

[0051] Where, N i This indicates the number of times the action transitions from the previous state to the current state.

[0052] The r explore The reward metric based on the current state refers to the state to which the target application transitions after the current action is performed. The higher the proportion of actions that have never been performed in the current state, the more helpful the current action is in testing the target application, and thus the reward metric *r* is assigned to the current action. explore r explore The calculation formula is:

[0053]

[0054] Where, m i and n i These represent the number of actions in the current state and the number of actions that have not yet been executed, respectively.

[0055] Furthermore, step 5, which involves training the action selection strategy using a reinforcement learning algorithm, is specifically implemented as follows:

[0056] Given a web application, automatically generate a set of test cases. First, record the target application's homepage link and initialize the test case set, strategy, and state set.

[0057] The process executes several cycles. At the start of each cycle, the target application is reset by revisiting its homepage link. Then, the state and actions are extracted from the homepage. Within each cycle, an action sequence is generated by executing several steps. In the i-th step, the current action is selected from the previous state. After executing the current action, the target application redirects to the current webpage, where the current state and available actions are extracted. The state set is updated using the extracted current state, and the reward for the current action is calculated based on the previous state, the current action, and the reward r for the current action. i Update the strategy and add the current action to the action sequence. The action sequence generated in each cycle is saved to the test case set.

[0058] The update strategy is implemented through the update function Q, and the update method is as follows:

[0059] Q(s i-1 , act i )←Q(s i-1 , act i )+α(r i +γQ * (s i a i+1 )-Q(s i-1 a i ))

[0060] Q(s i-1 , act i ) indicates the state in the previous state s i-1 Execute the current action (act) i Contribution to the target application being tested; when the current action is executed. i Make the previous state s i-1 From the current action i Transition to the current state s i At that time, calculate the current reward r. i And update function Q, Q * (s i+1 a i+1 ) indicates the state from the next state s i+1 The initial cumulative maximum contribution; the cumulative reward will be discounted by a discount rate γ∈[0,1], where α∈[0,1] represents the learning rate.

[0061] Secondly, this invention provides an automatic test case generation system for web applications based on reinforcement learning, comprising: a state extraction module, an action extraction module, a state graph construction module, a reward model module, and a reinforcement learning agent module; wherein:

[0062] State Extraction Module: Extracting states from web pages: Obtain the node tree of the web page and initialize the state with the node tree; then traverse the state to remove redundant elements of the node tree and their related edges that do not affect the page visualization, simplifying the number of elements in the state; the simplified node tree is called the rendering tree, and the elements on the rendering tree are traversed in a width-first manner. If two elements are similar, they are considered to have similar functions. For an element, if there is another element that is similar to it, the two elements are considered to have similar functions. Elements with similar functions are identified as the same state, thus extracting the state from the web page.

[0063] Action extraction module: Identifies accessible elements in the extracted state and generates actions for accessible elements: Identifies actions through elements in the node tree, traverses the elements in the original node tree, and if an element meets the predefined conditions, it is considered that there is an accessible element in the action and an action for the accessible element is generated.

[0064] State Graph Construction Module: Constructs a state graph with states as nodes and actions that transition between states as edges, reflecting the transition relationships between states. The state graph includes a state set, an action set, action-triggered state transitions, and an initial state. When adding the current state and current action to the state graph, it is necessary to determine whether the current state already exists in the state graph and whether the current state is a known state to avoid redundant states in the state graph. For the current action, the action set of the state graph is traversed to determine whether there is a known action in the action set that is the same as the current action to avoid redundant actions in the state graph.

[0065] Reward Model Module: Based on the states and actions in the state diagram, evaluate the contribution of each action to the state execution, design a reward model, and calculate the reward for the action using the reward model; whenever the current action is completed, the state transitions from the previous state to the current state, and the reward is calculated for the current action according to the reward model.

[0066] The reinforcement learning agent module trains an action selection strategy using reinforcement learning algorithms based on the state, action, and reward, selects an action according to the action selection strategy, and outputs the selected action. The output action is executed on the web application, causing the web application to jump to another webpage and continue executing the state extraction module, action extraction module, state graph construction module, and reward model module. The sequence of actions executed on the web application is saved as test cases.

[0067] Thirdly, the present invention provides an electronic device (computer, server, smartphone, etc.) including a memory and a processor, wherein the memory stores a computer program configured to be executed by the processor, and the computer program includes instructions for performing each step of a reinforcement learning-based automatic generation method for web application test cases according to the present invention.

[0068] Fourthly, the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) storing a computer program, which, when executed by a computer, implements the various steps of the present invention's automatic generation method for Web application test cases based on reinforcement learning.

[0069] The advantages of this invention compared to the prior art are:

[0070] (1) This invention can automatically provide test cases for target Web applications, greatly improving the coverage of existing test cases and discovering potential defects as early as possible. In the process of test case generation, by collecting page information, extracting states and actions to construct a state space, and using reinforcement learning algorithms to execute the state space, the automation level of Web application testing is improved, and the usability of Web applications is further improved.

[0071] (2) Existing random-based methods explore the state space randomly, making it difficult to generate effective action sequences. In model-based methods, the model can provide information for generating effective action sequences. However, existing model-based methods struggle to build complete models for the target application. Therefore, these methods can only generate limited action sequences. To generate test cases that cover multiple states, this invention mainly includes two stages: state and action. After each action is executed, the state and action are extracted from the webpage. To effectively identify states, this invention uses the technical feature of similar elements representing the same function, merging similar functional elements. Webpages with similar functions are identified as the same state, and then a reinforcement learning agent selects the appropriate action and executes it on the target application. Based on the execution result, the reward for the selected action is updated. To improve the efficiency of state execution, an innovative reward model is designed to guide interaction with the Web application like a human tester. Reinforcement learning is used to train the action selection strategy, thus eliminating the need for manual test case generation and greatly improving the coverage of existing test cases, allowing potential defects to be discovered earlier. In the test case generation process, the automation level of Web application testing is improved, further enhancing the usability of the Web application. Attached Figure Description

[0072] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.

[0073] Figure 2 This is a block diagram showing the components of the tool of the present invention;

[0074] Figure 3 This is a flowchart illustrating the process of extracting state from a webpage in this invention;

[0075] Figure 4 This is a flowchart for identifying accessible elements in the states extracted in this invention. Detailed Implementation

[0076] The present invention will now be described in detail with reference to the accompanying drawings.

[0077] like Figure 1 and 2 As shown, the specific implementation of the method of the present invention is as follows:

[0078] Step 1: As Figure 3 As shown, the state is extracted from the webpage: the node tree of the webpage is obtained. In this embodiment of the invention, a DOM tree is used, and the state is initialized with the DOM tree; then, the state is traversed to delete redundant DOM tree elements and their related edges that have no impact on the page visualization, and the number of elements in the state is simplified; the simplified DOM tree is called the rendering tree, and the elements on the rendering tree are traversed in a width-first manner. If two elements are similar, they are considered to have similar functions. For an element e, if there is another element e′ that is similar to the element e, then the element e and the other element e′ are considered to have similar functions. Elements with similar functions are merged, and elements with similar functions are identified as the same state, thereby extracting the state s from the webpage.

[0079] The specific implementation is as follows:

[0080] (11) Obtain the DOM tree (i.e. node tree) of the webpage and initialize the state with the DOM tree, that is, initialize the node set and edge set of state s with the node set and edge set of the DOM tree respectively;

[0081] (12) Traverse the state and delete redundant elements. If an element has one and only one child element, delete the element e from the node set of the state, delete the edge associated with the element from the edge set of the state, and add the edge from the parent element of the element to the child element of the element to the edge set of the state s.

[0082] (13) Perform a width-first traversal on the rendering tree. For an element e, if there is another element e′ that is similar to the element e, then the element e and the other element e′ are considered to have the same service function, and the element e is used to represent the service function. Then, further traverse the child elements of the two elements e and e′, and search for similar elements to the child elements of the element e in the child elements of the other element e′. Otherwise, if there are no similar elements to the element, stop searching for similar elements to the child elements of the element e.

[0083] In step (13), the determination that another element e′ is similar to the first element e includes: the other element e′ has a similar structure to the first element e, or the other element e′ has a similar style to the first element e, wherein:

[0084] (131) Another element e′ has a similar structure to the element e: the fuzziness of the path language of the elements is used to identify whether the elements have similar structures, that is, to find similar elements by ignoring the index of the ancestor elements in its path language.

[0085] (132) Another element e′ has a similar style to this element e: The style similarity between another element e′ and this element e is defined as the weighted average of their edit distances of class name, position, size, hyperlink href, external file reference src, and identifier id; if the style similarity between another element e′ and this element e is greater than or equal to a pre-set threshold, then the other element e′ is considered to have a similar style to this element.

[0086] Step 2: As Figure 4 As shown, in the extracted state, accessible elements are identified and actions for accessible elements are generated: Actions are identified by elements in the DOM tree. Elements in the original DOM tree are traversed. If an element e meets the predefined conditions, it is considered that there is an accessible element e in the action act, and the action act for the accessible element is generated.

[0087] The above-defined condition is one of the following three:

[0088] (21) If the tagName of an element e matches the default setting, then the element e is considered to have been operated on, and an action to access the element e is generated.

[0089] (22) If the label name of an element e is an input control <input> or multi-line plain text editing control <textarea>and the type of the one element e is one of radio and checkbox, the one element e is considered to be a clickable element, and an action of type click is generated for the one element e to access the one element e;< / textarea>

[0090] (3) If the label name and type of an element e match the user configuration, the element e is considered to be operable, and an action to access the element e is generated.

[0091] Step 3: Construct a state graph with states as nodes and actions that transition between states as edges, reflecting the transition relationships between states. The state graph includes a state set, an action set, actions that can trigger state transitions, and an initial state. The state M = (S, A, ∑, s0), where S is the state set, A is the action set, actions can trigger state transitions, ∑: S × A → S is the set of state transitions, and s0 is the initial state. When adding states and actions to the state graph, it is necessary to determine whether state s already exists in the state graph and whether the state is a known state, thereby avoiding redundant states in the state graph that could lead to state space explosion. For action act, traverse the action set A of the state graph and determine whether there is a known action act′ that is the same as action act, to avoid redundant actions in the state graph and avoid action space explosion.

[0092] When adding states and actions to the state diagram, the specific determination of whether they already exist in the state diagram is as follows:

[0093] (31) Compare the current state s one by one. i Is the current state s the same as the existing state s′? i The current state s is the same as the existing state s′. i Set it to the existing state s′; otherwise, if the current state s i Unlike the existing state s′, this indicates that the current state s i The current state s does not yet exist in the state set of state graph M. i Add the states to the state set of state graph M; then, add transitions to the state graph from the previous state to the current state s. i The transfer;

[0094] (32) In the current state s i At the same time, it is also necessary to set the current state s i Each action `act` is compared with an existing action `act'` in the state diagram. If the current action `act`... i If the current action is the same as an existing action, then the current action will not be added to the list. i Add to the action set; conversely, if the current action is not added to the action set... i If the current action is different from the existing action `act'`, then the current action `act` will be... i Add to action set A; for the current action act i Traverse the action set A of the state graph to check if there exists an existing action act′∈A and the current action act. iIf two actions are of the same type and access similar elements, then the two actions are considered to be the same.

[0095] In step (31), it is determined whether the current state s i When the current state is the same as an existing state s′, construct a state index tree and use similarity to determine if the current state s′ is the same as the existing state s′. i If the similarity between the current state and an existing state s′ in the state set is higher than a pre-set threshold, then the current state s is considered to be... i Same as the existing state s′;

[0096] The similarity determination is as follows:

[0097] Given the current state s i Construct a state index tree, and use similarity judgment to compare state s with the state index tree to determine whether there is an existing state s′ that matches the current state s in the state graph M. i same;

[0098] Specifically, when constructing the state index tree, the current state s is first... i All elements are labeled to indicate which state they belong to; then, the state index tree composed of existing states s′ is obtained. If the state index tree has not yet been built, the state index tree is set to the current state s. i And there is no state s that is the same as the current state s. i Same state;

[0099] If a state index tree exists, traverse the state index tree to calculate the current state s. i The number of similar elements between the current state s and the existing state s′; at the same time, the current state s is also considered. i The elements are merged into the state index tree, and the current state s is calculated. i The number of elements similar to the existing state s′;

[0100] The method for obtaining the number of similar elements between state s and the existing state s′ is as follows:

[0101] (1) Initialize simCnt to count the number of similar elements between state s and each existing state;

[0102] (2) Get the root elements sRoot and iRoot of state s and state index tree respectively, and get the child elements sChilds of element sRoot;

[0103] (3) The sub-elements sc∈sCkilds of sRoot and their descendants are merged into the state index subtree rooted at element iRoot by the function mgAndCnt(), and the number of similar elements between the descendants of element sc and the descendants of element iRoot is calculated.

[0104] (4) The function mgAndCnt() merges the element into the state index subtree rooted at element iRoot, and counts the number of similar elements between the subtree rooted at element (i.e., the substate) and the state index subtree rooted at element iRoot.

[0105] (5) First, obtain the child elements iChilds of element iRoot, and find the element simEle with the highest similarity to element element among these child elements. The element similarity between them is sim. If the element similarity sim is greater than the preset threshold eleSimThreshold, then element simElement and element are considered similar;

[0106] (6) Based on the label on the element simElement, we know that the element simElement belongs to the existing state stateId. Therefore, simCnt sub [stateId] is incremented by 1; furthermore, the child elements of element simElement are merged into the state index subtree rooted at element simElement, and the number of similar elements between the descendant elements of element simElement and the state index subtree rooted at element simElement is counted (simCnt). c Conversely, if the element similarity sim is less than the pre-set threshold eleSimThreshold, then the elements simElement and element are considered dissimilar.

[0107] (7) Add the element and edge (iRoot, element) to the state index tree using the function addSubTree().

[0108] Finally, it returns the number of elements in the subtree rooted at element that are similar to the existing state.

[0109] After obtaining the number of similar elements between state s and the existing state s′, the similarity between state s and the existing state s′ is calculated. The similarity between state s and the existing state s′ is defined as:

[0110]

[0111] Where #similarNum is the number of similar elements between state s and the existing state s′, and #s and #s′ represent the number of elements between state s and the existing state s′, respectively.

[0112] Determine the maximum similarity between state s and the existing state s′. If the maximum similarity is greater than a preset threshold, then state s and the existing state s′ are considered to be the same state; otherwise, state s and the existing state s′ are not the same state.

[0113] Step 4: Based on the states and actions in the state diagram, evaluate the contribution of each action to the execution of the state space, design a reward model, and calculate the reward for each action using the reward model; whenever the current action... i Execution complete, state changes from the previous state s i-1 Transition to the current state s i According to the reward model, the current action is... i Calculate rewards;

[0114] Reward model r i Defined as:

[0115]

[0116] If the current action is executed... i Then, the target application redirects to an external link, or the current state s i If it is the same as the previous state, then it is the current action (act). i Assign a negative reward, i.e., penalty < 0;

[0117] Conversely, for other cases, a positive reward r′ is calculated. i (act i ):

[0118] r′ i =w loc *r loc +w cur *r curiosity +w freq *r freq +w explore *r explore

[0119] Among them, w loc w cur w freq and w explore The corresponding weights for each reward indicator are all greater than 0;

[0120] The r loc For the current action i Position reward metric: Let the reward metric be the position of the previous action.i-1 and the current action i The elements being operated on are e i-1 and e i The agent chooses to access an element e. i The action, in which the element e i Adjacent element e i-1 r loc The formula is:

[0121]

[0122] Where h(e) and w(e) represent the height and width of element e, respectively, and dist(e) i-1 e i ) represents two adjacent elements e i-1 and e i The Leibniz distance between them;

[0123] The r curiosity For the current action i The reward metric for curiosity is the reward for achieving the desired result in the current state s. i The state s exists above the previous state s i-1 Elements that do not exist in r are variable elements. curiosity The calculation formula is:

[0124]

[0125] That is, if the current action is... i The element being accessed is a changing element, and the current action is... i The assigned reward value is 1 / mutants(s) i ),mutants(s i ) indicates the current state s i The number of changing elements;

[0126] The r freq For reward metrics based on the frequency of action transitions, if from the previous state s i-1 After the current action (act) i Transition to the current state s i The more times an action is executed, the higher its reward metric r is based on execution frequency. freq The smaller r is freq Defined as:

[0127]

[0128] Where, N i Indicates the state from the previous state s i-1 After the current action (act) i Transition to the current state si The number of times it is executed;

[0129] The r explore Based on the current state s i The reward metric refers to the performance of the current action. i Then, the target application transitions to the current state s. i If the current state s i The higher the percentage of actions that have never been executed, the better the current action (act) is. i It helps in testing the target application and assigning the current action to the application. i Reward metric r explore r explore The calculation formula is:

[0130]

[0131] Where, m i and n i These represent the current state s respectively. i The number of actions and the number of actions that have not been performed.

[0132] Step 5: Based on the state in Step 1, the action output in Step 2, and the reward output in Step 4, train the action selection policy using a reinforcement learning algorithm, select an action according to the action selection policy, and output the selected action.

[0133] The specific implementation of training an action selection strategy using reinforcement learning algorithms is as follows:

[0134] Given a web application, automatically generate a set of test cases T. First, record the URL of the target application's homepage and initialize the test case set T, strategy π, and state set S.

[0135] The process executes several cycles N. At the start of each cycle, the target application is reset by revisiting its homepage URL to reach homepage p0. Then, the state and actions are extracted from homepage p0. Within each cycle, an action sequence as is generated by executing several steps. In the i-th step, the action sequence is executed in the previous state s. i-1 Select the current action (act) i Execute the current action (act) i Then, the target application redirects to the current webpage. i From the current webpage p i Extract the current state s i And actionable actions, using the extracted current state s i Update the state set S and assign the current action (act) to the actions in the available actions. i Calculate the reward r for the current action i Based on the previous state s i-1Current action (act) i Current state s i The reward r for the current action i Update strategy π and set the current action act. i Add the action sequence as to the action sequence as generated in each cycle, and save the action sequence as generated in each cycle to the test case set T;

[0136] The update strategy π is implemented through the update function Q, and the update method is as follows:

[0137] Q(s i-1 , act i )←Q(s i-1 , act i )+α(r i +γQ * (s i a i+1 )-Q(s i-1 a i ))

[0138] Q(s i-1 , act i ) indicates the state in the previous state s i-1 Execute the current action (act) i Contribution to the target application being tested; when the current action is executed. i Make the previous state s i-1 From the current action i Transition to the current state s i At that time, calculate the current reward r. i And update function Q, Q * (s i+1 a i+1 ) indicates the state from the next state s i+1 The initial cumulative maximum contribution; the cumulative reward will be discounted by a discount rate γ∈[0,1], where α∈[0,1] represents the learning rate.

[0139] Step 6: Perform the action output in Step 5 on the web application to redirect the web application to another webpage;

[0140] Step 7: Continue with steps 1-6 and save the sequence of actions performed on the web application as test cases.

[0141] Based on the same inventive concept, another embodiment of the present invention provides an electronic device (computer, server, smartphone, etc.) including a memory and a processor, wherein the memory stores a computer program configured to be executed by the processor, and the computer program includes instructions for performing the steps of the method of the present invention.

[0142] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk), which stores a computer program that, when executed by a computer, implements the various steps of the method of the present invention.

[0143] The above embodiments are provided merely for the purpose of describing the present invention and are not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A method for automatically generating test cases for web applications based on reinforcement learning, characterized in that, Includes the following steps: Step 1: Extract state from webpage: Obtain the node tree of the webpage and initialize the state with the node tree; Then, the state is traversed to remove redundant node tree elements and their related edges that do not affect the page visualization, simplifying the number of elements in the state; the simplified node tree is called the rendering tree, and the elements in the rendering tree are traversed in a width-first manner. If two elements are similar, they are considered to have similar functions. For an element, if there is another element that is similar to it, the two elements are considered to have similar functions. Elements with similar functions are identified as the same state, thus extracting the state from the webpage. Step 2: Identify accessible elements in the extracted state and generate actions for accessible elements: Identify actions through elements in the node tree, traverse the elements in the original node tree, and if an element meets the predefined conditions, it is considered that there is an accessible element in the action, and an action for the accessible element is generated. Step 3: Construct a state graph with states as nodes and actions that transition between states as edges, reflecting the transition relationships between states. The state graph includes a state set, an action set, action-triggered state transitions, and an initial state. Furthermore, when adding the current state and current action to the state graph, it is necessary to determine whether the current state already exists in the state graph and whether the current state is a known state to avoid redundant states in the state graph; for the current action, it is necessary to traverse the action set of the state graph and determine whether there is a known action in the action set that is the same as the current action to avoid redundant actions in the state graph. Step 4: Based on the states and actions in the state diagram, evaluate the contribution of each action to the state execution, design a reward model, and calculate the reward for the action using the reward model; whenever an action is completed, the state transitions from the previous state to the current state, and the reward for the current action is calculated according to the reward model. Step 5: Based on the state in Step 1, the action output in Step 2, and the reward in Step 4, train the action selection policy using a reinforcement learning algorithm, select an action according to the action selection policy, and output the selected action. Step 6: Perform the action output in Step 5 on the web application to redirect the web application to another webpage; Step 7: Continue with steps 1-6 and save the sequence of actions performed on the web application as test cases.

2. The method for automatically generating test cases for web applications based on reinforcement learning according to claim 1, characterized in that: The specific implementation process of step 1 is as follows: (11) Obtain the DOM tree of the webpage and initialize the state with the DOM tree, that is, initialize the node set and edge set of the state with the node set and edge set of the DOM tree respectively; (12) Traverse the state and delete redundant elements. If an element has one and only one child element, delete the element from the node set of the state, delete the edge associated with the element from the edge set of the state, and add an edge from the parent element of the element to the child element of the element to the edge set of the state. (13) Perform a width-first traversal on the rendering tree. For an element, if there is another element that is similar to the element, then the element and the other element are considered to have the same service function, and the element is used to represent the service function. Then, further traverse the child elements of the two elements and search for similar elements to the child elements of the element in the child elements of the other element. Otherwise, if there are no similar elements to the element, stop searching for similar elements to the child elements of the element.

3. The method for automatically generating test cases for web applications based on reinforcement learning according to claim 2, characterized in that: In step (13), the determination that another element is similar to the first element includes: the other element has a similar structure to the first element, or the other element has a similar style to the first element, wherein: (131) Another element has a similar structure to this element: use the fuzziness of the path language of the elements to identify whether the elements have similar structures, and find similar elements by ignoring the index of the ancestor elements in its path language. (132) Another element has a similar style to this element: The style similarity between another element and this element is defined as the weighted average of their edit distances for class name, position, size, hyperlink, external file reference and identifier; if the style similarity between another element and this element is greater than or equal to a pre-set threshold, then the other element is considered to have a similar style to this element.

4. The method for automatically generating test cases for web applications based on reinforcement learning according to claim 1, characterized in that: In step 2, the predefined condition is one of the following three: (21) If an element’s tag name matches the default setting, then the element is considered to have been manipulated, and an action to access the element is generated. (22) If an element’s label name is an input control or a multiline plain text editing control, and the element’s type is either a radio button or a checkbox, then the element is considered a clickable element, and a clickable action is generated for the element to access the element. (23) If an element’s tag name and type match the user configuration, the element is considered operable, and an action to access the element is generated.

5. The method for automatically generating test cases for web applications based on reinforcement learning according to claim 1, characterized in that: In step 3, when adding states and actions to the state diagram, it is determined whether they already exist in the state diagram. The specific judgment is as follows: (31) Compare each state with an existing state. If the current state is the same as an existing state, set the current state as an existing state. Otherwise, if the current state is different from an existing state, it means that the current state does not yet exist in the state set of the state diagram. Add the current state to the state set of the state diagram. Then, add transitions to the state diagram that move from the previous state to the current state; (32) In the current state, each action in the current state needs to be compared with the existing actions in the state diagram. If the current action is the same as the existing action, the current action will not be added to the action set. Otherwise, if the current action is different from the existing action, the current action will be added to the action set. For an action, the action set of the state diagram is traversed to check if there is an existing action that is the same as the current action. If the two actions are of the same type and the elements they access are similar, then the two actions are considered to be the same.

6. The method for automatically generating test cases for web applications based on reinforcement learning according to claim 5, characterized in that: In step (1), if the current state is the same as an existing state, a state index tree is constructed and the similarity is used for judgment. If the similarity between the current state and an existing state in the state set is higher than a preset threshold, the current state is considered to be the same as an existing state. The similarity determination is as follows: Given a state, construct a state index tree. Use similarity to compare the state with the state index tree to determine if there is an existing state in the state graph that is the same as a certain state. Specifically, when constructing the state index tree, firstly, all elements in the state are labeled to indicate which state the element belongs to; then, the state index tree composed of existing states is obtained. If the state index tree has not yet been constructed, the state index tree is set to the current state, and there is no state that is the same as the current state. If a state index tree exists, traverse the state index tree and calculate the number of similar elements between the state and existing states; at the same time, merge the elements of the state into the state index tree and calculate the number of similar elements between the state and existing states. After obtaining the number of similar elements between the state and existing states, the similarity between the state and existing states is calculated. The similarity between the state and existing states is defined as follows: Where #similarNum is the number of similar elements between state s and the existing state s', #s and #s' represent the number of elements between state s and the existing state s', respectively; min represents the minimum. Determine the maximum similarity between the current state and the existing state. If the maximum similarity is greater than a preset threshold, the current state and the existing state are considered to be the same state; otherwise, the current state and the existing state are not the same state.

7. The method for automatically generating test cases for web applications based on reinforcement learning as described in claim 1, characterized in that: In step 4, the reward model is as follows: r i To calculate the reward, if the current state is external webpage retrieval, or the current state is the same as the previous state, then a negative reward (penalty < 0) is assigned to the current action; otherwise, a positive reward r' is calculated. i : r′ i =w loc *r loc +w cur *r curiosity +w freq *r freq +w explore *r explore Among them, w loc w cur w freq and w explore The corresponding weights for each reward indicator are all greater than 0; The r loc The reward metric is based on the current action position: it refers to the reward for the agent choosing to access an element e. i The action and the element e i Adjacent element e i-1 r loc The formula is: Where h(e) and w(e) each represent an element e i Height and width, dist(e i-1 ,e i ) indicates that it is related to the element e. i Adjacent element e i-1 The Leibniz distance between them; The r curiosity The reward metric is based on curiosity about the current action. Elements that exist in the current state but not in the previous state are considered changing elements. curiosity The calculation formula is: That is, if the element accessed by the current action is a mutable element, the reward value assigned to the current action is 1. i ),mutants(s i ) indicates the current state s i The number of changing elements; The r freq This is a reward metric based on the frequency of action transitions. It refers to the reward metric *r* based on the frequency of action execution, where the more times an action transitions from the previous state to the current state, the higher the reward metric *r*. freq The smaller r is freq Defined as: Where, N i This indicates the number of times the action transitions from the previous state to the current state. The r explore The reward metric based on the current state refers to the state to which the target application transitions after the current action is performed. The higher the proportion of actions that have never been performed in the current state, the more helpful the current action is in testing the target application, and thus the reward metric *r* is assigned to the current action. explore r explore The calculation formula is: Where, m i and n i These represent the number of actions in the current state and the number of actions that have not yet been executed, respectively.

8. The method for automatically generating test cases for web applications based on reinforcement learning as described in claim 1, characterized in that: Step 5, which uses a reinforcement learning algorithm to train the action selection strategy, is specifically implemented as follows: Given a web application, automatically generate a set of test cases. First, record the target application's homepage link and initialize the test case set, strategy, and state set. The process executes several cycles. At the start of each cycle, the target application is reset by revisiting its homepage link. Then, the state and actions are extracted from the homepage. Within each cycle, an action sequence is generated by executing several steps. In the i-th step, the current action is selected from the previous state. After executing the current action, the target application redirects to the current webpage, where the current state and available actions are extracted. The state set is updated using the extracted current state, and the reward for the current action is calculated based on the previous state, the current action, and the reward r for the current action. i Update the strategy and add the current action to the action sequence. The action sequence generated in each cycle is saved to the test case set. The update strategy is implemented through the update function Q, and the update method is as follows: Q(s i-1 ,act i )←Q(s i-1 ,act i )+α(r i +γQ * (s i ,a i+1 )-Q(s i-1 ,a i )) Q(s i-1 ,act i ) indicates the state in the previous state s i-1 Execute the current action (act) i Contribution to the target application being tested; when the current action is executed. i Make the previous state s i-1 From the current action i Transition to the current state s i At that time, calculate the reward r. i And update function Q, Q * (s i+1 ,a i+1 ) indicates the state from the next state s i+1 The initial cumulative maximum contribution; the cumulative reward will be discounted by a discount rate γ∈[0,1], where α∈[0,1] represents the learning rate.

9. A system for automatically generating test cases for web applications based on reinforcement learning, characterized in that, include: The module comprises a state extraction module, an action extraction module, a state graph construction module, a reward model module, and a reinforcement learning agent module; among which: State Extraction Module: Extracting states from web pages: Obtain the node tree of the web page and initialize the state with the node tree; then traverse the state to remove redundant elements of the node tree and their related edges that do not affect the page visualization, simplifying the number of elements in the state; the simplified node tree is called the rendering tree, and the elements on the rendering tree are traversed in a width-first manner. If two elements are similar, they are considered to have similar functions. For an element, if there is another element that is similar to it, the two elements are considered to have similar functions. Elements with similar functions are identified as the same state, thus extracting the state from the web page. Action extraction module: Identifies accessible elements in the extracted state and generates actions for accessible elements: Identifies actions through elements in the node tree, traverses the elements in the original node tree, and if an element meets the predefined conditions, it is considered that there is an accessible element in the action and an action for the accessible element is generated. State Graph Construction Module: Constructs a state graph with states as nodes and actions that transition between states as edges, reflecting the transition relationships between states. The state graph includes a state set, an action set, action-triggered state transitions, and an initial state. When adding the current state and current action to the state graph, it is necessary to determine whether the current state already exists in the state graph and whether the current state is a known state to avoid redundant states in the state graph. For the current action, the action set of the state graph is traversed to determine whether there is a known action in the action set that is the same as the current action to avoid redundant actions in the state graph. Reward Model Module: Based on the states and actions in the state diagram, evaluate the contribution of each action to the state execution, design a reward model, and calculate the reward for the action using the reward model; whenever the current action is completed, the state transitions from the previous state to the current state, and the reward is calculated for the current action according to the reward model. The reinforcement learning agent module trains an action selection strategy using reinforcement learning algorithms based on the state, action, and reward, selects an action according to the action selection strategy, and outputs the selected action. The output action is executed on the web application, causing the web application to jump to another webpage and continue executing the state extraction module, action extraction module, state graph construction module, and reward model module. The sequence of actions executed on the web application is saved as test cases.

10. An electronic device comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, characterized in that: The computer program executes the method according to any one of claims 1-8 of the present invention; or a computer-readable storage medium storing a computer program that executes the method according to any one of claims 1-8 of the present invention.

Images