Machine learning based test script adaptive generation execution method

Abstract

This invention relates to the field of computer software testing technology. It discloses a machine learning-based adaptive generation and execution method for test scripts. The method includes acquiring pre-action interface images, post-action interface images, and interaction response records; determining reference control nodes; generating displacement and occlusion relationships; forming an interface topology graph; generating a page constraint graph based on the interface topology graph and interaction response records; inputting the interface topology graph, page constraint graph, post-action interface images, and interaction response records into a preset graph neural network; outputting a continuation script fragment; executing the continuation script fragment; collecting and updating the interface images and execution records; and rewriting the continuation script fragment accordingly to obtain an anomaly reproduction script fragment. This maintains script execution continuity, compresses repeated traversals, and ensures complete reproduction of the anomaly path.

Description

Technical Field

[0001] This invention relates to the field of computer software testing technology, and more specifically, to a machine learning-based adaptive generation and execution method for test scripts. Background Technology

[0002] In application automation testing scenarios, test scripts typically run on terminal devices or test execution environments. Due to limitations such as real-time interface rendering, interaction response latency, and fluctuations in execution resources, existing technologies usually employ abstract processing approaches such as page state determination, control association matching, path recording and playback, and abnormal event filtering to maintain script execution and traversal progress. Such methods are generally suitable for conditions where the interface structure is relatively stable, page jump relationships are relatively fixed, and interference events change gradually.

[0003] During the traversal of complex business interfaces, frequent changes in page structure and intermittent appearance of unexpected pop-ups can weaken the consistency of page state recognition and path connection. This can lead to script execution interruption, repeated page entry, and missing abnormal trigger chains, ultimately making it difficult to form a continuously traceable abnormal reproduction path. Therefore, the technical problem that needs to be solved is how to ensure the complete reproduction of abnormal paths while maintaining the continuity of script execution and compressing repeated traversals.

[0004] In view of this, the present invention proposes a machine learning-based adaptive generation and execution method for test scripts to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned shortcomings of the prior art, this invention provides a machine learning-based adaptive generation and execution method for test scripts.

[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, it provides a machine learning-based adaptive generation and execution method for test scripts, including: Acquire the interface image before the action, the interface image after the action, and the interaction response record. Determine the reference control node based on the relative position, hierarchical connection, and event entry relationship of the control nodes in the interface images before and after the action. The displacement relationship is generated based on the positional change of the non-reference control node relative to the reference control node, and the occlusion relationship is generated based on the coverage change of the non-reference control node on the reference control node. The interface topology diagram is generated based on the displacement relationship and the occlusion relationship. A page constraint graph is generated based on the interface topology graph and interaction response records. The interface topology graph, page constraint graph, post-action interface image and interaction response records are input into a preset graph neural network, and the output is a continuation script fragment. The continuation script fragment is executed and the updated interface image and execution record are collected. Based on the updated interface image and execution record, the continuation script fragment is rewritten to obtain the exception reproduction script fragment.

[0007] In some embodiments, determining a reference control node based on the relative positional relationship, hierarchical connection relationship, and event entry point relationship of control nodes in the pre-action interface image and the post-action interface image includes: Extract the common control nodes from the pre-action interface image and the post-action interface image, and create a front list and a back list according to the front-back arrangement of controls in the same layer; Extract control nodes whose front and back arrangements have not been swapped from the front and back lists, and extract the upward branch chain and downward trigger chain corresponding to the control nodes from the current control tree; Extract the trigger source position and callback landing position corresponding to the control node from the interaction response record, and map the trigger source position to the downlink trigger chain and the callback landing position to the uplink branch chain; Retain the target control node that falls into both the upward branch chain mapping result and the downward trigger chain mapping result, and designate the target control node as the reference control node.

[0008] In some embodiments, extracting the trigger source location and callback endpoint location corresponding to the control node from the interaction response record includes: Extract the trigger registration items, callback registration items, and page return registration items corresponding to the control nodes from the interaction response records, and establish trigger registration fragments and return registration fragments according to the same trigger chain; Extract the trigger source container from the trigger registration fragment, extract the return landing point container from the return registration fragment, and map the trigger source container and the return landing point container to the container nodes in the current control tree, respectively. Tracing down the child chain carrying the event entry marker along the container node corresponding to the trigger source container, we obtain the trigger source location. Tracing up the parent chain that receives the page return along the container node corresponding to the return point container, we obtain the callback point location.

[0009] In some embodiments, an interface topology graph is generated based on displacement and occlusion relationships, including: A front reference area is established based on the distribution of reference control nodes in the interface image before the action, and a back reference area is established based on the distribution of reference control nodes in the interface image after the action. Extract the non-reference control nodes that fall between the front reference area and the back reference area, and pair the attachment positions of the non-reference control nodes relative to the front reference area with the attachment positions of the non-reference control nodes relative to the back reference area to obtain the displacement association relationship. Extract the non-reference control nodes that cover the display area of ​​the reference control node to obtain the occlusion relationship, and write the reference control node, non-reference control node, displacement relationship and occlusion relationship into the relationship graph to obtain the interface topology relationship graph.

[0010] In some embodiments, generating a page constraint diagram based on the interface topology diagram and interaction response records includes: Extract the control nodes that trigger the page to enter and the control nodes that trigger the page to return from the interaction response record, extract the original page jump relationship between the pages, and map the control nodes to the corresponding nodes in the interface topology diagram to obtain the entry mapping node and return mapping node. Extract the entry-to-arrival chain from the entry-mapping node along the interface topology graph, extract the return-to-acceptance chain from the return-to-mapping node along the interface topology graph, and extract the reference control node that falls into both the entry-to-arrival chain and the return-to-acceptance chain. Using the reference control node as the boundary, extract the page entry segment in the cut-in chain and the page return segment in the return chain; Write the page ingress segment and page return segment into the page navigation relationship, and replace the original page navigation relationship that crosses the page ingress segment and page return segment with a restricted navigation relationship to obtain the page constraint diagram.

[0011] In some embodiments, the page cut-in segment in the cut-in chain is extracted, using the reference control node as the boundary, including: Extract the forward node chain from the cut-in arrival chain. The forward node chain is located before the reference control node. Extract page entry records from the interaction response records, and the page entry records correspond to the forward node chain; Based on the page identifier change items in the page entry record, locate the entry start node in the forward node chain; Based on the outgoing edge position of the starting node in the interface topology graph, candidate segments are extracted, and the node chains carrying page entry records in the candidate segments are determined as page entry segments.

[0012] In some embodiments, locating the starting node in the forward node chain based on the page identifier change item in the page ingress record includes: Extract action trigger items, page load items, and first-time responsive control items from the page entry record; Map the control node corresponding to the action trigger item to the forward node chain, and map the control node corresponding to the first respondable control item to the forward node chain; Extract the control nodes between the control node corresponding to the action trigger item and the control node corresponding to the first respondable control item, and determine the control node that connects to the page loading item as the starting node for the cut-in.

[0013] In some embodiments, the continuation script fragment is rewritten based on the updated interface image and execution record to obtain the exception reproduction script fragment, including: Map the control nodes in the updated interface image to the interface topology graph, and extract the off-target control nodes that do not fall into the interface topology graph; Map the failure response items and page dwell items in the execution log to the continuation script fragments, and locate the deviation positions in the continuation script fragments; Based on the display area of ​​the off-axis control node in the updated interface image, extract the corresponding reference control node and occlusion relationship from the interface topology diagram; Based on the corresponding reference control node and the occlusion relationship, a deviation branch insertion statement is generated, and the deviation branch insertion statement and the failure response item are written into the deviation position in the continuation script fragment to obtain the exception reproduction script fragment.

[0014] In some embodiments, a deviation branch insertion statement is generated based on the corresponding reference control node and the occlusion association, including: Locate the display area that is off-center from the control node in the updated interface image, and extract the occlusion relationship that intersects with the display area from the interface topology graph; Tracing back along the occlusion relationship to the corresponding reference control node, and extracting the original operation statement of the corresponding reference control node from the continuation script fragment; Bind the original operation statement to the display area of ​​the off-target control node, and add the record statement corresponding to the failure response item to obtain the off-target branch insertion statement.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention acquires the interface image before the action, the interface image after the action, and the interaction response record. It determines the reference control node based on the relative position, hierarchical connection, and event entry point relationships of the control nodes. Then, it generates displacement relationships based on the positional changes of non-reference control nodes relative to the reference control node, and generates occlusion relationships based on the coverage changes of non-reference control nodes over the reference control node. Finally, it generates an interface topology graph based on the displacement and occlusion relationships, ensuring that page structure changes and pop-up occlusion are constrained within the same topology. Subsequently, it generates a page constraint graph based on the interface topology graph and the interaction response record. The interface topology graph, page constraint graph, the interface image after the action, and the interaction response record are input into a preset graph neural network, which outputs a continuation script fragment. This allows the script execution to continue along the current page relationship even after interface changes. Finally, it executes the continuation script fragment and collects updated interface images and execution records. Based on the updated interface images and execution records, it rewrites the continuation script fragment to obtain an anomaly reproduction script fragment. This maintains the continuity of script execution while compressing repetitive traversals, ensuring complete reproduction of the anomaly path. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the machine learning-based adaptive generation and execution method for test scripts in this invention. Figure 2 This is a schematic diagram of the structure of the machine learning-based test script adaptive generation and execution system in this invention; Figure 3 This is a flowchart illustrating the page cut-in segment extraction method in this invention; Figure 4 This is a flowchart illustrating the starting node positioning method in this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. In the following detailed description, many specific details are set forth to provide a thorough understanding of the exemplary embodiments described. However, it will be apparent to those skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other exemplary embodiments, well-known structures have not been described in detail to avoid unnecessarily obscuring the concepts of this disclosure. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention. Furthermore, the various aspects described in the embodiments may be combined arbitrarily without conflict.

[0018] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0019] Example 1 Figure 1 This disclosure illustrates a machine learning-based adaptive generation and execution method for test scripts, comprising at least one embodiment, including: S10: Obtain the interface image before the action, the interface image after the action, and the interaction response record. Determine the reference control node based on the relative position, hierarchical connection, and event entry relationship of the control nodes in the interface image before and after the action. In this embodiment, the pre-action interface image is the application interface image to be tested collected before the current interactive action is executed, and the post-action interface image is the application interface image to be tested collected after the current interactive action is executed. The interaction response record includes event records such as control triggering, page entry, page return, and callback registration. The reference control node refers to the control node that still has a structural reference function between the pre-action interface image and the post-action interface image. The reference control node is not simply a control whose position has not changed, but a control node that simultaneously satisfies the relative position maintenance relationship, the hierarchical connection maintenance relationship, and the event entry maintenance relationship.

[0020] Understandably, the relative position preservation relationship is used to eliminate misjudgment of controls caused by local rendering jitter of the page, the hierarchical connection preservation relationship is used to confirm that the parent-child link of the control node in the current control tree has not been replaced, and the event entry preservation relationship is used to confirm that the control node still inherits the same trigger source position and callback landing position. The above three types of relationships jointly constrain the reference control node, so that the reference control node can serve as the reference basis for generating subsequent displacement association relationships and occlusion association relationships.

[0021] For example, in a mobile business processing page, the interface image before the action includes a title bar control N1, a submit button N2, a back control N3, and a loading prompt control N4. The interface image after the action includes a title bar control N1, a submit button N2, a back control N3, and a pop-up close control N5. The interaction response record shows that the submit button N2 triggers the page transition, and the back control N3 receives the page return. If the arrangement relationship, control tree link, and event entry of the title bar control N1 and the back control N3 in the before and after interfaces are maintained, then the title bar control N1 and the back control N3 can be determined as reference control nodes.

[0022] The reference control nodes are determined based on the relative positions, hierarchical connections, and event entry points of the control nodes in the pre-action and post-action interface images, including: Extract the common control nodes from the pre-action interface image and the post-action interface image, and create a front list and a back list according to the front-back arrangement of controls in the same layer; Extract control nodes whose front and back arrangements have not been swapped from the front and back lists, and extract the upward branch chain and downward trigger chain corresponding to the control nodes from the current control tree; Extract the trigger source position and callback landing position corresponding to the control node from the interaction response record, and map the trigger source position to the downlink trigger chain and the callback landing position to the uplink branch chain; Retain the target control node that falls into both the upward branch chain mapping result and the downward trigger chain mapping result, and designate the target control node as the reference control node.

[0023] In this embodiment, the front list and the back list are used to record the front and back arrangement of common control nodes in the same layer of controls. The front list is derived from the interface image before the action, and the back list is derived from the interface image after the action. Control nodes whose front and back arrangement has not been swapped means that the adjacent control relationship of the control node in the two lists has not been replaced. This process does not require the control coordinates to be exactly the same, but uses the relative arrangement between control nodes as the page structure reference to reduce the impact of terminal resolution changes and local rendering jitter on the determination of reference control nodes.

[0024] Understandably, the upward branch chain and the downward trigger chain constrain control nodes from two directions in the current control tree. The upward branch chain represents the parent affiliation of the control node in the page structure, and the downward trigger chain represents the triggerable child chain associated with the control node. The trigger source position and callback landing point position are used to verify whether the control node still inherits the original interaction chain. Mapping the trigger source position to the downward trigger chain and the callback landing point position to the upward branch chain can exclude control nodes that are only stable in position in the interface image but whose event entry has changed.

[0025] For example, using the aforementioned mobile service processing page, the interface image before the action includes a title bar control N1, a submit button N2, a return control N3, and a loading prompt control N4. The interface image after the action includes a title bar control N1, a submit button N2, a return control N3, and a pop-up close control N5. The common control nodes are {N1, N2, N3}. The front list is N1, N2, N3, and the back list is N1, N2, N3. The upward branch chain of the return control N3 is page B, navigation bar, N3. The downward trigger chain is N3, the return event entry point. The callback landing point of the return control N3 in the interaction response record falls into the upward branch chain, and the trigger source position falls into the downward trigger chain. Therefore, the return control N3 can be used as the target control node and is determined as the reference control node.

[0026] Extract the trigger source position and callback landing point position corresponding to the control node from the interaction response record, including: Extract the trigger registration items, callback registration items, and page return registration items corresponding to the control nodes from the interaction response records, and establish trigger registration fragments and return registration fragments according to the same trigger chain; Extract the trigger source container from the trigger registration fragment, extract the return landing point container from the return registration fragment, and map the trigger source container and the return landing point container to the container nodes in the current control tree, respectively. Tracing down the child chain carrying the event entry marker along the container node corresponding to the trigger source container, we obtain the trigger source location. Tracing up the parent chain that receives the page return along the container node corresponding to the return point container, we obtain the callback point location.

[0027] In this embodiment, the trigger registration item is an event item in the interaction response record that records the source of the control trigger, the callback registration item is an event item in the interaction response record that records the return position of the control response, and the page return registration item is an event item in the interaction response record that records the page return link. Establishing trigger registration segments and return registration segments according to the same trigger link is to group the event records of the same control node in the trigger phase and return phase, so as to avoid mixing event records from different page branches into the link judgment of the same control node.

[0028] It is understandable that the trigger source container is the page container in the trigger registration fragment that carries the control trigger event, and the callback landing point container is the page container in the callback registration fragment that receives the page callback event. By mapping the trigger source container and the callback landing point container to the container nodes in the current control tree, the event position in the interaction response record can be converted into the structural position in the control tree, thereby providing a traceable tree-like link basis for determining the trigger source position and the callback landing point position.

[0029] Furthermore, by tracing down the sub-chain carrying the event entry marker along the container node corresponding to the trigger source container, the actual trigger entry point of the control node can be obtained. By tracing up the parent chain that receives the page return along the container node corresponding to the return point container, the callback point where the control node responds to the return can be obtained. This process combines the event record with the current control tree, so that the trigger source position and the callback point position no longer depend solely on the log field itself.

[0030] For example, using the aforementioned mobile service processing page, the interaction response record corresponding to the return control N3 includes trigger registration item E1, callback registration item E2, and page return registration item E3. The trigger registration fragment is {E1, N3, return event entry}, and the return registration fragment is {E2, E3, page A entry}. The trigger source container is the navigation bar container C1, and the return landing point container is the page A root container C2. In the current control tree, C1 corresponds to page B, the navigation bar, and C2 corresponds to page A, the root container. Tracing down from C1, we get N3, and the return event entry is used as the trigger source position. Tracing up from C2, we get page A, the root container, as the callback landing point position.

[0031] It should be noted that, through the above processing, the event items in the interaction response record can be transformed into the trigger source position and callback landing point position in the current control tree. Compared with the method of matching only based on the event name or page name, this processing can preserve the real trigger chain of the control node when the page jump relationship is disturbed by pop-ups or intermediate pages, which is convenient for subsequent judgment of the event entry to maintain the relationship and improves the reliability of the determination of the reference control node.

[0032] S20: Generate displacement relationships based on the positional changes of non-reference control nodes relative to reference control nodes, generate occlusion relationships based on the coverage changes of non-reference control nodes over reference control nodes, and generate an interface topology graph based on displacement and occlusion relationships; In this embodiment, a non-reference control node refers to a control node that has not been determined as a reference control node. The displacement relationship is used to represent the positional change of a non-reference control node relative to a reference control node. The occlusion relationship is used to represent the coverage change of a non-reference control node on the display area of ​​a reference control node. The interface topology diagram is used to carry reference control nodes, non-reference control nodes, displacement relationships, and occlusion relationships, so that page changes are no longer only manifested as changes in control positions, but as changes in the structural relationships between control nodes.

[0033] Understandably, reference control nodes provide stable references, while non-reference control nodes provide sources of change. By comparing the positional changes of non-reference control nodes relative to reference control nodes, we can identify partial page reflows, control drifting, and the insertion of new controls. By identifying the coverage changes of non-reference control nodes over reference control nodes, we can identify the interference of pop-ups, loading layers, and floating layer controls on the original business controls. By writing displacement relationships and occlusion relationships together into the interface topology diagram, we can preserve the source of the impact of page changes on the script execution path.

[0034] For example, using the aforementioned mobile business processing page, the reference control node is the return control N3, and the non-reference control nodes include the submit button N2, the loading prompt control N4, and the pop-up close control N5. Before the action, N2 is located in the form area below N3 in the interface image. After the action, N2 moves downward relative to N3 in the interface image, and N5 covers the display area of ​​N3. Thus, a displacement relationship is formed between N2 and N3, and an occlusion relationship is formed between N5 and N3. The interface topology diagram can be represented as {nodes: N2, N3, N5, displacement relationship: N2-N3, occlusion relationship: N5-N3}. This interface topology diagram is used to generate the page constraint diagram later.

[0035] An interface topology graph is generated based on displacement and occlusion relationships, including: A front reference area is established based on the distribution of reference control nodes in the interface image before the action, and a back reference area is established based on the distribution of reference control nodes in the interface image after the action. Extract the non-reference control nodes that fall between the front reference area and the back reference area, and pair the attachment positions of the non-reference control nodes relative to the front reference area with the attachment positions of the non-reference control nodes relative to the back reference area to obtain the displacement association relationship. Extract the non-reference control nodes that cover the display area of ​​the reference control node to obtain the occlusion relationship, and write the reference control node, non-reference control node, displacement relationship and occlusion relationship into the relationship graph to obtain the interface topology relationship graph.

[0036] In this embodiment, the front reference area is a region formed based on the distribution position of the reference control nodes in the interface image before the action, and the back reference area is a region formed based on the distribution position of the reference control nodes in the interface image after the action. The front and back reference areas are used to accommodate the interface position changes of the same group of reference control nodes before and after the interactive action. The attachment position is the connection position of the non-reference control node relative to the front or back reference area. By pairing the attachment positions, the displacement relationship of the non-reference control node relative to the reference control node can be determined without relying on the absolute coordinates of the control.

[0037] It is understandable that non-reference control nodes that cover the display area of ​​reference control nodes usually correspond to pop-ups, overlays, loading prompts, or occluding controls. Extracting these non-reference control nodes and generating occlusion relationships allows the interface topology diagram to simultaneously express both position migration and occlusion interference. Writing reference control nodes, non-reference control nodes, displacement relationships, and occlusion relationships into the relationship diagram is to convert interface changes into structured data that can be processed by subsequent page constraint diagrams and preset graph neural networks.

[0038] For example, using the aforementioned mobile service processing page, the reference control node is the return control N3, while the submit button N2 and the pop-up close control N5 are both non-reference control nodes. The relationship between the controls in the interface image before and after the action is shown in Table 1. Table 1: Example Table of Control Relationship Generation According to Table 1, the attachment position of the submit button N2 relative to the return control N3 changes, thus forming a displacement relationship between the submit button N2 and the return control N3. The pop-up close control N5 covers the display area of ​​the return control N3, thus forming an occlusion relationship between the pop-up close control N5 and the return control N3. The resulting interface topology diagram can be represented as {nodes: N2, N3, N5, displacement relationship: N2-N3, occlusion relationship: N5-N3}. This interface topology diagram serves as the basis for generating the page constraint diagram subsequently.

[0039] S30: Generate a page constraint graph based on the interface topology graph and interaction response records, and input the interface topology graph, page constraint graph, post-action interface image and interaction response records into a preset graph neural network to output a continuation script fragment; A page constraint diagram is generated based on the interface topology diagram and interaction response records, including: Extract the control nodes that trigger the page to enter and the control nodes that trigger the page to return from the interaction response record, extract the original page jump relationship between the pages, and map the control nodes to the corresponding nodes in the interface topology diagram to obtain the entry mapping node and return mapping node. Extract the entry-to-arrival chain from the entry-mapping node along the interface topology graph, extract the return-to-acceptance chain from the return-to-mapping node along the interface topology graph, and extract the reference control node that falls into both the entry-to-arrival chain and the return-to-acceptance chain. Using the reference control node as the boundary, extract the page entry segment in the cut-in chain and the page return segment in the return chain; Write the page ingress segment and page return segment into the page navigation relationship, and replace the original page navigation relationship that crosses the page ingress segment and page return segment with a restricted navigation relationship to obtain the page constraint diagram.

[0040] In this embodiment, the page constraint graph is a graph structure generated by combining the interface topology graph with the interaction response record. The interface topology graph is used to express the displacement and occlusion relationships between control nodes, and the interaction response record is used to express the page entry, page return, and original page jump relationships between pages. Combining the two to generate the page constraint graph can transform the structural changes at the control level into script path constraints at the page level, avoiding the pre-set graph neural network from selecting subsequent operations based solely on the interface image after the action.

[0041] It is understandable that the cut-in mapping node originates from the corresponding node in the interface topology diagram of the control node that triggers the page cut-in, and the return mapping node originates from the corresponding node in the interface topology diagram of the control node that triggers the page return. The cut-in arrival chain represents the node chain from the cut-in mapping node along the interface topology diagram to the reference control node, and the return acceptance chain represents the node chain from the return mapping node along the interface topology diagram to accept the page return. The reference control node that falls into both the cut-in arrival chain and the return acceptance chain can serve as the boundary between the page cut-in segment and the page return segment.

[0042] Furthermore, the page ingress segment represents the chain of control nodes that the page traverses when entering the target page from the original page, and the page return segment represents the chain of control nodes that the page traverses when returning from the target page to the original page. The restricted jump relationship is the jump relationship formed after replacing the original page jump relationship with the page ingress segment and the page return segment. Through the restricted jump relationship, the part of the original page jump relationship that crosses obscuring or offset controls can be gathered into the node chain that can be expressed by the interface topology diagram, thereby forming a page constraint diagram.

[0043] For example, using the aforementioned mobile service processing page, the interface topology diagram is {nodes: N2, N3, N5, displacement relationship: N2-N3, occlusion relationship: N5-N3}. In the interaction response record, the submit button N2 triggers a transition from page A to page B, and the return control N3 triggers a return from page B to page A. The original page jump relationship is page A - page B. The generation process of the page constraint diagram is shown in Table 2: Table 2: Example Table of Page Constraint Diagram Generation According to Table 2, the page entry segment N2-N3 and the page return segment N3-page A are written into the page jump relationship. The part of the original page jump relationship page A-page B that crosses the occlusion relationship N5-N3 is replaced with the restricted jump relationship, thus obtaining the page constraint graph. The page constraint graph, the interface topology relationship graph, the interface image after the action and the interaction response record are input into the preset graph neural network. The preset graph neural network outputs the continuation script fragment.

[0044] like Figure 3 As shown, using the reference control node as the boundary, the page cut-in segment in the cut-in chain is extracted, including: Extract the forward node chain from the cut-in arrival chain. The forward node chain is located before the reference control node. Extract page entry records from the interaction response records, and the page entry records correspond to the forward node chain; Based on the page identifier change items in the page entry record, locate the entry start node in the forward node chain; Based on the outgoing edge position of the starting node in the interface topology graph, candidate segments are extracted, and the node chains carrying page entry records in the candidate segments are determined as page entry segments.

[0045] In this embodiment, the forward node chain is the node chain located before the reference control node in the cut-in arrival chain, the page cut-in record is the record in the interaction response record used to represent the page entering the target page from the original page, the page identifier change item is used to indicate the location where the page switch occurs, the cut-in start node is the starting point of the page cut-in segment, and the cut-in candidate segment is the node chain that is cut off from the cut-in start node and along the outward edge direction of the interface topology graph.

[0046] Understandably, directly using the entire arrival chain as the page entry segment could easily include ordinary control operations before the page entry into the page constraint diagram. Therefore, it is necessary to first extract the forward node chain before the reference control node, and then use the page identifier change item in the page entry record to locate the entry start node, so that the page entry segment only contains the control node chain that actually participates in the page entry, avoiding including irrelevant dwell nodes or occluding nodes as part of the page entry segment.

[0047] For example, using the aforementioned mobile service processing page, the entry chain is N2, N5, N3, and the reference control node is N3. Then the forward node chain is N2, N5. The interaction response record shows that the submit button N2 triggers the page A to switch to page B. The page identifier change item is from page A to page B. The entry starting node is N2. In the interface topology diagram, the outgoing edge of N2 points to N3. At the same time, there is an occlusion relationship between N5 and N3. Therefore, the node chain carrying the page entry record is N2, N3. This node chain is determined as the page entry segment.

[0048] like Figure 4 As shown, based on the page identifier change item in the page ingress record, the starting node of the ingress in the forward node chain is located, including: Extract action trigger items, page load items, and first-time responsive control items from the page entry record; Map the control node corresponding to the action trigger item to the forward node chain, and map the control node corresponding to the first respondable control item to the forward node chain; Extract the control nodes between the control node corresponding to the action trigger item and the control node corresponding to the first respondable control item, and determine the control node that connects to the page loading item as the starting node for the cut-in.

[0049] In this embodiment, the action trigger item is the control event that triggers the page switching recorded in the page entry record, the page loading item is the event that the target page starts loading recorded in the page entry record, and the first respondable control item is the first control event that can receive interaction after the page entry. By extracting the action trigger item, the page loading item, and the first respondable control item at the same time, the page entry process can be limited to the control trigger, page loading, and control response, avoiding the need to locate the entry start node solely based on changes in the page name.

[0050] It is understandable that mapping the control node corresponding to the action trigger and the control node corresponding to the first respondable control to the forward node chain is to determine the front and back boundaries of the page cut-in in the same node chain. The control node located between the two control nodes and connected to the page loading item inherits both the trigger action and the page loading process, so it can be determined as the cut-in starting node.

[0051] For example, using the aforementioned mobile service processing page, the forward node chain is N2, N5. The action trigger item in the page entry record corresponds to the submit button N2, the page loading item corresponds to the page B loading event L1, the first respondable control item corresponds to the return control N3 in page B, the control node corresponding to the action trigger item is mapped to N2 after the forward node chain, the control node corresponding to the first respondable control item is mapped to N3 after the entry arrival chain, and the control node connecting N2 and N3 to the page loading item L1 is N2. Therefore, N2 is determined as the entry starting node.

[0052] It should be noted that, through the above processing, the starting node for the page transition is jointly defined by the action trigger item, the page loading item, and the first respondable control item. Compared with the method of determining the starting node for the page transition solely based on the first record of the page identifier change item, this processing can eliminate the influence of intermediate interfering controls such as the pop-up close control N5 on the page transition boundary, making the page transition segment closer to the actual page switching chain.

[0053] S40: Execute the continuation script fragment and collect the updated interface image and execution record. Rewrite the continuation script fragment based on the updated interface image and execution record to obtain the exception reproduction script fragment.

[0054] Based on the updated interface image and execution record, the continuation script fragment was rewritten to obtain the exception reproduction script fragment, including: Map the control nodes in the updated interface image to the interface topology graph, and extract the off-target control nodes that do not fall into the interface topology graph; Map the failure response items and page dwell items in the execution log to the continuation script fragments, and locate the deviation positions in the continuation script fragments; Based on the display area of ​​the off-axis control node in the updated interface image, extract the corresponding reference control node and occlusion relationship from the interface topology diagram; Based on the corresponding reference control node and the occlusion relationship, a deviation branch insertion statement is generated, and the deviation branch insertion statement and the failure response item are written into the deviation position in the continuation script fragment to obtain the exception reproduction script fragment.

[0055] In this embodiment, the updated interface image is the interface image captured after executing the continuation script fragment. The execution record includes the execution status of each operation statement in the continuation script fragment, failure response items, and page dwell items. The off-target control node is the control node in the updated interface image that failed to be mapped to the interface topology diagram. The off-target position is the script position in the continuation script fragment corresponding to the failure response item and the page dwell item.

[0056] It is understandable that mapping the control nodes in the updated interface image to the interface topology diagram is to determine whether the current interface is still within the known control relationship range. Mapping the failure response items and page dwell items to the continuation script fragment is to determine the location where the script is interrupted or dwelled. Combining the two can simultaneously locate the deviation objects on the interface side and the deviation positions on the script side, avoiding rerunning the entire script based solely on the execution failure record.

[0057] Furthermore, based on the display area of ​​the deviated control node in the updated interface image, the corresponding reference control node and occlusion relationship are extracted from the interface topology diagram. It can be determined whether the deviated control node occludes the reference control node. The deviation branch insertion statement is a script statement inserted into the continuation script fragment. This deviation branch insertion statement carries the occlusion handling, original operation and failure response record, so that the exception reproduction script fragment can retain the interface state and operation context when the deviation occurs.

[0058] For example, using the aforementioned mobile service processing page, the pre-defined continuation script fragment output by the graph neural network includes clicking the submit button N2, waiting for page B to load, and clicking the back control N3. After executing the continuation script fragment, an updated interface image is captured. In the updated interface image, a pop-up close control N5 appears and covers the back control N3. In the execution record, clicking the back control N3 corresponds to the failure response item F1 and the page stay item T1. When N5 is mapped to the interface topology graph, N5 corresponds to the occlusion relationship N5-N3. After F1 and T1 are mapped to the continuation script fragment, the offset position is located before clicking the back control N3.

[0059] Generate offset branch insertion statements based on the corresponding reference control node and occlusion relationship, including: Locate the display area that is off-center from the control node in the updated interface image, and extract the occlusion relationship that intersects with the display area from the interface topology graph; Tracing back along the occlusion relationship to the corresponding reference control node, and extracting the original operation statement of the corresponding reference control node from the continuation script fragment; Bind the original operation statement to the display area of ​​the off-target control node, and add the record statement corresponding to the failure response item to obtain the off-target branch insertion statement.

[0060] In this embodiment, the display area of ​​the off-target control node is used to characterize the occupied range of the off-target control node in the updated interface image. The occlusion association relationship intersecting with the display area is used to characterize the coverage relationship formed by the off-target control node on the existing reference control node. Tracing back to the corresponding reference control node along the occlusion association relationship is to re-associate the newly added interference object in the updated interface image to the original operation position in the continuation script fragment, so as to avoid treating the off-target control node as a new independent path.

[0061] It is understandable that the original operation statement is a continuation of the existing script statement in the script fragment that targets the corresponding reference control node. Binding the original operation statement to the display area of ​​the offset control node is to preserve the interface condition of "the original operation object is obscured by the offset control node" in the exception reproduction script fragment. Adding the record statement corresponding to the failure response item is to preserve the response result of execution failure in the script fragment, so that the offset branch insertion statement simultaneously includes the operation object, the obscured object, and the failure response.

[0062] It should be noted that, through the above processing, the deviation branch insertion statement is not simply inserted to close the pop-up or retry the operation. Instead, it binds the display area of ​​the deviation control node, the occlusion relationship, the corresponding reference control node, and the original operation statement together. Compared with the method of only recording the screenshot of the abnormal page or re-executing the original operation statement, this processing can retain the source of the interface occlusion and the position of the script interruption when the exception occurs, so that the exception reproduction script fragment has a traceable reproduction chain.

[0063] Example 2 Please see Figure 2 As shown, based on the same inventive concept, this embodiment discloses a machine learning-based adaptive test script generation and execution system. For details not covered in this embodiment, please refer to the relevant sections of Embodiment 1. The system includes: Reference determination module: used to acquire the interface image before the action, the interface image after the action, and the interaction response record, and to determine the reference control node based on the relative position relationship, hierarchical connection relationship, and event entry relationship of the control nodes in the interface image before the action and the interface image after the action. Topology generation module: used to generate displacement relationships based on the positional changes of non-reference control nodes relative to reference control nodes, generate occlusion relationships based on the coverage changes of non-reference control nodes on reference control nodes, and generate an interface topology graph based on displacement and occlusion relationships; Script continuation module: Used to generate a page constraint graph based on the interface topology graph and interaction response records, and input the interface topology graph, page constraint graph, post-action interface image and interaction response records into a preset graph neural network to output continuation script fragments; Reproducibility and rewriting module: This module executes the continuation script fragment and collects updated interface images and execution records. Based on the updated interface images and execution records, it rewrites the continuation script fragment to obtain the exception reproduction script fragment.

[0064] The detailed description above, in conjunction with the accompanying drawings, describes examples but does not represent all examples that can be implemented or fall within the scope of the claims. The terms “example” and “exemplary” are used in this specification to mean “serving as an example, instance or illustration” and do not mean “superior to or better than other examples”.

[0065] Throughout this specification, the phrase "an embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Therefore, the use of these phrases may refer to more than one embodiment. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0066] It should also be noted that these embodiments may be described as processes depicted as flowcharts, structural diagrams, or block diagrams. Although a flowchart may describe the operations as sequential processes, many of these operations can be performed in parallel or concurrently, and the order of these operations may be rearranged.

Claims

1. A method for adaptive generation of test scripts based on machine learning for execution, characterized in that, include: Acquire the interface image before the action, the interface image after the action, and the interaction response record. Determine the reference control node based on the relative position, hierarchical connection, and event entry relationship of the control nodes in the interface images before and after the action. The displacement relationship is generated based on the positional change of the non-reference control node relative to the reference control node, and the occlusion relationship is generated based on the coverage change of the non-reference control node on the reference control node. The interface topology diagram is generated based on the displacement relationship and the occlusion relationship. A page constraint graph is generated based on the interface topology graph and interaction response records. The interface topology graph, page constraint graph, post-action interface image and interaction response records are input into a preset graph neural network, and the output is a continuation script fragment. The continuation script fragment is executed and the updated interface image and execution record are collected. Based on the updated interface image and execution record, the continuation script fragment is rewritten to obtain the exception reproduction script fragment.

2. The machine learning based test script adaptive generation execution method according to claim 1, wherein, The reference control nodes are determined based on the relative positions, hierarchical connections, and event entry points of the control nodes in the pre-action and post-action interface images, including: Extract the common control nodes from the pre-action interface image and the post-action interface image, and create a front list and a back list according to the front-back arrangement of controls in the same layer; Extract control nodes whose front and back arrangements have not been swapped from the front and back lists, and extract the upward branch chain and downward trigger chain corresponding to the control nodes from the current control tree; Extract the trigger source position and callback landing position corresponding to the control node from the interaction response record, and map the trigger source position to the downlink trigger chain and the callback landing position to the uplink branch chain; Retain the target control node that falls into both the upward branch chain mapping result and the downward trigger chain mapping result, and designate the target control node as the reference control node.

3. The machine learning based test script adaptive generation execution method according to claim 2, wherein, Extract the trigger source position and callback landing point position corresponding to the control node from the interaction response record, including: Extract the trigger registration items, callback registration items, and page return registration items corresponding to the control nodes from the interaction response records, and establish trigger registration fragments and return registration fragments according to the same trigger chain; Extract the trigger source container from the trigger registration fragment, extract the return landing point container from the return registration fragment, and map the trigger source container and the return landing point container to the container nodes in the current control tree, respectively. Tracing down the child chain carrying the event entry marker along the container node corresponding to the trigger source container, we obtain the trigger source location. Tracing up the parent chain that receives the page return along the container node corresponding to the return point container, we obtain the callback point location.

4. The machine learning based test script adaptive generation execution method of claim 1, wherein, An interface topology graph is generated based on displacement and occlusion relationships, including: A front reference area is established based on the distribution of reference control nodes in the interface image before the action, and a back reference area is established based on the distribution of reference control nodes in the interface image after the action. Extract the non-reference control nodes that fall between the front reference area and the back reference area, and pair the attachment positions of the non-reference control nodes relative to the front reference area with the attachment positions of the non-reference control nodes relative to the back reference area to obtain the displacement association relationship. Extract the non-reference control nodes that cover the display area of ​​the reference control node to obtain the occlusion relationship, and write the reference control node, non-reference control node, displacement relationship and occlusion relationship into the relationship graph to obtain the interface topology relationship graph.

5. The machine learning-based adaptive generation and execution method for test scripts according to claim 1, characterized in that, A page constraint diagram is generated based on the interface topology diagram and interaction response records, including: Extract the control nodes that trigger the page to enter and the control nodes that trigger the page to return from the interaction response record, extract the original page jump relationship between the pages, and map the control nodes to the corresponding nodes in the interface topology diagram to obtain the entry mapping node and return mapping node. Extract the entry-to-arrival chain from the entry-mapping node along the interface topology graph, extract the return-to-acceptance chain from the return-to-mapping node along the interface topology graph, and extract the reference control node that falls into both the entry-to-arrival chain and the return-to-acceptance chain. Using the reference control node as the boundary, extract the page entry segment in the cut-in chain and the page return segment in the return chain; Write the page ingress segment and page return segment into the page navigation relationship, and replace the original page navigation relationship that crosses the page ingress segment and page return segment with a restricted navigation relationship to obtain the page constraint diagram.

6. The machine learning-based adaptive generation and execution method for test scripts according to claim 5, characterized in that, Using the reference control node as the boundary, extract the page entry segment in the path that leads to the cut-in chain, including: Extract the forward node chain from the cut-in arrival chain. The forward node chain is located before the reference control node. Extract page entry records from the interaction response records, and the page entry records correspond to the forward node chain; Based on the page identifier change items in the page entry record, locate the entry start node in the forward node chain; Based on the outgoing edge position of the starting node in the interface topology graph, candidate segments are extracted, and the node chains carrying page entry records in the candidate segments are determined as page entry segments.

7. The machine learning-based adaptive generation and execution method for test scripts according to claim 6, characterized in that, Based on the page identifier change entries in the page ingress record, locate the ingress starting node in the forward node chain, including: Extract action trigger items, page load items, and first-time responsive control items from the page entry record; Map the control node corresponding to the action trigger item to the forward node chain, and map the control node corresponding to the first respondable control item to the forward node chain; Extract the control nodes between the control node corresponding to the action trigger item and the control node corresponding to the first respondable control item, and determine the control node that connects to the page loading item as the starting node for the cut-in.

8. The machine learning-based adaptive generation and execution method for test scripts according to claim 1, characterized in that, Based on the updated interface image and execution record, the continuation script fragment was rewritten to obtain the exception reproduction script fragment, including: Map the control nodes in the updated interface image to the interface topology graph, and extract the off-target control nodes that do not fall into the interface topology graph; Map the failure response items and page dwell items in the execution log to the continuation script fragments, and locate the deviation positions in the continuation script fragments; Based on the display area of ​​the off-axis control node in the updated interface image, extract the corresponding reference control node and occlusion relationship from the interface topology diagram; Based on the corresponding reference control node and the occlusion relationship, a deviation branch insertion statement is generated, and the deviation branch insertion statement and the failure response item are written into the deviation position in the continuation script fragment to obtain the exception reproduction script fragment.

9. The machine learning-based adaptive generation and execution method for test scripts according to claim 8, characterized in that, Generate offset branch insertion statements based on the corresponding reference control node and occlusion relationship, including: Locate the display area that is off-center from the control node in the updated interface image, and extract the occlusion relationship that intersects with the display area from the interface topology graph; Tracing back along the occlusion relationship to the corresponding reference control node, and extracting the original operation statement of the corresponding reference control node in the continuation script fragment; Bind the original operation statement to the display area of ​​the off-target control node, and add the record statement corresponding to the failure response item to obtain the off-target branch insertion statement.

Images