Interface jumping method and device, equipment and storage medium

Abstract

The invention provides an interface jumping method and device, equipment and a storage medium, and belongs to the technical field of computers. According to the method, the shortest path for jumping from the first interface to the second interface and the multiple operation behaviors on the shortest path are found according to the operation track model without recording the operation for jumping from the first interface to the second interface or saving the operation as a script, and then interface jumping is realized by executing the multiple operation behaviors, so that the interface jumping speed is improved, and the interface jumping efficiency is improved. The method can be applied to automatic testing and other scenes, has high expansion performance, and has high robustness due to the fact that the multiple operation behaviors are behaviors in the operation track model, and the risk of misoperation of an interface does not exist.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an interface jump method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology, computer devices include more and more interfaces. Users can jump between different interfaces through a series of operations to browse the contents of different interfaces. Correspondingly, computer devices can record user operations and save them as scripts. By executing the script, the computer device can realize interface playback, that is, realize interface jumps in sequence according to the user's operation mode and browsing order. However, the above-mentioned interface playback method is completely dependent on the script, with poor scalability and low robustness, and there is a risk of misoperation of the interface during playback. Therefore, a new interface playback method is needed to realize automatic interface jump. Summary of the invention

[0003] The embodiments of the present application provide an interface jump method, device, equipment and storage medium, which can be applied to scenarios such as automated testing, have high scalability, and have no risk of misoperating the interface, and have high robustness.

[0004] According to one aspect of an embodiment of the present application, an interface jump method is provided, the method comprising:

[0005] Obtain a first interface and a second interface, where the first interface is the current interface and the second interface is the target interface to jump to;

[0006] Determine, according to the operation trajectory model, an interface jump trajectory event of jumping from the first interface to the second interface, wherein the nodes in the operation trajectory model represent interfaces, the directed edges between the nodes represent the jump relationship between the nodes based on the operation behavior, and the interface jump trajectory event includes the shortest path from the first interface to the second interface, multiple operation behaviors in the shortest path, and the time sequence between the multiple operation behaviors;

[0007] The interface jump track event is executed to display a jump from the first interface to the second interface in sequence.

[0008] According to another aspect of an embodiment of the present application, an interface jump device is provided, the device comprising:

[0009] An acquisition module, used to acquire a first interface and a second interface, wherein the first interface is a current interface and the second interface is a target interface to jump to;

[0010] a determination module, configured to determine, according to an operation trajectory model, an interface jump trajectory event of jumping from the first interface to the second interface, wherein nodes in the operation trajectory model represent interfaces, directed edges between nodes represent jump relationships between nodes based on operation behaviors, and the interface jump trajectory event includes a shortest path from the first interface to the second interface, multiple operation behaviors in the shortest path, and a time sequence between the multiple operation behaviors;

[0011] An execution module is used to execute the interface jump track event, and display the jump from the first interface to the second interface in sequence.

[0012] In some embodiments, the determination module is used to traverse the edge information in the operation trajectory model through a breadth-first traversal algorithm to determine the shortest path, and the edge information includes information used to characterize the operation behavior for implementing the jump between nodes and the node jump situation before and after the operation behavior; based on the nodes and edges included in the shortest path, the interface jump trajectory event is determined.

[0013] In some embodiments, the execution module is used to obtain the fifth interface to be jumped to in response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface during the execution of the interface jump trajectory event; if the shortest path passes through the fifth interface and the timing of the fifth interface is later than the fourth interface, continue to execute the interface jump trajectory event with the fifth interface as the starting point; if the shortest path passes through the fifth interface and the timing of the fifth interface is earlier than the fourth interface, determine that the shortest path is unreachable, regenerate and execute the interface jump trajectory event containing other paths; if the shortest path does not pass through the fifth interface, regenerate and execute the interface jump trajectory event containing the shortest path from the fifth interface to the second interface.

[0014] In some embodiments, the execution module is used to determine the shortest path from the fifth interface to the second interface if the shortest path does not pass through the fifth interface; regenerate an interface jump trajectory event based on the nodes and edges contained in the shortest path; and execute the regenerated interface jump trajectory event.

[0015] In some embodiments, the execution module is used to, during the execution of the interface jump trajectory event, in response to jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, immediately execute the second operation behavior on the fourth interface; in response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, if the currently displayed interface is not the fourth interface after a preset time period, stop executing the interface jump trajectory event.

[0016] In some embodiments, the execution module is used to stop executing the interface jump trajectory event in response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, if after a preset time, the currently displayed interface is not the interface passed by the target path.

[0017] In some embodiments, the determination module is further used to determine other paths when the shortest path is unreachable; determine a new interface jump trajectory event according to the nodes and edges included in the other paths;

[0018] The execution module is also used to execute the new interface jump track event.

[0019] In some embodiments, the plurality of operating behaviors include triggering controls on the interface;

[0020] In the process of executing the interface jump track event, when the content of the interface changes, the execution module is used to execute the corresponding operation behavior based on any of the following methods:

[0021] Determine the target control corresponding to the operation behavior according to the control identifier, and trigger the target control;

[0022] In the case where the control identifier is repeated, determining a target control corresponding to the operation behavior based on the relative position of the content in the interface before and after the change, and triggering the target control;

[0023] Based on the node factor of the node corresponding to the interface, determine the target control corresponding to the operation behavior, and trigger the target control, wherein the node factor includes node sub-factors that characterize the interface features of the interface from different dimensions;

[0024] Based on the large language model, the operation behavior after the content of the interface occurs is predicted through a one-shot strategy, and the predicted behavior is executed.

[0025] In some embodiments, the acquisition module is used to parse the input data to obtain the second interface, and the input data is any one of text, voice or options.

[0026] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the interface jump method described in the above aspects.

[0027] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the interface jump method described in the above aspect.

[0028] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the interface jump method described in the above aspects.

[0029] The embodiment of the present application provides an interface jump solution, by specifying the target interface to jump to, the shortest operation path from the current interface to the target interface can be determined based on the jump relationship between interfaces based on operation behaviors according to the operation trajectory model, and then the operation behaviors can be executed in sequence according to the shortest operation path, so as to realize the jump from the current interface to the target interface in sequence. The above method does not need to record the operation of jumping from the first interface to the second interface, nor does it need to be saved as a script. Instead, it finds the shortest path from the first interface to the second interface and multiple operation behaviors on the shortest path according to the operation trajectory model, and then realizes the interface jump by executing the multiple operation behaviors. It can be applied to scenarios such as automated testing, has high scalability, and because the above multiple operation behaviors are behaviors in the operation trajectory model, there is no risk of misoperation of the interface, and has high robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0032] Figure 2 is a schematic diagram of an operation trajectory model provided according to an embodiment of the present application;

[0033] Figure 3 This is a flow chart of an interface jump method provided by an embodiment of the present application;

[0034] Figure 4 is a flow chart of another interface jump method provided in an embodiment of the present application;

[0035] Figure 5 is a schematic diagram of determining the shortest path provided according to an embodiment of the present application;

[0036] Figure 6 It is a schematic diagram of a flow chart of determining a control on an interface according to an embodiment of the present application;

[0037] Figure 7 It is a structural schematic diagram of an interface jump device provided in an embodiment of the present application;

[0038] Figure 8 It is a structural block diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] In this article, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0041] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the interfaces involved in this application are all obtained with full authorization.

[0042] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 , the implementation environment includes a terminal 101 and a server 102. The terminal 101 can be directly or indirectly connected to the server 102 through wired or wireless communication.

[0043] In some embodiments, the terminal 101 may be a mobile phone, a desktop computer, a laptop computer, a tablet computer, etc. The terminal 101 may have various applications installed and running, and interfaces included in different applications may not be completely the same.

[0044] In some embodiments, server 102 is an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0045] In some embodiments, the server 102 undertakes the main computing work and the terminal 101 undertakes the secondary computing work; or, the server 102 undertakes the secondary computing work and the terminal 101 undertakes the main computing work; or, the server 102 and the terminal 101 adopt a distributed computing architecture to perform collaborative computing.

[0046] The interface jump method provided in this application is implemented through an operation trajectory model. The operation trajectory model (Trajectory Structure) is introduced below.

[0047] Figure 2 is a schematic diagram of an operation trajectory model provided according to an embodiment of the present application. Figure 2 As shown, Figure 2 (a) in the figure shows an operation process by way of example, wherein the interface through which the operation process passes is identified as 201, and the operation behavior is identified as 202. Accordingly, Figure 2 (b) in the figure exemplarily shows some nodes in the operation trajectory model 203 and the directed edges between the some nodes. The directed edges correspond to the operation behaviors in the interface and are used to connect different nodes (interface switching occurs after the operation) or connect the same node (no interface switching occurs after the operation). Among them, nodes 2031-node 2035 represent interface 1-interface 5 respectively. It should be noted that the node includes node information, which is used to characterize the interface features of the interface from different dimensions to ensure the accuracy of subsequent interface retrieval and positioning; the directed edge includes edge information, which is used to indicate the operation behavior and the node jump situation before and after the operation behavior, to ensure the accurate restoration of the operation in the subsequent interface. Optionally, multiple nodes and directed edges in the operation trajectory model constitute at least one operation trajectory path. The operation trajectory path includes path information, which is used to characterize the sequence between the indication interface and different operation behaviors to ensure that complex operations are performed in sequence.

[0048] Figure 3 is a flow chart of an interface jump method provided in an embodiment of the present application. The method is executed by a computer device, see Figure 3 , the method comprising:

[0049] 301. Obtain a first interface and a second interface, where the first interface is a current interface and the second interface is a target interface to jump to.

[0050] In an embodiment of the present application, the first interface is the interface currently displayed by the computer device. The computer device can identify the current interface through a visual service to determine the interface identifier of the current interface, and can also determine the interface identifier of the current interface through the code at the bottom of the interface. The second interface is the target interface to which you want to jump. The second interface can be determined based on task information or input data. Accordingly, the computer device can parse the task information or input data to determine the interface identifier of the second interface. Among them, the input data can describe the function or describe the interface content, and the second interface includes the function or the interface content.

[0051] 302. Determine an interface jump track event from the first interface to the second interface according to the operation track model.

[0052] In the embodiment of the present application, the nodes in the operation trajectory model represent interfaces, and the directed edges between the nodes represent the jump relationships between the nodes based on the operation behaviors, see Figure 2 As shown, no further details are given here. The interface jump trajectory event includes the shortest path from the first interface to the second interface, multiple operation behaviors in the shortest path, and the timing between multiple operation behaviors. For example, the interface jump trajectory event includes: event 1→interface a→event 2→interface b→event 3→interface c. Among them, event 1, event 2 and event 3 are multiple operation behaviors. Interface a, interface b and interface c are interfaces on the shortest path.

[0053] 303. Execute an interface jump track event, and display the jump from the first interface to the second interface in sequence.

[0054] In an embodiment of the present application, the computer device can execute the above-mentioned interface jump track event with the first interface as the starting point, thereby starting from the first interface, jumping to other interfaces in sequence, and finally jumping to the second interface.

[0055] The interface jump method provided in the embodiment of the present application can determine the shortest operation path from the current interface to the target interface by specifying the target interface to jump to, based on the jump relationship between interfaces based on operation behaviors according to the operation trajectory model, and then execute the operation behaviors in sequence according to the shortest operation path, so as to realize the jump from the current interface to the target interface in sequence. The above method does not need to record the operation of jumping from the first interface to the second interface, nor does it need to be saved as a script. Instead, it finds the shortest path from the first interface to the second interface and multiple operation behaviors on the shortest path according to the operation trajectory model, and then realizes the interface jump by executing the multiple operation behaviors. It can be applied to scenarios such as automated testing, has high scalability, and because the above multiple operation behaviors are behaviors in the operation trajectory model, there is no risk of misoperation of the interface, and has high robustness.

[0056] Figure 4 is a flowchart of another interface jump method provided by an embodiment of the present application. The method is executed by a computer device, see Figure 4 , the method comprising:

[0057] 401. Obtain a first interface and a second interface.

[0058] In the embodiment of the present application, the first interface is the current interface, and the second interface is the target interface to jump to. The method for obtaining the first interface and the second interface is the same as the above step 301, and will not be repeated here.

[0059] In some embodiments, the computer device may parse the input data to obtain the second interface, wherein the input data is any one of text, voice or options.

[0060] For example, if the user inputs "open xx interface" by voice, or enters "xx function" in the search box, or selects "xx option", the computer device can determine the interface that the user wants to open by parsing the input data, and use that interface as the second interface.

[0061] It should be noted that the interface of a computer device is usually represented by an activity, but in actual scenarios, there are cases where a single activity corresponds to multiple fragments, or a single activity corresponds to multiple pages. In this case, it is difficult to distinguish the two interfaces based on the name of the activity, so the interface cannot be distinguished directly based on the name of the activity.

[0062] In order to solve this problem, in the embodiment of the present application, the interface is represented by a node, and based on the concept of the node, the concept of the node group (NodeGroup) is extended.

[0063] In some embodiments, the node group corresponds to the activity at the code level, that is, different activities correspond to different node groups. The node group is composed of at least one node, and the interface of the node at the visual level corresponds.

[0064] In an optional implementation, when generating the operation trajectory model, the computer device determines node group information of the node group based on the node information, and generates the operation trajectory model based on the node group information, the edge information, and the path information.

[0065] In order to enable the node group information to characterize each node contained in the node group and to distinguish different node groups, the node group information at least includes a node list and a node group factor (NodeGroupFactor). The node list includes nodes belonging to the node group, and the node group factor is used for the node group characteristics of the node group.

[0066] In some embodiments, the computer device divides nodes with the same activity name into the same node group. Optionally, at the code level, the node group is uniquely indicated by the combination of the application package name + activity name, that is, the node group factor includes the application package name (packageName) and the activity name (activityName) of the application. Optionally, at the code level, the node group is uniquely indicated by the combination of the application package name + activity name + activity title, that is, the node group factor includes the application package name (packageName), the activity name (activityName) and the activity title (activityTitle) of the application.

[0067] 402. Traverse the edge information in the operation trajectory model by using a breadth-first traversal algorithm to determine the shortest path.

[0068] In the embodiment of the present application, the nodes in the operation trajectory model represent interfaces, and the directed edges between the nodes represent the jump relationship between the nodes based on the operation behavior. Figure 2 The content will not be repeated here.

[0069] Among them, the nodes in the operation trajectory model include node information. In some embodiments, the node information of the node includes at least the node identifier and the node factor of the node. The node identifier is a unique identifier assigned to the node (which has no practical meaning) and can be represented in the form of a string. The node identifiers of different nodes are different, and the same node has the same node identifier. The node factor (Node Factor) is composed of several fields that characterize interface features from different dimensions, among which the fields that characterize interface features can be called node sub-factors.

[0070] In some embodiments, the node factor may include at least one of windowName (window name), operableElements (operable elements), layoutVector (layout vector), screenVector (screen vector) and summaryFunction (function description).

[0071] Among them, the window name of the window to which the interface belongs can be extracted from the interface code. The interface usually includes a large number of operable elements, such as button controls, expandable list controls, text input box controls, upload controls, etc., and these operable elements can reflect the interface characteristics of the interface to a certain extent. In the case of version updates, the interface content of the same interface may change significantly, but the change in the overall layout of the interface is usually small. In addition to judging different interfaces (nodes) from the dimension of interface layout, different interfaces can also be judged from a more intuitive visual level. During the interface development stage, developers usually add interface function description information to the interface code. Even if the interface effect is iteratively updated later, the interface function description information usually does not change.

[0072] In some embodiments, the interface includes multiple elements, and the node includes node elements. The node element information of the node element includes at least the node element identifier and the node element factor of the node element. The node element identifier is a unique identifier assigned to the node element (with no actual meaning), which can be represented in the form of a string. The node element identifiers of different node elements are different. Optionally, the node element factor (NodeElement Factor) is composed of several fields that characterize the control features of the interface control from different dimensions, wherein the fields that characterize the control features can be referred to as node element sub-factors. Similar to the role of the node factor, the role of the node element factor is to retrieve and locate the interface controls of the features in the interface.

[0073] In some embodiments, the node element factors may include packageName (package name), className (class name), ResourceId (resource identifier), contentDescription (content description), Text (text), listOrder (list), parentChain (parent chain), and preNodeElementFactor&postNodeElementFactor (node ​​tuple factor of the left interface control and node tuple factor of the right interface control).

[0074] When reproducing an operation (flashback), it is necessary to know what operation was performed in the interface and the changes in the interface before and after the operation. Therefore, in order to ensure the accuracy of the reproduction of the operation, the edge information corresponding to the operation behavior at least has the ability to indicate the operation behavior and the interface before and after the operation is performed.

[0075] In some embodiments, the edge information includes at least: behavior information of the operation behavior, the starting node identifier (fromNodeID) of the starting interface corresponding to the starting node before the operation behavior, the ending node identifier (toNodeID) of the ending interface corresponding to the ending node after the operation behavior, and the edge identifier (similar to the unique identifier of the node identifier and the node element identifier). Accordingly, the edge information includes information used to characterize the operation behavior for implementing the jump between nodes and the node jump situation before and after the operation behavior.

[0076] It should be noted that behavior information, as an important part of side information, needs to be able to clearly indicate the operation behavior.

[0077] In some embodiments, the behavior information includes the behavior type (ActionType) of the operation behavior. The common behavior class (CommonAction) includes all the operation behaviors of the user and the system, so as to ensure the accuracy of the recorded operation behaviors. Moreover, the behavior type of each operation behavior belongs to the behavior type class.

[0078] For example, taking the common operations in mobile phones as an example, the behavior type class may include at least one of the following behavior types: GoHome (return to the home page), RecentApps (recent applications), GoBack (return), StartApp (start application), Click (click), DoubleClick (double-click), LongClick (long press), InputText (input text), Scroll (scroll), Concurrent (collaborative operation, that is, operation behaviors that occur at the same time), Continuous (continuous operation, referring to operation behaviors performed in sequence), Stop (terminate), Wait (wait, a scenario where the interface changes automatically over time), and Unknown (unknown behavior, as a fallback behavior).

[0079] In some embodiments, when determining the behavior type of the operation behavior as the behavior information, if the operation behavior is related to an interface control in the interface, the computer device may also determine the node element identifier of the node element corresponding to the interface control as the behavior information.

[0080] For example, when the operation behavior is a click operation, the behavior information corresponding to the operation behavior includes the operation type "Click" and the node element identifier of the node element corresponding to the clicked button; for another example, when the operation behavior is a text input operation, the behavior information corresponding to the operation behavior includes the operation type "InputText" and the node element identifier of the node element corresponding to the text input box where the text is entered.

[0081] The shortest path is the path starting from the first interface, passing through the least number of interfaces, and jumping to the second interface. The path includes multiple nodes and directed edges.

[0082] For example, Figure 5 is a schematic diagram of determining the shortest path according to an embodiment of the present application, see Figure 5 As shown in (a) in the figure, part of the edge information in the operation trajectory model is exemplarily shown. For example, edge 1 is a jump from interface A to interface B, and edge 10 is a jump from interface E to interface F. If the first page is page A and the second page is page G, then the shortest path is the path formed by the nodes connected by edge 2, edge 4, and edge 8.

[0083] 403. Determine an interface jump trajectory event according to the nodes and edges included in the shortest path.

[0084] In an embodiment of the present application, the interface jump trajectory event includes the shortest path from the first interface to the second interface, multiple operation behaviors in the shortest path, and the timing between the multiple operation behaviors.

[0085] In some embodiments, the path information of the operation path includes at least a path identifier and a behavior edge list.

[0086] Among them, similar to the node identifier, node element identifier and behavior edge identifier, the path identifier is a unique identifier assigned to the operation path (with no actual meaning), which can be expressed in the form of a string, where the path identifiers of different operation paths are different. The behavior edge list contains a series of behavior edges corresponding to the operation behaviors, and the behavior edges are arranged in the time sequence of the operation behaviors. For example, the behavior edge at the head of the list in the behavior edge list corresponds to the starting operation behavior, and the behavior edge at the end of the list corresponds to the ending operation behavior.

[0087] In some embodiments, the behavior edge list includes the behavior edge identifier in the edge information corresponding to the behavior edge.

[0088] Since the edge information of the behavior edge can indicate the nodes (interfaces) involved before and after the operation behavior and the behavior information of the executed operation behavior, the edge information of the behavior edge arranged in time sequence in the behavior edge list can accurately describe the execution order of a series of operation behaviors and what kind of operation behaviors are executed in the interface.

[0089] 404. Execute interface jump track event.

[0090] In an embodiment of the present application, the computer device can execute the interface jump trajectory event determined by the above steps, and execute the operation behaviors in sequence from the first interface to jump the interface. For the convenience of description, the interface before the jump is called the previous interface, and the interface after the jump is called the next interface. For any interface jump event, if the interface jumped from the previous interface is the corresponding next interface, it means that the execution is correct; if the interface jumped from the previous interface is not the corresponding next interface, but other interfaces, it means that an exception has occurred in the execution process. The computer device can repair the exception by dynamically calculating the path to finally jump to the second page.

[0091] In some embodiments, the computer device can intelligently execute the interface jump track event to repair the anomalies that occur during the execution process. Accordingly, in the process of executing the interface jump track event, in response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, the fifth interface to jump to is obtained. Among them, the fifth interface is also the currently displayed interface, and the third interface is the interface displayed before the jump. The third interface can be the first interface, or it can be an interface other than the first interface and the second interface, and the embodiment of the present application does not limit this.

[0092] After acquiring the fifth interface, the computer device can dynamically select a processing method by determining the relationship between the fifth interface and the fourth interface.

[0093] Correspondingly, if the shortest path passes through the fifth interface and the timing of the fifth interface is later than that of the fourth interface, it means that when executing the interface jump trajectory event, the fourth interface is skipped and the subsequent interface is reached directly without affecting the arrival of the second interface. Therefore, the computer device can continue to execute the interface jump trajectory event with the fifth interface as the starting point.

[0094] If the shortest path passes through the fifth interface, and the timing of the fifth interface is earlier than that of the fourth interface, it means that when executing the interface jump track event, it returns to the interface that has been passed. At this time, the interface that has been passed forms a closed loop and cannot jump to the subsequent interface, and it is also impossible to reach the second interface. Therefore, the computer device can determine that the shortest path is unreachable, regenerate an interface jump track event containing other paths, and then execute the regenerated interface jump track event. Among them, the target interface of the regenerated interface jump track event is still the second interface, and the starting interface is the currently displayed interface, that is, the fifth interface.

[0095] If the shortest path does not pass through the fifth interface, it means that when executing the interface jump trajectory event, it has jumped out of the shortest path and entered other paths, and it may not be possible to reach the second interface. Therefore, the computer device can regenerate an interface jump trajectory event containing the shortest path from the fifth interface to the second interface, and then execute the interface jump trajectory event. Correspondingly, if the shortest path does not pass through the fifth interface, the computer device can determine the shortest path from the fifth interface to the second interface. Then, the computer device can regenerate the interface jump trajectory event based on the nodes and edges contained in the shortest path. Finally, the computer device executes the regenerated interface jump trajectory event.

[0096] For example, the interface jump sequence in the shortest path is interface A→interface B→interface C→interface D. If, when executing an interface jump track event, after performing an operation in interface A, it does not jump to interface B, but jumps to interface C, that is, skips interface B, it will not be considered as an event execution failure, and the subsequent operation will continue from interface C. If, when executing an interface jump track event, after performing an operation in interface A, it does not jump to interface B, but jumps to interface E, since interface E is not an interface in the shortest path, the computer device can dynamically calculate the path from interface E to interface D, and then regenerate the interface jump track event and execute it.

[0097] In some embodiments, when a computer device executes an interface jump track event, if the execution is correct, the next operation behavior can be quickly executed for fast playback; if the execution is incorrect and the expected page is not jumped to after a period of time, the execution is stopped to avoid erroneous operations.

[0098] Correspondingly, in the process of executing the interface jump track event, in response to jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, the computer device can immediately execute the second operation behavior on the fourth interface to achieve fast playback.

[0099] In response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, if the currently displayed interface is not the fourth interface after a preset time, the computer device stops executing the interface jump track event. Alternatively, if the currently displayed interface is not the interface passed by the target path after a preset time, that is, the target path is not returned in time, the computer device stops executing the interface jump track event.

[0100] In some embodiments, there are multiple paths for jumping from the first interface to the second interface. When the shortest path is unreachable, the computer device can determine other paths. Then, the computer device determines a new interface jump trajectory event based on the nodes and edges included in the other paths. Finally, the computer device executes the new interface jump trajectory event.

[0101] For example, when there are multiple paths between interface A and interface B, the computer device will start exploring from the shortest path. If the shortest path is unreachable, the computer device will return to interface A or return to the desktop and reselect other paths for exploration.

[0102] It should be noted that the multiple operation behaviors performed in the process of executing the interface jump trajectory event include triggering the controls on the interface. Accordingly, when the content of the interface changes, the computer device executes the corresponding operation behavior based on any of the following methods. Method 1: Determine the target control corresponding to the operation behavior according to the control identifier, and trigger the target control. Method 2: In the case of repeated control identifiers, determine the target control corresponding to the operation behavior based on the relative position of the content in the interface before and after the change, and trigger the target control. Method 3: Determine the target control corresponding to the operation behavior based on the node factor of the corresponding node of the interface, and trigger the target control. The node factor contains node sub-factors that characterize the interface features of the interface from different dimensions. Method 4: Based on the large language model, the one-shot strategy is used to predict the operation behavior after the content of the interface occurs, and the predicted behavior is executed.

[0103] For example, Figure 6 is a flow chart of determining a control on an interface according to an embodiment of the present application. Figure 6 As shown, the left side is the interface before the change, referred to as the old interface. The right side is the interface after the change, referred to as the new interface. First, the interface element identifier (the control identifier of the control in the interface) is used to determine whether it is the same control, so as to achieve accurate matching. If multiple controls are matched in the new interface through the interface element identifier, or no control is matched, the control can be matched in the new interface through lazy calculation (lazy count). If multiple controls are matched, they are filtered; if no match is found, the most suitable control can be dynamically found through the previous and next controls. Among them, lazy calculation includes Parent-Chain calculation, ListOrder calculation, and Left-Right calculation. The search methods include Parent-Chain, Left-Right, ListOrder, and Random. Finally, based on the large language model, reasonable operation behaviors can be dynamically inferred through the one-shot strategy to determine the corresponding control.

[0104] 405. The display jumps from the first interface to the second interface in sequence.

[0105] In an embodiment of the present application, during the process of executing an interface jump track event, the computer device can display the interfaces to be jumped to in sequence.

[0106] The interface jump scheme provided in the embodiment of the present application can determine the shortest operation path from the current interface to the target interface by specifying the target interface to jump to, based on the jump relationship between interfaces based on operation behaviors according to the operation trajectory model, and then the operation behaviors can be executed in sequence according to the shortest operation path, so as to realize the jump from the current interface to the target interface in sequence. The above method does not need to record the operation of jumping from the first interface to the second interface, nor does it need to be saved as a script. Instead, it finds the shortest path from the first interface to the second interface and multiple operation behaviors on the shortest path according to the operation trajectory model, and then realizes the interface jump by executing the multiple operation behaviors. It can be applied to scenarios such as automated testing, has high scalability, and because the above multiple operation behaviors are behaviors in the operation trajectory model, there is no risk of misoperation of the interface, and has high robustness.

[0107] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0108] Figure 7 is a structural diagram of an interface jump device provided in an embodiment of the present application. Figure 7 The device includes: an acquisition module 701, a determination module 702 and an execution module 703.

[0109] An acquisition module 701 is used to acquire a first interface and a second interface, where the first interface is the current interface and the second interface is the target interface to jump to;

[0110] A determination module 702 is used to determine an interface jump trajectory event from a first interface to a second interface according to an operation trajectory model, wherein nodes in the operation trajectory model represent interfaces, and directed edges between nodes represent jump relationships between nodes based on operation behaviors. The interface jump trajectory event includes the shortest path from the first interface to the second interface, multiple operation behaviors in the shortest path, and a time sequence between the multiple operation behaviors;

[0111] The execution module 703 is used to execute the interface jump track event, and the display jumps from the first interface to the second interface in sequence.

[0112] In some embodiments, the determination module 702 is used to traverse the edge information in the operation trajectory model through a breadth-first traversal algorithm to determine the shortest path, where the edge information includes information used to characterize the operation behavior for implementing the jump between nodes and the node jump situation before and after the operation behavior; based on the nodes and edges included in the shortest path, the interface jump trajectory event is determined.

[0113] In some embodiments, the execution module 703 is used to, in the process of executing an interface jump trajectory event, in response to not jumping to the corresponding fourth interface after executing a first operation behavior on a third interface, obtain the fifth interface to jump to; if the shortest path passes through the fifth interface and the timing of the fifth interface is later than the fourth interface, continue to execute the interface jump trajectory event with the fifth interface as the starting point; if the shortest path passes through the fifth interface and the timing of the fifth interface is earlier than the fourth interface, determine that the shortest path is unreachable, regenerate and execute the interface jump trajectory event containing other paths; if the shortest path does not pass through the fifth interface, regenerate and execute the interface jump trajectory event containing the shortest path from the fifth interface to the second interface.

[0114] In some embodiments, the execution module 703 is used to determine the shortest path from the fifth interface to the second interface if the shortest path does not pass through the fifth interface; regenerate the interface jump trajectory event based on the nodes and edges included in the shortest path; and execute the regenerated interface jump trajectory event.

[0115] In some embodiments, the execution module 703 is used to, during the execution of the interface jump track event, in response to jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, immediately execute the second operation behavior on the fourth interface; in response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, if after a preset time, the currently displayed interface is not the fourth interface, stop executing the interface jump track event.

[0116] In some embodiments, the execution module 703 is used to stop executing the interface jump track event in response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, if after a preset time, the currently displayed interface is not the interface passed by the target path.

[0117] In some embodiments, the determination module 702 is further used to determine other paths when the shortest path is unreachable; determine a new interface jump trajectory event according to the nodes and edges included in the other paths;

[0118] The execution module 703 is also used to execute a new interface jump track event.

[0119] In some embodiments, the plurality of operating behaviors include triggering controls on the interface;

[0120] In the process of executing the interface jump track event, when the content of the interface changes, the execution module 703 is used to execute the corresponding operation behavior based on any of the following methods:

[0121] Determine the target control corresponding to the operation behavior according to the control identifier, and trigger the target control;

[0122] In the case of duplicate control identifiers, the target control corresponding to the operation behavior is determined based on the relative position of the content in the interface before and after the change, and the target control is triggered;

[0123] Based on the node factors of the corresponding nodes of the interface, determine the target control corresponding to the operation behavior and trigger the target control. The node factors include node sub-factors that characterize the interface features of the interface from different dimensions.

[0124] Based on the large language model, a one-shot strategy is used to predict the operational behavior after the content of the interface occurs, and the predicted behavior is executed.

[0125] In some embodiments, the acquisition module 701 is used to parse the input data to obtain the second interface, and the input data is any one of text, voice or options.

[0126] The interface jump scheme provided in the embodiment of the present application can determine the shortest operation path from the current interface to the target interface by specifying the target interface to jump to, based on the jump relationship between interfaces based on operation behaviors according to the operation trajectory model, and then the operation behaviors can be executed in sequence according to the shortest operation path, so as to realize the jump from the current interface to the target interface in sequence. The above method does not need to record the operation of jumping from the first interface to the second interface, nor does it need to be saved as a script. Instead, it finds the shortest path from the first interface to the second interface and multiple operation behaviors on the shortest path according to the operation trajectory model, and then realizes the interface jump by executing the multiple operation behaviors. It can be applied to scenarios such as automated testing, has high scalability, and because the above multiple operation behaviors are behaviors in the operation trajectory model, there is no risk of misoperation of the interface, and has high robustness.

[0127] An embodiment of the present application provides a computer device, which includes a processor and a memory; the memory stores at least one program code, and the at least one program code is used to be executed by the processor to implement the interface jump method provided in the above-mentioned various method embodiments.

[0128] A computer device may be configured as a terminal, Figure 8 800 is a block diagram of a terminal provided in an embodiment of the present application. In some embodiments, the terminal 800 is a terminal such as a smart phone, a tablet computer, a wearable device, etc. that can access a wireless local area network as a wireless station. The terminal 800 in the present application includes at least one or more of the following components: a processor 810, a memory 820, and at least two wireless links 830.

[0129] In some embodiments, the processor 810 includes one or more processing cores. The processor 810 uses various interfaces and lines to connect various parts of the entire terminal 800, and executes various functions and processes data of the terminal 800 by running or executing program codes stored in the memory 820, and calling data stored in the memory 820. In some embodiments, the processor 810 is implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 810 can integrate one or more combinations of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the display screen; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 810, but may be implemented by a separate chip.

[0130] In some embodiments, the processor 810 is used to control the working conditions of at least two wireless links 830. Accordingly, the processor 810 is a processor integrated with a wireless fidelity (Wireless Fidelity, Wi-Fi) chip. The Wi-Fi chip is a chip with dual Wi-Fi processing capabilities. For example, the Wi-Fi chip is a dual-band dual concurrent (Dual Band Dual Concurrent, DBDC) chip, or a dual-band synchronous (Dual Band Simultaneous, DBS) chip, etc.

[0131] In some embodiments, the memory 820 includes a random access memory (RAM), and in some embodiments, the memory 820 includes a read-only memory (ROM). In some embodiments, the memory 820 includes a non-transitory computer-readable storage medium. The memory 820 can be used to store program codes. The memory 820 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data created according to the use of the terminal 800 (such as audio data, a phone book), etc.

[0132] In some embodiments, the memory 820 stores reception schemes for receiving beacon frames of different wireless links 830, as well as identifiers of access nodes connected to different wireless links 830, identifiers of wireless links 830, and the like.

[0133] The at least two wireless links 830 are used to connect different access points (APs) to receive downlink data sent by the APs, wherein the different access points are access points in the same router or access points in different routers.

[0134] In some embodiments, the terminal 800 also includes a display screen. The display screen is a display component for displaying a user interface. In some embodiments, the display screen is a display screen with a touch function, through which the user can use any suitable object such as a finger or a touch pen to perform touch operations on the display screen. In some embodiments, the display screen is usually arranged on the front panel of the terminal 800. In some embodiments, the display screen is designed to be a full screen, a curved screen, a special-shaped screen, a double-sided screen or a folding screen. In some embodiments, the display screen is also designed to be a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., which is not limited in this embodiment.

[0135] In addition, those skilled in the art will appreciate that the structure of the terminal 800 shown in the above drawings does not limit the terminal 800, and the terminal 800 includes more or fewer components than shown in the drawings, or combines certain components, or arranges components differently. For example, the terminal 800 also includes components such as a microphone, a speaker, an input unit, a sensor, an audio circuit, a module, a power supply, and a Bluetooth module, which will not be described in detail here.

[0136] The present application also provides a computer-readable storage medium, which stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the interface jump method shown in the above embodiments.

[0137] The present application also provides a computer program product, which stores at least one program code, and the at least one program code is used to be executed by a processor to implement the interface jump method shown in the above embodiments.

[0138] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0139] Those skilled in the art can understand that all or part of the steps in the interface jump method of the above embodiment can be completed by hardware, or by a program to instruct the relevant hardware to complete, and the above program can be stored in a computer-readable storage medium, and the above storage medium can be a read-only memory, a disk or an optical disk, etc. The above is only an optional embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An interface jump method, characterized in that: The method comprises: Obtain a first interface and a second interface, where the first interface is the current interface and the second interface is the target interface to jump to; Determine, according to the operation trajectory model, an interface jump trajectory event of jumping from the first interface to the second interface, wherein the nodes in the operation trajectory model represent interfaces, the directed edges between the nodes represent the jump relationship between the nodes based on the operation behavior, and the interface jump trajectory event includes the shortest path from the first interface to the second interface, multiple operation behaviors in the shortest path, and the time sequence between the multiple operation behaviors; The interface jump track event is executed to display a jump from the first interface to the second interface in sequence.

2. The method according to claim 1, characterized in that: The determining, according to the operation trajectory model, an interface jump trajectory event of jumping from the first interface to the second interface includes: Traversing the edge information in the operation trajectory model by a breadth-first traversal algorithm to determine the shortest path, wherein the edge information includes information for characterizing the operation behavior for implementing the jump between nodes and the node jump situation before and after the operation behavior; The interface jump trajectory event is determined according to the nodes and edges included in the shortest path.

3. The method according to claim 1, characterized in that The executing the interface jump track event includes: In the process of executing the interface jump track event, in response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, obtaining the fifth interface to jump to; If the shortest path passes through the fifth interface, and the timing of the fifth interface is later than that of the fourth interface, the interface jump track event continues to be executed starting from the fifth interface; If the shortest path passes through the fifth interface, and the timing of the fifth interface is earlier than that of the fourth interface, it is determined that the shortest path is unreachable, and an interface jump trajectory event containing other paths is regenerated and executed; If the shortest path does not pass through the fifth interface, an interface jump track event including the shortest path from the fifth interface to the second interface is regenerated and executed.

4. The method according to claim 3, characterized in that If the shortest path does not pass through the fifth interface, regenerating and executing an interface jump track event including the shortest path from the fifth interface to the second interface comprises: If the shortest path does not pass through the fifth interface, determine the shortest path from the fifth interface to the second interface; Regenerate an interface jump trajectory event according to the nodes and edges included in the shortest path; Executes the regenerated interface jump track event.

5. The method according to claim 1, characterized in that The executing the interface jump track event includes: During the process of executing the interface jump track event, in response to jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, immediately executing the second operation behavior on the fourth interface; In response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, if the currently displayed interface is not the fourth interface after a preset time period, stop executing the interface jump track event.

6. The method according to claim 1, characterized in that The executing the interface jump track event includes: In response to not jumping to the corresponding fourth interface after executing the first operation behavior on the third interface, if after a preset time, the currently displayed interface is not the interface passed by the target path, stop executing the interface jump track event.

7. The method according to claim 1, characterized in that There are multiple paths to jump from the first interface to the second interface; The method further comprises: If the shortest path is unreachable, determine other paths; Determine a new interface jump trajectory event according to the nodes and edges included in the other paths; Execute the new interface jump track event.

8. The method according to any one of claims 1 to 7, characterized in that: The multiple operation behaviors include triggering controls on the interface; In the process of executing the interface jump track event, when the content of the interface changes, the corresponding operation behavior is performed based on any of the following methods: Determine the target control corresponding to the operation behavior according to the control identifier, and trigger the target control; In the case where the control identifier is repeated, determining a target control corresponding to the operation behavior based on the relative position of the content in the interface before and after the change, and triggering the target control; Based on the node factor of the node corresponding to the interface, determine the target control corresponding to the operation behavior, and trigger the target control, wherein the node factor includes node sub-factors that characterize the interface features of the interface from different dimensions; Based on the large language model, the operation behavior after the content of the interface occurs is predicted through a one-shot strategy, and the predicted behavior is executed.

9. The method according to any one of claims 1 to 7, characterized in that: The step of obtaining the second interface includes: The input data is parsed to obtain the second interface, where the input data is any one of text, voice or options.

10. An interface jump device, characterized in that: The device comprises: An acquisition module, used to acquire a first interface and a second interface, wherein the first interface is a current interface and the second interface is a target interface to jump to; a determination module, configured to determine, according to an operation trajectory model, an interface jump trajectory event of jumping from the first interface to the second interface, wherein nodes in the operation trajectory model represent interfaces, directed edges between nodes represent jump relationships between nodes based on operation behaviors, and the interface jump trajectory event includes a shortest path from the first interface to the second interface, multiple operation behaviors in the shortest path, and a time sequence between the multiple operation behaviors; An execution module is used to execute the interface jump track event, and display the jump from the first interface to the second interface in sequence.

11. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the interface jump method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the interface jump method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the interface jump method as described in any one of claims 1 to 9.

Images