Interface retrieval method and apparatus, terminal, storage medium and product

Abstract

Embodiments of the present application belong to the technical field of artificial intelligence, and disclose an interface retrieval method and apparatus, a terminal, a storage medium, and a product. The method comprises: in response to a retrieval instruction, performing feature extraction on an interface retrieval condition indicated by the retrieval instruction, to obtain an interface retrieval feature; on the basis of the interface retrieval feature, a feature library corresponding to a historical operation trajectory, the interface retrieval condition, and the historical operation trajectory, determining first target function information, the historical operation trajectory being used for recording function information of at least one interface in a historical operation process, the feature library being used for storing a feature corresponding to the function information of the at least one interface, and the function information being used for indicating a function of the interface; and determining a first target interface to which the first target function information belongs, the first target interface being an interface retrieved on the basis of the retrieval instruction.

Description

Interface retrieval methods, devices, terminals, storage media and products

[0001] This application claims priority to Chinese Patent Application No. 202411824971.8, filed on December 11, 2024, entitled "Interface Retrieval Method, Apparatus, Terminal, Storage Medium and Product", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence technology, and in particular to an interface retrieval method, device, terminal, storage medium and product. Background Technology

[0003] Global search, a commonly used user interface search method, is generally divided into in-application search and system-level search. In-application search is implemented by different applications themselves, with each application implementing its own search capabilities. System-level search is implemented by system applications and provided by the terminal manufacturer. Summary of the Invention

[0004] This application provides an interface retrieval method, apparatus, terminal, storage medium, and product. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide an interface retrieval method, the method comprising:

[0006] In response to a search command, feature extraction is performed on the interface search conditions indicated by the search command to obtain interface search features; based on the interface search features, a feature library corresponding to historical operation trajectories, the interface search conditions, and the historical operation trajectories, first target function information is determined, wherein the historical operation trajectories are used to record the function information of at least one interface during historical operations, the feature library is used to store the features corresponding to the function information of the at least one interface, and the function information is used to indicate the functions of the interface; a first target interface to which the first target function information belongs is determined, wherein the first target interface is the interface retrieved based on the search command.

[0007] On the other hand, embodiments of this application provide an interface retrieval device, the device comprising: a first processing module, configured to extract features from interface retrieval conditions indicated by the retrieval command in response to a retrieval command, thereby obtaining interface retrieval features; a first determining module, configured to determine first target function information based on the interface retrieval features, a feature library corresponding to historical operation trajectories, the interface retrieval conditions, and the historical operation trajectory, wherein the historical operation trajectory is used to record function information of at least one interface during historical operations, the feature library is used to store features corresponding to the function information of the at least one interface, and the function information is used to indicate the functions possessed by the interface; and a second determining module, configured to determine a first target interface to which the first target function information belongs, wherein the first target interface is an interface retrieved based on the retrieval command.

[0008] On the other hand, embodiments of this application provide a terminal, the terminal including a processor and a memory, the memory storing at least one computer instruction, the at least one computer instruction being loaded and executed by the processor to implement the interface retrieval method as described above. On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one computer instruction, the at least one computer instruction being executed by a processor to implement the interface retrieval method as described above. On the other hand, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the interface retrieval method as described above. Attached Figure Description

[0009] Figure 1 is a schematic diagram illustrating the historical operation trajectory corresponding to the operation process in an exemplary embodiment of this application;

[0010] Figure 2 is a flowchart illustrating an exemplary embodiment of the interface retrieval method of this application;

[0011] Figure 3 is a schematic diagram illustrating an exemplary embodiment of the interface retrieval method of this application;

[0012] Figure 4 is a schematic diagram illustrating unit functional information related to an application, as shown in an exemplary embodiment of this application;

[0013] Figure 5 is a flowchart illustrating an exemplary embodiment of the interface retrieval method of this application;

[0014] Figure 6 is a schematic diagram illustrating installation boot information in an exemplary embodiment of this application;

[0015] Figure 7 is a flowchart illustrating an exemplary embodiment of the interface retrieval method of this application;

[0016] Figure 8 is an interface diagram illustrating the process involved in the interface retrieval method according to an exemplary embodiment of this application;

[0017] Figure 9 is a schematic diagram illustrating the first target functional information in an exemplary embodiment of this application;

[0018] Figure 10 is a schematic diagram illustrating first target path information in an exemplary embodiment of this application;

[0019] Figure 11 is an interface diagram illustrating the interface involving the first target path information in an exemplary embodiment of this application;

[0020] Figure 12 is a schematic diagram illustrating target tutorial information in an exemplary embodiment of this application;

[0021] Figure 13 is a structural block diagram of an overall solution illustrating an exemplary embodiment of this application;

[0022] Figure 14 is a structural block diagram of an exemplary embodiment of the present application illustrating an interface retrieval device;

[0023] Figure 15 is a structural block diagram of a terminal shown in an exemplary embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0025] In this document, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, historical operation trajectories and voice commands involved in this application were obtained with full authorization.

[0026] The following technical solutions are provided in the embodiments of this application:

[0027] An interface retrieval method, the method comprising:

[0028] In response to a search command, feature extraction is performed on the interface search conditions indicated by the search command to obtain interface search features; based on the interface search features, a feature library corresponding to historical operation trajectories, the interface search conditions, and the historical operation trajectories, first target function information is determined, wherein the historical operation trajectories are used to record the function information of at least one interface during historical operations, the feature library is used to store the features corresponding to the function information of the at least one interface, and the function information is used to indicate the functions of the interface; a first target interface to which the first target function information belongs is determined, wherein the first target interface is the interface retrieved based on the search command.

[0029] In some embodiments, determining the first target function information based on the interface retrieval features, the feature library corresponding to the historical operation trajectory, the interface retrieval conditions, and the historical operation trajectory includes:

[0030] Based on the interface retrieval features and the feature library, at least one first functional information is determined, wherein the first functional information is information of matching functions retrieved based on the interface retrieval features and the feature library; based on the interface retrieval conditions and the functional information of the at least one interface, at least one second functional information is determined, wherein the second functional information is information of matching functions retrieved based on the interface retrieval conditions and the functional information of the at least one interface; based on the at least one first functional information and the at least one second functional information, the first target functional information is determined.

[0031] In some embodiments, determining at least one first functional information based on the interface retrieval features and the feature library includes:

[0032] Based on the interface retrieval features, features matching the interface retrieval features are searched from the first feature library. The first feature library is used to store features corresponding to the functional information of at least one interface in the first historical operation trajectory. The first historical operation trajectory is the historical operation trajectory generated by the application installed on the terminal. If at least one first feature is found, at least one first functional information corresponding to the at least one first feature is determined.

[0033] In some embodiments, determining at least one first functional information based on the interface retrieval features and the feature library includes:

[0034] A query request is sent to the server, the query request carrying the interface retrieval features, so that the server searches for at least one first feature matching the interface retrieval features from a second feature library based on the interface retrieval features, and sends at least one first function information corresponding to the at least one first feature to the terminal. The second feature library includes features corresponding to the function information of at least one interface in a second historical operation trajectory, the second historical operation trajectory being the historical operation trajectory generated by applications not installed on the terminal; and the terminal receives at least one first function information sent by the server.

[0035] In some embodiments, the method further includes:

[0036] A first target application to which the at least one first functional information belongs is determined; if the first target application is an application not installed on the terminal, installation guidance information for the first target application is generated based on the first target application, and the installation guidance information for the first target application is used to instruct the process of installing the first target application; the installation guidance information for the first target application is displayed on the current interface.

[0037] In some embodiments, the method further includes:

[0038] Determine the functional information of at least one interface in the first historical operation trajectory. The functional information includes at least one of unit function information and interface function description information. The unit function information is used to indicate the smallest unit function of the interface, and the interface function description information is used to indicate the interface function of the interface. Perform feature extraction on the functional information of the at least one interface to obtain the first feature library.

[0039] In some embodiments, the method further includes:

[0040] The access permissions of the first feature library are set to be accessible, so that other services can access the first feature library to assist in the execution of the other service's tasks. The other service is a service in the terminal other than executing the search instruction, or the other service is a service in a third-party system.

[0041] In some embodiments, determining the first target function information based on the at least one first function information and the at least one second function information includes:

[0042] The at least one first functional information and the at least one second functional information are merged and sorted; based on the sorted functional information sequence, the first target functional information is determined.

[0043] In some embodiments, the at least one first functional information is sorted sequentially based on its matching degree with interface retrieval features, and the at least one second functional information is sorted sequentially based on its matching degree with the interface retrieval conditions; the merging and sorting of the at least one first functional information and the at least one second functional information includes:

[0044] For any one of the at least one first functional information and the at least one second functional information, the weight of the functional information is determined based on the order of the functional information, and the weight of the functional information is negatively correlated with the order; the at least one first functional information and the at least one second functional information are sorted based on the weight of the at least one first functional information and the weight of the at least one second functional information.

[0045] In some embodiments, determining at least one second functional information based on the interface retrieval conditions and the functional information of the at least one interface includes:

[0046] The text information corresponding to the interface search conditions is segmented to obtain multiple keywords; the matching degree between the multiple keywords and the functional information of the at least one interface is determined; based on the matching degree between the multiple keywords and the functional information of the at least one interface, at least one second functional information is determined from the functional information of the at least one interface, wherein the matching degree between the at least one functional information and the multiple keywords satisfies the matching degree condition.

[0047] In some embodiments, the method further includes:

[0048] Based on the first target interface, first target path information is determined from the historical operation trajectory. The historical operation trajectory is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation and the path information of the operation path corresponding to the interface during the historical operation. The first target operation path to which the first target path information belongs is the operation path to the first target interface. Based on the first target edge information included in the first target path information, the first target operation behavior indicated by the first target edge information is executed. The first target edge information includes information used to characterize the first target operation behavior and the interface jump situation before and after the first target operation behavior.

[0049] In some embodiments, the method further includes:

[0050] Based on the first target edge information included in the first target path information, target tutorial information corresponding to the first target edge information is generated. The target tutorial information is used to guide the completion of the first target operation behavior indicated by the first target edge information; the target tutorial information is displayed on the current interface.

[0051] In some embodiments, generating target tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information includes:

[0052] Based on the first target edge information included in the first target path information, first tutorial information corresponding to the first target edge information is generated. The first tutorial information is used to guide the execution of the first target operation behavior indicated by the first target edge information in text. Based on the first tutorial information, the target tutorial information is determined.

[0053] In some embodiments, determining the target tutorial information based on the first tutorial information includes:

[0054] The first tutorial information is determined as the target tutorial information; or...

[0055] Based on at least one of the functional information in the historical operation trajectory, the target edge information, and the first tutorial information, second tutorial information is obtained, and the second tutorial information is determined as the target tutorial information. The target tutorial information is a guide to perform the first target operation behavior indicated by the first target edge using graphic information.

[0056] In some embodiments, determining the target tutorial information based on the first tutorial information includes: obtaining second tutorial information by taking at least one of the function information in the historical operation trajectory, the first target edge information, and the first tutorial information; converting the second tutorial information into tutorial information in video format to obtain the target tutorial information, wherein the target tutorial information is a video guide to perform the first target operation behavior indicated by the first target edge.

[0057] In some embodiments, the method further includes:

[0058] The system receives a voice command to request the display of a second target interface. Based on the voice command, it determines second target path information from the historical operation trajectory. The historical operation trajectory also records edge information of the corresponding action edge within the interface during historical operations and path information of the corresponding operation path within the interface during historical operations. The second target operation path to which the second target path information belongs is the operation path leading to the second target interface. Based on the second target edge information included in the second target path information, the system executes the second target operation action indicated by the second target edge information. The second target edge information includes information characterizing the second target operation action and the interface jump situation before and after the second target operation action.

[0059] In this embodiment, a recall retrieval is performed based on interface retrieval features and feature library, and another recall retrieval is performed based on interface retrieval conditions and interface functional information. The first target interface that the user wants to retrieve is determined by comprehensively considering the results of the two recall retrievals, thereby realizing two-way recall retrieval to retrieve the first target interface and improving the accuracy of the retrieved first target interface.

[0060] Referring to Figure 1, in this embodiment, a historical operation trajectory (Trajectory Structure) is used to record the operation process. The historical operation trajectory 13 consists of nodes 131, behavior edges 132, and operation paths 133. Nodes 131 correspond to the interfaces 11 traversed during the operation; behavior edges 132 correspond to the operation behaviors within the interfaces, and are edges 12 used to connect different nodes (where interface switching occurs after the operation) or the same node (where interface switching does not occur after the operation); the operation path 133 corresponds to the sequence of operation behaviors. Furthermore, to ensure the accuracy of the operation process recording, the node information 141 of node 131 characterizes the interface features from a functional perspective, ensuring the accuracy of subsequent interface retrieval and location; for example, node information 141 can be functional information, which indicates the functions the interface possesses. Correspondingly, the historical operation trajectory 13 is used to record the functional information of the interfaces during historical operations. The edge information 142 of behavior edge 132 indicates the operation behavior and the node jump situation before and after the operation behavior, ensuring the accurate restoration of subsequent operations in the interface; the path information 143 of operation path 133 represents the execution sequence between different operation behaviors, ensuring that complex operations are executed in sequence.

[0061] Compared to screenshots and videos, historical operation trajectories require less data and can be generated automatically without manual triggering by the user. Furthermore, they can be automatically traced back and reproduced based on node information, edge information, and path information, improving the efficiency of operation tracing and reproduction.

[0062] Functional Information: The interface is a visual user interface, such as a desktop, application interface, system interface, etc. In this embodiment, the interface is not limited to the visible screen area, but includes the complete interface both within and outside the visible screen area. Accordingly, even if the displayed content within the visible screen area changes significantly before and after scrolling, the interface before and after scrolling will not be considered different interfaces.

[0063] In this application's embodiments, "different interfaces" refers to interfaces with significant differences in layout, content, and style. Typically, when an interface switch occurs, the interfaces before and after the switch are considered different interfaces. For example, dynamic changes in text or images within the interface, or changes in list elements within the interface, will not be considered different interfaces. However, when clicking on a control within the interface triggers a page jump, the interfaces before and after the jump are considered different interfaces.

[0064] The functional information includes at least one of unit function information and interface function description information. Unit function information indicates the smallest unit function of the interface, while interface function description information indicates the interface functions. In one possible implementation, at least one of the unit function information and interface function description information is determined using a large language model.

[0065] Edge Information: Edge information contains information representing the operation behavior and the interface transition before and after the operation behavior. The behavior edge corresponding to the operation behavior is a directed edge, and the direction of the behavior edge is the interface transition direction, that is, the behavior edge points from the interface before the transition to the interface after the transition. Optionally, the operation behavior refers to a user manual operation, such as triggering a control within the interface; or, the operation behavior refers to a system behavior, such as an automatic interface transition after a preset time. In some embodiments, multiple behavior types are pre-set, and the edge information of the behavior edge contains the behavior type to which the operation behavior belongs. Since it is necessary to know what operation was performed within the interface and the changes in the interface before and after the operation behavior when reproducing the operation behavior, in order to ensure the accuracy of the operation behavior reproduction, the edge information corresponding to the operation behavior must at least have the ability to indicate the operation behavior and the interface before and after the operation behavior.

[0066] In some embodiments, the edge information includes: behavior information of the operation, the starting node identifier (fromNodeID) of the starting node corresponding to the starting interface before the operation, the ending node identifier (toNodeID) of the ending interface corresponding to the ending node after the operation, and an edge identifier (a unique identifier similar to a node identifier and a node element identifier). Accordingly, when determining the edge information of the behavior edge corresponding to the operation within the interface during the operation process, the behavior information of the operation within the interface during the operation process is determined, and the starting node identifier of the starting node corresponding to the starting interface before the operation and the ending node identifier of the ending interface corresponding to the ending node after the operation are determined, thereby determining the behavior information, the starting node identifier, and the ending node identifier as the edge information of the behavior edge corresponding to the operation.

[0067] Specifically, when the interface remains unchanged before and after an operation, the start and end node identifiers in the edge information corresponding to that operation are the same; when the interface changes before and after an operation, the start and end node identifiers in the edge information corresponding to that operation are different. As a crucial component of the edge information, the behavior information needs to clearly indicate the operation. In one possible implementation, this behavior information includes the action type (ActionType) of the operation.

[0068] In some embodiments, a CommonAction class is defined, which includes all user and system operations to ensure the accuracy of recorded operations. Furthermore, the behavior type of each operation belongs to a behavior type class. Optionally, this behavior type class is an enumeration class. In an illustrative example, taking common operations on a mobile phone as an example, this behavior type class may include at least one of the following behavior types: GoHome (return to home page), RecentApps, GoBack, StartApp, Click, DoubleClick, LongClick, InputText, Scroll, Concurrent (cooperative operations, i.e., operations that occur simultaneously), Continuous (sequential operations, operations executed sequentially), Stop, Wait (waiting, in scenarios where the interface changes automatically over time), and Unknown (unknown behavior, as a fallback behavior).

[0069] Since some operations are directed at UI controls within the interface, in order to ensure that the operation can be determined based on the behavior information, in some embodiments, when determining the behavior type of the operation as behavior information, if the operation is related to a UI control within the interface, the node element identifier of the corresponding node element of the UI control is also determined as behavior information.

[0070] For example, when the operation is a click operation, the corresponding behavior information includes the operation type "Click" and the node element identifier of the node element corresponding to the clicked button; as another example, when the operation is a text input operation, the corresponding behavior information includes the operation type "InputText" and the node element identifier of the text input box to which the text is entered.

[0071] In addition, since the operation may involve a series of consecutive actions, to facilitate the backtracking of preceding actions, in one possible implementation, when an action has a preceding action, the operation information of the preceding action is determined as the edge information of the action corresponding to the current action; that is, the edge information also includes the preceding action information (preAction). For example, for the action of clicking the login button, the preceding actions may include entering the account in the first text box and entering the password in the second text box. Optionally, the preceding action information can be represented in the form of a list, where each item corresponds to a preceding action, and the preceding actions can be represented by the jump between nodes, such as starting node identifier → ending node identifier (the starting node identifier and the ending node identifier can be the same).

[0072] In this embodiment, the starting node identifier of the starting node before the operation, the ending node identifier of the ending node after the operation, and the behavior information of the operation are used as edge information, which improves the accuracy of operation behavior recording. Furthermore, the behavior information includes the behavior type of the operation and contains the node element identifier of the target interface control, ensuring the accuracy of subsequent restoration of the operation of the interface control.

[0073] Path Information: Since historical operations may require a series of actions and transitions between multiple interfaces, the action edges corresponding to the actions need to be sorted according to their execution sequence to obtain the operation path. This operation path is a directed path composed of multiple nodes and the action edges between them, with different nodes corresponding to different interfaces. Accordingly, the path information consists of edge information, indicating the sequential relationship between the action edges.

[0074] In some embodiments, after determining the functional information, edge information, and path information, a historical operation trajectory is generated based on the functional information, edge information, and path information. The historical operation trajectory corresponding to the historical operation process is generated based on the functional information of the interface involved in the historical operation process, the edge information of the involved behavior edges, and the path information of the involved operation paths. In some embodiments, the generated historical operation trajectory is stored so that interface retrieval, location, and operation reproduction can be performed subsequently based on the historical operation trajectory.

[0075] In some embodiments, the path information of an operation path includes at least a path identifier and a list of behavior edges. In one possible implementation, when determining the path information of an operation path corresponding to an operation process based on the edge information, the path identifier of the operation path corresponding to the operation process is determined, and the list of behavior edges is determined based on the edge information, thereby identifying the path identifier and the list of behavior edges as the path information of the operation path. Similar to node identifiers, node element identifiers, and behavior edge identifiers, the path identifier is a unique identifier assigned to an operation path (without any practical meaning), and can be represented as a string, where different operation paths have different path identifiers. The behavior edge list contains a series of behavior edges corresponding to operation behaviors, and the behavior edges are arranged according to the temporal order of the operation behaviors. For example, the behavior edge at the beginning of the list corresponds to the starting operation behavior, while the behavior edge at the end of the list corresponds to the ending operation behavior.

[0076] In some embodiments, the behavior edge list includes behavior edge identifiers from the edge information corresponding to the behavior edges. Since the edge information of a behavior edge can indicate the nodes (interfaces) involved before and after an operation, as well as the behavior information of the performed operation, the sequential execution order of a series of operations and the specific operation performed on the interface can be accurately described based on the edge information of the behavior edges arranged chronologically in the behavior edge list. In some embodiments, to facilitate subsequent operation path retrieval based on the functions implemented through a series of operations, the path information also includes a path function. Accordingly, the path function of the operation path is determined, and the path function is identified as the path information of the operation path.

[0077] Regarding the method for determining the path function: In one possible implementation, the path function can be generated based on the interface elements (such as interface text, the package name of the application to which the interface belongs) and the text content corresponding to the interface controls triggered during the operation. For example, if the application package name of the interface involved in the operation is (com.android.setting), which is the system settings application, and the text content corresponding to the interface controls triggered during the operation is "display and brightness" and "screen color mode", the generated path function would be "set screen color mode in system application".

[0078] In another possible implementation, the interface images involved in the operation can be input into a large language model, and the path function can be generated through the large language model. This embodiment does not limit the specific method of generating the path function. In this embodiment, a list of behavior edges is generated based on the behavior edges corresponding to the operation behaviors involved in the operation, and the list of behavior edges is used as the path information of the operation path, which ensures the accuracy of subsequent operation reconstruction based on the path information; furthermore, including the path function of the operation path as part of the path information is beneficial for subsequent operation path retrieval based on the functions to be implemented in the operation, thus expanding the application scenarios of historical operation trajectories.

[0079] In this embodiment, feature extraction is performed on the functional information of at least one interface in the historical operation trajectory to obtain a feature library. In this embodiment, when searching for an interface, one recall retrieval is performed based on the feature library, and another recall retrieval is performed based on the functional information of at least one interface, thereby achieving retrieval of the first target interface based on two recall retrievals.

[0080] In one possible implementation, the historical operation trajectory is further used to record the edge information of the corresponding action edges within the interface during the historical operation process and the path information of the corresponding operation paths within the interface during the historical operation process. The edge information is used to represent the operation action and the interface jump situation before and after the operation action, and the path information is used to represent the temporal sequence between operation actions. After retrieving the first target interface, the first target path information can be determined from the historical operation trajectory. The first target operation path to which the first target path information belongs is the operation path to reach the first target interface. Based on the first target edge information included in the first target operation path, which includes information representing the first target operation action and the interface jump situation before and after the first target operation action, the first target operation action indicated by the first target edge information is executed, thereby jumping to the first target interface, realizing the automatic execution and jumping of the first target interface.

[0081] The interface retrieval method provided in this application embodiment can be executed by a terminal with retrieval function. The terminal can be a smartphone, tablet computer, smart wearable device, personal computer, etc. Please refer to Figure 2, which shows a flowchart of an exemplary embodiment of the interface retrieval method provided in this application. This embodiment uses the method applied to a terminal as an example for illustration, and the method may include the following steps:

[0082] Step 201: In response to the search command, extract features from the interface search conditions indicated by the search command to obtain interface search features.

[0083] The interface retrieval feature can be any feature that can characterize the interface retrieval conditions; for example, the interface retrieval feature is an interface retrieval vector. Then this step can be: in response to the retrieval command, vectorize the interface retrieval conditions indicated by the retrieval command to obtain the interface retrieval vector. The interface retrieval vector is used to indicate key information in the interface retrieval conditions; for example, if the interface retrieval condition is "I want to open and view chat history containing milk tea," then the interface retrieval vector can be "milk tea" and "chat history"; as another example, if the interface retrieval condition is "I want to buy a dress," then the interface retrieval vector is "buy" and "dress"; as another example, if the interface retrieval condition is "I want to view the 5 o'clock alarm clock," then the interface retrieval vector is "5 o'clock" and "alarm clock"; as another example, if the interface retrieval condition is "I want to view the 500 yuan payment record," then the interface retrieval vector is "500 yuan" and "payment record," etc. The interface retrieval conditions indicated by the retrieval command can be voice or text. For example, the interface retrieval condition is text describing the interface function; for example, the interface retrieval condition is the text "I want to turn on care mode." For example, the interface search criteria could be the voice describing the interface's functions; for instance, the interface search criteria could be the voice saying "I want to turn on care mode".

[0084] Step 202: Based on the interface retrieval features, the feature library corresponding to the historical operation trajectory, the interface retrieval conditions, and the historical operation trajectory, determine the first target function information. The historical operation trajectory is used to record the function information of at least one interface during the historical operation process. The feature library is used to store the features corresponding to the function information of at least one interface. The function information is used to indicate the functions that the interface has.

[0085] This application embodiment employs a multi-path retrieval method for interface retrieval. One retrieval path can be based on interface retrieval features and a feature library, while the other retrieval path is based on interface retrieval conditions and functional information of at least one interface. Accordingly, this step can be implemented through the following steps (1) to (3):

[0086] (1) Based on the interface retrieval features and the feature library, at least one first functional information is determined. The first functional information is the information of the matching function retrieved based on the interface retrieval features and the feature library. The feature library is used to store the features corresponding to the functional information of at least one interface in the historical operation trajectory. The historical operation trajectory is used to record the functional information of the interface during the historical operation process. The functional information is used to indicate the functions that the interface has.

[0087] The features corresponding to the functional information of the interface are functional vectors obtained by vectorizing the functional information of the interface. The functional vectors of the interface are used to indicate the key information corresponding to the interface functions; for example, if the interface is a payment interface, the functional vector of the interface can be the amount information included in the payment interface; if the interface is an application interface, the functional vector of the interface can be the application information of the application interface; if the interface is an alarm clock interface, the functional vector of the interface can be the time information of the alarm clock interface; if the interface is a food delivery interface, the functional vector of the interface can be the order status information of the food delivery interface; if the interface is a chat interface of a social application, the functional vector of the interface can be the chat history of the chat interface, etc.

[0088] At least one feature corresponding to the first functional information and the interface retrieval feature have a matching degree that satisfies a first matching degree condition. The first matching degree condition can be selecting a first preset number of features with the highest matching degree or selecting features with a matching degree exceeding the first preset matching degree. In one possible implementation, the terminal determines the matching degree between the interface retrieval feature and each feature in the feature library; based on the matching degree between the interface retrieval feature and each feature, it determines at least one feature from the feature library whose matching degree satisfies the first matching degree condition, determines the functional information corresponding to the at least one feature, and obtains at least one first functional information. For example, based on the matching degree between the interface retrieval feature and each feature, the terminal determines the four features with the highest matching degree from the feature library, and determines the first functional information corresponding to these four features.

[0089] Functional information includes at least one of unit function information and interface function description information. Unit function information indicates the smallest unit function of the interface. The smallest unit function refers to an independent, basic, and composable technical or functional module in a terminal application scenario, used to assist the user in completing one or more small tasks within a complex task. Furthermore, the smallest unit function is characterized by being small, precise, efficient, and scalable. For example, turning Bluetooth on, turning Bluetooth off, clearing all notification messages, silencing the phone, and taking a photo on a terminal can all be considered smallest unit functions. The smallest unit function is the basic unit constituting the application functions of the terminal. By combining and expanding these smallest unit functions, a more complex, efficient, and user-responsive application experience can be created. For example, a function belonging to the smallest unit function must satisfy at least one of the following: independence, basic nature, composability, scalability, accurate description, summarization, and redundancy removal.

[0090] Independence refers to the strong autonomy of the smallest unit function, enabling it to perform a function independently without relying on other smallest unit functions. Fundamentality refers to the versatility of the smallest unit function, allowing it to function in multiple scenarios or applications, and many complex functions are built upon these smallest unit functions. Composability refers to the ability of the smallest unit function to be combined with other smallest unit functions to build more complex functions, meeting user needs in different task scenarios. Extensibility refers to the ability of the smallest unit function to be extended or upgraded, continuously improving its performance and features to adapt to constantly changing technological environments and user needs. Accurate description refers to the use of precise descriptions for the smallest unit function; for example, opening **, editing **, viewing **, entering **, adjusting **, or changing **. Summarizing refers to the fact that for list items with similar content, such as contact lists or chat lists, the behavior of each list is similar; therefore, atomic capabilities can be summarized as descriptions like "send a message to **" or "view chat history with **". Redundancy removal refers to the fact that functions such as returning to the previous level, going back to the previous level, and returning to the home page are not considered smallest unit functions.

[0091] The interface function description information is used to indicate the interface functions. For example, the interface function description information can summarize the function of the interface in one sentence, and the interface function description information is general and comprehensive. The interface function description information does not change with the dynamic changes of the interface. For example, if the content in the input box in the interface changes dynamically, the interface function description information does not include the content in the input box.

[0092] In one possible implementation, after the terminal identifies the first feature library, it can further open the first feature library for use by other services to meet their needs. Accordingly, this process can involve setting the access permissions of the first feature library to be accessible, allowing other services to access the first feature library to assist in performing their tasks. These other services can be services on the terminal other than those executing search instructions, or services from third-party systems. For example, other services could be the terminal's intelligent assistant, thereby automatically completing tasks with the help of the first feature library. Furthermore, other services could be services in group-buying applications or short video applications, enabling these applications to provide better services to users with the help of the first feature library.

[0093] In another possible implementation, the terminal can also generate executable scripts corresponding to the features included in the first feature library. For example, the first feature library includes features corresponding to unit function information (which can be called the features of the smallest unit function), and the terminal generates executable scripts corresponding to the features of the smallest unit function. Furthermore, the terminal can store the application to which the feature of the smallest unit function belongs and the application status of the application, which can be installed or not installed. For example, referring to Table 1, the first feature library includes features of three smallest unit functions, whose feature numbers can be F01, F02, and F03, and the unit function information corresponding to the features of the three smallest unit functions are: View Device Memory, Open Care Mode, and View Current Altitude; the applications to which View Device Memory, Open Care Mode, and View Current Altitude belong are System Settings (a third-party application), Social Application (a third-party application), and Map Application (a third-party application), respectively, and the application statuses of System Settings, Social Application, and Map Application are installed, installed, and not installed, respectively; the executable scripts corresponding to View Device Memory, Open Care Mode, and View Current Altitude are executable script 1, executable script 2, and executable script 3.

[0094] Table 1

[0095] Before step 202, the terminal needs to first generate historical operation trajectories, and then generate a feature library based on these historical operation trajectories. The process of the terminal generating the feature library based on the historical operation trajectories can be as follows: the terminal determines the functional information of at least one interface in the historical operation trajectory, the functional information including at least one of unit function information and interface function description information; features are extracted from the functional information of the at least one interface to obtain the feature library. In one possible implementation, the step of the terminal determining the functional information of at least one interface in the historical operation trajectory can be as follows: the terminal determines at least one of the unit function information and interface function description information of at least one interface using a large language model.

[0096] It should be noted that the historical operation trajectory includes a first historical operation trajectory and a second historical operation trajectory. The first historical operation trajectory corresponds to the historical operation records generated by applications (first-party applications and / or third-party applications) installed on the terminal; the second historical operation trajectory corresponds to the historical operation records generated by applications not installed on the terminal (which can be historical operation records from other terminals). Therefore, the feature library generated in this step can be the first feature library corresponding to the first historical operation trajectory, while the server generates the second feature library corresponding to the second historical operation trajectory. How the first functional information is matched using the first and second feature libraries will be explained in detail in subsequent embodiments.

[0097] Please refer to Figure 3. In the preprocessing stage: the terminal generates historical operation trajectories, including a first historical operation trajectory and a second historical operation trajectory. The first historical operation trajectory corresponds to historical operation records generated by applications (first-party applications and / or third-party applications) installed on the terminal. The second historical operation trajectory corresponds to historical operation records generated by applications not installed on the terminal (which can be historical operation records from other terminals). The second historical operation trajectory can be generated by the server and then deployed to the terminal; alternatively, the second historical operation trajectory can be directly deployed on the server. After the terminal generates the historical operation trajectory, the terminal uses a large language model to determine the interface function description information and unit function information of at least one interface in the historical operation trajectory. Then, the terminal performs feature extraction on the interface function description information and unit function information to obtain a feature library, which facilitates subsequent matching of function information based on the feature library.

[0098] In one possible implementation, after the terminal determines the interface function description information and unit function information of at least one interface in the historical operation trajectory, it generates a function list corresponding to each interface. This function list includes the interface function description information and unit function information of that interface, facilitating subsequent determination of the interface to which the function information belongs based on the function list. For example, referring to Figure 4, the terminal determines that the function list of the social application includes "View Phone Number," "View Login Password," "View Voice Lock," "View Emergency Contacts," "View Logged-in Devices," "View More Security Settings," "View Security Center," and "View Personal Information and Permissions." The unit function information includes "View Care Mode Description," "View Care Mode Functions," and "Enable Care Mode." The feature library obtained by the terminal through feature extraction of the unit function information of the interfaces includes Feature 1, Feature 2, ..., Feature M, where M is an integer greater than 2.

[0099] (2) Based on the interface search conditions and the functional information of at least one interface, determine at least one second functional information, wherein the second functional information is the information of the matching function retrieved based on the interface search conditions and the functional information of at least one interface.

[0100] The matching degree between at least one feature corresponding to a second functional information and at least one functional information of an interface satisfies a second matching degree condition. The second matching degree condition can be selecting a second preset number of features with the highest matching degree or selecting features with a matching degree exceeding the second preset matching degree. An interface includes at least one functional information; for example, an interface includes multiple unit functional information and an interface functional description information.

[0101] In one possible implementation, the terminal segments the text information corresponding to the interface search conditions to obtain multiple keywords; determines the matching degree between the multiple keywords and the functional information of at least one interface; and, based on the matching degree between the multiple keywords and the functional information of at least one interface, determines at least one second functional information from the functional information of at least one interface, wherein the matching degree between the at least one second functional information and the multiple keywords satisfies the second matching degree condition. For example, the terminal uses the Best Matching 25 (bm25) function to determine the matching degree between the multiple keywords and the functional information of at least one interface, and determines at least one second functional information from the functional information of at least one interface based on the matching degree between the multiple keywords and the functional information of at least one interface. Here, the bm25 function is a ranking function in information retrieval used to evaluate the relevance of text information under a given search query; before using the bm25 function, the text information needs to be segmented to convert the interface search conditions into multiple keywords. For example, if the interface search condition is "I want to turn on the Wireless Fidelity (WiFi) WiFi switch", the terminal will perform word segmentation on the interface search condition to obtain the word segmentation results as ["I", "want", "turn on", "WiFi", "switch"]. Then, the bm25 function is used to match the keywords in the interface search condition with the functional information of at least one interface to obtain at least one secondary functional information.

[0102] (3) Determine the first target function information based on at least one first function information and at least one second function information.

[0103] This step can be achieved through the following steps (3-1) to (3-2), including:

[0104] (3-1) Merge and sort at least one first functional information and at least one second functional information.

[0105] At least one first functional information and at least one second functional information are merged and sorted in descending order of weight. In one possible implementation, at least one first functional information is sorted sequentially based on its matching degree with interface retrieval features, and at least one second functional information is sorted sequentially based on its matching degree with interface retrieval conditions. Accordingly, the terminal's step of merging and sorting at least one first functional information and at least one second functional information can be as follows: for any one of the at least one first functional information and at least one second functional information, the terminal determines the weight of the functional information based on its order, where the weight of the functional information is negatively correlated with its order; and sorts the at least one first functional information and at least one second functional information based on their respective weights.

[0106] For ease of description, at least one first functional information, ranked sequentially based on its matching degree with interface search features, is referred to as the first search result; at least one second functional information, ranked sequentially based on its matching degree with interface search conditions, is referred to as the second search result. Correspondingly, the step of the terminal determining the weight of the functional information based on its order can be as follows: if the functional information is in the first search result but not in the second search result, the terminal determines the weight of the functional information based on its order in the first search result; if the functional information is in the second search result but not in the first search result, the terminal determines the weight of the functional information based on its order in the second search result; if the functional information is in both the first and second search results, the terminal determines the weight of the functional information based on both its order in the first and second search results. For example, the terminal determines the first weight of the functional information based on its order in the first search result, determines the second weight based on its order in the second search result, and then performs a weighted sum of the first and second weights to obtain the final weight of the functional information.

[0107] For example, the first search result is A(1 / 2), B(1 / 4), C(1 / 8), D(1 / 16); the second search result is D(1 / 2), B(1 / 4), A(1 / 8), C(1 / 16); then the weight of A is 1 / 2 + 1 / 8 = 5 / 8; the weight of B is 1 / 4 + 1 / 4 = 1 / 2; the weight of D is 1 / 16 + 1 / 2 = 7 / 16; the weight of C is 1 / 8 + 1 / 16 = 3 / 16; therefore, the functional information sequence obtained by merging and sorting A, B, C, and D by the terminal is A, B, D, C.

[0108] In this step, at least one first functional information and at least one second functional information can also be merged and sorted based on the order of arrival from smallest to largest weight.

[0109] (3-2) Based on the sequence of functional information obtained by sorting, determine the first target functional information.

[0110] In one possible implementation, the terminal automatically determines the first target function information based on the weights of the function information. Accordingly, step (3-2) could be: the terminal, based on the sorted sequence of function information, determines the function information with the highest weight in the sequence as the first target function information; for example, if the function information sequence is sorted in descending order of weight, the terminal determines the first function information in the sequence as the first target function information; or, if the function information sequence is sorted in ascending order of weight, the terminal determines the last function information in the sequence as the first target function information. In another possible implementation, after the terminal sorts the function information sequence, the user selects the first target function information; accordingly, step (3-2) could be: the terminal displays the sorted sequence of function information, determines the selected function information in the sequence, and identifies the selected function information as the first target function information.

[0111] In this embodiment of the application, since the function retrieval has a high real-time requirement, a combination strategy of two-way recall (retrieving function information based on interface retrieval features and interface retrieval conditions respectively) + user selection (user manually selects function information) is adopted to determine the first target function information, thereby improving the accuracy of the determined first target function information.

[0112] Step 203: Determine the first target interface to which the first target function information belongs.

[0113] After the terminal determines the interface function description information and unit function information of at least one interface in the historical operation trajectory, it generates a function list corresponding to each interface. The function list corresponding to the interface includes the interface function description information and unit function information of that interface. In this step, the terminal determines the function list where the first target function information is located and determines the first target interface to which the function list belongs.

[0114] For example, please refer to Figure 3. In the recall stage: the terminal extracts features from the interface search conditions to obtain interface search features (embedding). Based on the interface search features and the feature library, one recall is performed to obtain the first search result; based on the interface search conditions and the interface's functional information, another recall (SQL-like) is performed to obtain the second search result. The first and second search results are sorted using the ranking (bm25) function to obtain the functional information sequence (ReRank). The first target interface is determined based on the functional information sequence.

[0115] In this embodiment, a two-pronged recall retrieval is performed based on interface retrieval features and a feature library, and another recall retrieval is performed based on interface retrieval conditions and interface functional information. The results of both recall retrievals are combined to determine the first target interface that the user wants to retrieve, thus achieving two-pronged recall retrieval to find the first target interface, improving the accuracy of the retrieved first target interface. Furthermore, in this embodiment, interface retrieval can be achieved based on historical operation trajectories, eliminating the need to configure the search engines for in-application retrieval and system-level retrieval line by line, thereby reducing the cost of interface retrieval.

[0116] Please refer to Figure 5, which shows a flowchart of an interface retrieval method provided in an exemplary embodiment of this application. This embodiment uses the method applied to a terminal as an example for illustration, and the method may include the following steps:

[0117] Step 501: In response to the search command, extract features from the interface search conditions indicated by the search command to obtain interface search features.

[0118] In some embodiments, this step is the same as step 201, and will not be described again here.

[0119] In one possible implementation, after the terminal completes step 501, it can execute steps 502-503 or steps 504-505; the terminal can also execute step 502 first, and if the first feature is found based on step 502, execute step 503; if the first feature is not found, execute step 504; the terminal can also execute steps 502-503 first to find the first feature on the terminal side, and then execute steps 504-505 to find the first feature on the service side.

[0120] Step 502: Based on the interface retrieval features, search for features that match the interface retrieval features from the first feature library. The first functional information is the information of the matching functions retrieved based on the interface retrieval features and the feature library.

[0121] The first feature library stores features corresponding to the functional information of at least one interface in the first historical operation trajectory. The first historical operation trajectory is the historical operation trajectory corresponding to the historical operation records generated by the applications installed on the terminal. After receiving a search instruction, the terminal first searches for features matching the interface search features in the first feature library; if at least one first feature is found, step 503 is executed; if no feature is found, step 504 is executed.

[0122] In one possible implementation, the terminal determines the matching degree between the interface retrieval features and each feature in the first feature library; based on the matching degree between the interface retrieval features and each feature in the first feature library, it determines at least one first feature from the first feature library whose matching degree satisfies a third matching degree condition. The third matching degree condition may be selecting a third preset number of features with the highest matching degree or selecting features with a matching degree exceeding a third preset matching degree.

[0123] Step 503: If at least one first feature is found, determine at least one first functional information corresponding to the at least one first feature.

[0124] The terminal stores the correspondence between features and functional information in advance; accordingly, based on at least one first feature, the terminal determines at least one functional information corresponding to at least one first feature from the correspondence between features and functional information.

[0125] Step 504: Send a query request to the server.

[0126] The query request carries interface retrieval features, enabling the server to search for at least one first feature matching the interface retrieval features from a second feature library, and send at least one first function information corresponding to the at least one first feature to the terminal. The second feature library includes features corresponding to the function information of at least one interface in a second historical operation trajectory, which is the historical operation trajectory corresponding to historical operation records generated by applications not installed on the terminal. The process by which the server determines at least one first feature from the second feature library based on the interface retrieval features is similar to the process by which the terminal determines at least one first feature from the first feature library based on the interface retrieval features, and will not be described further here.

[0127] Step 505: Receive at least one first function information sent by the server.

[0128] In one possible implementation, the terminal may also obtain the second feature library and the correspondence between features and functional information in the second feature library from the server in advance, and then store the second feature library and the correspondence between features and functional information locally; accordingly, steps 504-505 can be replaced as follows: if no feature matching the interface retrieval feature is found, the terminal searches for a feature matching the interface retrieval feature in the second feature library based on the interface retrieval feature to obtain at least one first feature, and then determines at least one functional information corresponding to at least one first feature from the correspondence between features and functional information based on at least one first feature.

[0129] In this embodiment, since the second feature library contains features corresponding to the functional information of the interface in applications not installed on the terminal, searching for functional information based on the second feature library can enable the retrieval of the target interface in applications not installed.

[0130] In one possible implementation, after the terminal finds at least one first functional information based on the second feature library, it can further guide the user to install at least one first target application to which the first functional information belongs. This process can be as follows: the terminal determines the first target application to which at least one first functional information belongs; if the first target application is not installed on the terminal, it generates installation guidance information for the first target application based on the first target application. This installation guidance information is used to instruct the user on the installation process of the first target application; and the installation guidance information is displayed on the current interface. The terminal can input the first target application into a large language model and output the installation guidance information for the first target application.

[0131] In another possible implementation, the installation guidance information of the first target application can be generated by the server. Accordingly, after the server finds at least one first functional information based on the second feature library, it determines the first target application to which the at least one first functional information belongs. Based on the first target application, the server generates installation guidance information for the first target application and sends the installation guidance information for the first target application to the terminal. The terminal receives the installation guidance information for the first target application sent by the server and displays the installation guidance information for the first target application on the current interface, so that the user can install the first target application based on the installation guidance information for the first target application.

[0132] In this embodiment of the application, for a first target application that is not installed on the terminal, after the user finds matching first functional information in the second feature database in the server through the interface search conditions, the terminal can provide the user with "recommended applications" to meet the user's needs, thereby improving the application installation rate.

[0133] Step 506: Based on the interface search conditions and the functional information of at least one interface, determine at least one second functional information, wherein the second functional information is the information of the matching function retrieved based on the interface search conditions and the functional information of at least one interface.

[0134] In some embodiments, this step is the same as step 203, and will not be described again here.

[0135] Step 507: Based on at least one first functional information and at least one second functional information, determine a first target interface. The first target interface is the interface to which the first target functional information in the at least one first functional information and at least one second functional information belongs, and the first target interface is the interface retrieved based on the search command.

[0136] In some embodiments, this step is the same as step 204, and will not be repeated here. In one possible implementation, after determining the first target interface, and confirming that the first target interface is a target interface for which no application is installed, the terminal may guide the user to install the second target application to which the first target interface belongs. Accordingly, this process can be as follows: the terminal determines the second target application to which the first target interface belongs; based on the second target application, it generates installation guidance information for the second target application, which is used to instruct the user on the process of installing the second target application; the installation guidance information for the second target application is displayed on the current interface so that the user can install the second target application based on the installation guidance information. The terminal inputs the second target application into the large language model and outputs the installation guidance information for the second target application.

[0137] In this embodiment of the application, for a second target application that is not installed on the terminal, after the user finds a matching first target interface in the second feature database in the server through interface search conditions, the terminal can provide the user with "recommended applications" to meet the user's needs, thereby improving the application installation rate.

[0138] For example, referring to Figure 6, the user searches for functional information in the server's second feature library via the terminal. If the first target functional information is found, installation guidance information for the second target application to which the first target functional information belongs is generated, thereby making functional recommendations based on the installation guidance information of the second target application. In one possible implementation, before generating the installation guidance information of the second target application, the terminal can also generate third tutorial information for the first target functional information, which is used to guide the use of the first target functional information. In one possible implementation, the step of the terminal generating the third tutorial information for the first target functional information can be: the terminal generates the third tutorial information for the first target functional information based on the first target functional information; wherein, the terminal inputs the first target functional information into a large language model and outputs the third tutorial information for the first target functional information.

[0139] In this embodiment of the application, when determining at least one first functional information, features matching the interface retrieval features are searched from the first feature library (the feature library corresponding to the installed application) and the second feature library (the feature library corresponding to the uninstalled application) based on the interface retrieval features. Therefore, it is possible not only to retrieve the target interface of the installed application, but also to retrieve the target interface of the uninstalled application, thereby improving the success rate of interface retrieval.

[0140] Please refer to Figure 7, which shows a flowchart of an exemplary embodiment of the interface retrieval method provided in this application. This embodiment uses the method applied to a terminal as an example for illustration, and the method may include the following steps:

[0141] Step 701: In response to the search command, extract features from the interface search conditions indicated by the search command to obtain interface search features.

[0142] In some embodiments, this step is the same as step 201, and will not be described again here.

[0143] Step 702: Based on the interface retrieval features and the feature library, determine at least one first functional information. The feature library is used to store the features corresponding to the functional information of at least one interface in the historical operation trajectory. The historical operation trajectory is the trajectory model corresponding to the historical operation process. The functional information is used to indicate the functions of the interface.

[0144] In some embodiments, this step is the same as step 202; or, this step can be implemented by the above steps 502-505, which will not be described again here.

[0145] Step 703: Based on the interface retrieval conditions and the functional information of at least one interface, determine at least one second functional information.

[0146] In some embodiments, this step is the same as step 203, and will not be described again here.

[0147] For example, referring to Figure 9, the terminal locates at least one first function information and at least one second function information based on "I want to turn on care mode" as "View care mode", "Turn on care mode", "View care mode description", "View care mode functions" and "View dark mode".

[0148] Step 704: Based on at least one first functional information and at least one second functional information, determine the first target interface, which is the interface retrieved based on the search command.

[0149] In some embodiments, this step is implemented in conjunction with step 204 or 507, and will not be described again here.

[0150] Step 705: Based on the first target interface, determine the first target path information from the historical operation trajectory. The historical operation trajectory is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation process and the path information of the operation path corresponding to the interface during the historical operation process. The first target operation path to which the first target path information belongs is the operation path to reach the first target interface.

[0151] This step can be achieved through the following steps (1) to (3), including:

[0152] (1) Based on the first target interface, determine the first target node corresponding to the first target interface from the historical operation trajectory.

[0153] Since the interface features of the interface are represented by functional information in the historical operation trajectory, and the identification of whether two interfaces are the same interface is usually done by interface feature matching, the terminal determines the first target node (recorded node) corresponding to the first target interface based on the functional information of the recorded nodes corresponding to the recorded interfaces in the historical operation trajectory. In some embodiments, the terminal generates functional information corresponding to the first target interface using the same functional information generation method as the recorded interfaces, and then determines the first target node corresponding to the first target interface by comparing the functional information of the recorded nodes corresponding to the recorded interfaces with the functional information corresponding to the first target interface.

[0154] In some embodiments, the functional information of a node includes a node identifier and node factors. The node identifier is a unique identifier assigned to a node (without practical meaning) and can be represented as a string. Different nodes have different node identifiers, while the same node has the same node identifier. Optionally, a node factor consists of several fields that characterize interface features from a functional dimension. These fields characterizing interface features can be called node sub-factors. When determining whether two nodes are the same node, the terminal matches the content of fields with the same field name in the two node factors, that is, it matches the node sub-factors in different node factors, thereby determining whether the two nodes are the same node based on the matching results from the functional dimension. In some embodiments, a node factor may include at least one of a first node sub-factor and a second node sub-factor.

[0155] First-level sub-factor: During the interface development phase, developers typically add interface function description information to the interface code. Even with subsequent interface effect iterations and updates, this interface function description information usually remains unchanged. Therefore, the terminal can determine whether two nodes are the same node from the dimension of interface function.

[0156] In one possible implementation, the terminal determines the first node sub-factor based on the interface function description information. This interface function description information can be extracted from the interface code. Furthermore, when identifying whether two nodes are the same node, the terminal can compare whether the first node sub-factors in the node factors of each of the two nodes are the same. If they are the same, the two nodes are determined to be the same node; if they are different, the terminal needs to make further determinations based on other node sub-factors.

[0157] Second Node Sub-Factor: In one possible implementation, the terminal determines the second node sub-factor based on the unit function information of the interface. The unit function information indicates the smallest unit function of the interface, such as opening function A, closing function A, viewing B, adding C, uploading D, etc. Regarding the method of generating unit function information, in some embodiments, the terminal generates the unit function information of the interface through a large language model. Optionally, the terminal inputs a screenshot of the interface and a prompt message into the large language model, instructing the large language model to extract the unit function information of the interface. This unit function information can be in the form of a list. In one possible implementation, the terminal can generate a node factor corresponding to the first target interface and compare this node factor with the node factors of already recorded nodes to determine the first target node corresponding to the first target interface. This step may include the following sub-steps:

[0158] (1-1) Based on the interface features of the functional dimension of the first target interface, generate the node factors corresponding to the first target interface.

[0159] In one possible implementation, the terminal generates corresponding node sub-factors based on the interface features of the first target interface's functional dimension, and then determines the node factors corresponding to the first target interface. In some embodiments, the node factors corresponding to the first target interface may include all or some of the aforementioned node sub-factors. For example, the terminal determines the first and second node sub-factors corresponding to the first target interface.

[0160] (1-2) Based on the node factors of the recorded nodes corresponding to the recorded interfaces in the historical operation trajectory and the node factors corresponding to the first target interface, determine the recorded interfaces that match the first target interface.

[0161] In one possible implementation, the terminal matches the node factors of the first target interface with the node factors corresponding to the recorded interfaces in the historical operation trajectory one by one, thereby determining the recorded interface that matches the first target interface. Optionally, the terminal can determine the recorded interface that matches the first target interface based on a single node sub-factor in the node factors, or by combining multiple node sub-factors in the node factors. In some embodiments, when the node factors include the aforementioned first to second node sub-factors, the terminal's determination of whether the first target interface matches the recorded interface may include the following cases.

[0162] Case 1: If the first target interface and the first node sub-factor corresponding to the recorded interface are the same, then the first target interface and the recorded interface are determined to be a match.

[0163] If two interfaces have completely identical functions, then these two interfaces should belong to the same interface. Therefore, in some embodiments, when the first node sub-factors of both the first target interface and the recorded interface are not empty, the terminal compares whether the first node sub-factors of the two are the same, and if the first node sub-factors of the two are the same, it determines that the first target interface matches the recorded interface.

[0164] Case 2: If the second node sub-factors corresponding to the first target interface and the recorded interface are the same, then the first target interface and the recorded interface are determined to be a match.

[0165] (1-3) The recorded node corresponding to the matched recorded interface is determined as the first target node corresponding to the first target interface.

[0166] In some embodiments, when a single recorded interface is matched, the terminal determines the recorded node corresponding to that recorded interface as the first target node corresponding to the first target interface; when multiple recorded interfaces are matched, the terminal determines the recorded node corresponding to the recorded interface with the highest matching degree as the first target node corresponding to the first target interface.

[0167] (2) Determine the first target behavior edge corresponding to the first target operation behavior involved in the first target operation path including the first target node from the historical operation trajectory.

[0168] In one possible implementation, this step can be achieved through the following steps (2-1) to (2-3), including:

[0169] (2-1) From the historical operation trajectory, determine the starting node corresponding to the starting interface before the first target operation and the ending node corresponding to the ending interface after the first target operation.

[0170] In one possible implementation, the terminal determines the starting node corresponding to the starting interface based on the node information corresponding to the starting interface, and obtains the starting node identifier of the starting node; based on the node information corresponding to the ending interface, it determines the ending node corresponding to the ending interface, and obtains the ending node identifier of the ending node.

[0171] In some embodiments, in order to ensure the accuracy of the determined termination node, when the duration of receiving the first target operation behavior reaches a preset duration (because there may be a response delay when the interface jumps), the terminal determines the current interface as the termination interface, and then determines the termination node corresponding to the termination interface.

[0172] (2-2) When the first target operation behavior is related to the first target interface control, determine the node element corresponding to the first target interface control from the historical operation trajectory.

[0173] In some embodiments, the terminal determines whether the first target operation is related to the first target interface control. If the first target operation is related to the first target interface control (i.e., the first target operation acts on the first target interface control), the terminal determines the node element corresponding to the first target interface control. If the first target operation is unrelated to the first target interface control (for example, the first target operation is an automatic interface jump behavior of the system after a specified time), the terminal does not need to determine the node element.

[0174] In one possible implementation, the historical operation trajectory includes node element information of the corresponding node elements of each recorded interface control within the recorded interface. Accordingly, the terminal determines the node element information corresponding to the first target interface control and matches this node element information with the node element information of the recorded interface control's corresponding node elements, thereby identifying the matched recorded node element as the node element corresponding to the first target interface control. In some embodiments, the terminal determines a node element factor (part of the node element information, used to characterize the node element's features from multiple dimensions) corresponding to the first target interface control, and then determines the node element corresponding to the first target interface control based on the matching degree between this node element factor and the node element factor of the recorded interface control's corresponding node elements.

[0175] (2-3) Based on the starting node, ending node, node elements and the first target behavior type of the first target operation behavior, determine the first target behavior edge corresponding to the first target operation behavior.

[0176] In one possible implementation, the terminal matches the start node identifier of the start node, the end node identifier of the end node, the node element identifier of the node element, and the first target behavior type of the first target operation behavior with the edge information of the recorded behavior edge corresponding to the recorded operation behavior in the historical operation trajectory, thereby determining the matched recorded behavior edge as the first target behavior edge corresponding to the first target operation behavior.

[0177] (3) Based on the functional information of the first target node and the first target edge information of the first target behavior edge, generate the first target path information of the first target operation path.

[0178] After identifying the first target nodes traversed along the first target operation path from the historical operation trajectory, and the first target behavior edges between these nodes, the terminal can generate the first target path information. Based on this first target path information, the terminal can determine the nodes along the first target operation path and the behavior edges between them.

[0179] In one possible implementation, a historical operation trajectory containing the target node corresponding to the first target interface is found. Then, within this historical operation trajectory, using the target node corresponding to the first target interface as the endpoint, a breadth-first traversal algorithm is used to determine the first target operation path reaching that target node. For example, referring to Figure 3, during the localization phase, the terminal locates the first target path information from the historical operation trajectory based on the first target interface.

[0180] The essence of the terminal determining the first target path information from historical operation trajectories is to find a first target operation path that can reach the first target interface from the starting interface. For example, the first target operation path from the starting interface to the first target interface is path 1 to path N, where N is an integer greater than 1. The terminal determines the complete path of the first target operation path using a breadth-first traversal algorithm as: path 1 to path 2 to path 3 to path 4 to path N.

[0181] Step 706: Based on the first target edge information included in the first target path information, execute the first target operation behavior indicated by the first target edge information. The first target edge information includes information used to characterize the first target operation behavior and the interface jump situation before and after the first target operation behavior.

[0182] After determining the first target path information, the terminal in this embodiment can execute the first target operation behavior indicated by the first target edge information included in the first target roadbed information, thereby automatically executing the first target operation behavior, that is, automatically jumping from the starting interface to the first target interface. For example, please continue to refer to Figure 3. In the flashback stage, the terminal automatically jumps from the starting interface to the first target interface based on the playback function.

[0183] For example, from a product perspective, the process of this application embodiment is shown in Figure 8. The terminal generates a historical operation trajectory in advance and generates a feature library based on the historical operation trajectory. The interface search condition can be "I want to turn on care mode". The terminal locates the first target interface based on the interface search condition, determines the first target path information based on the first target interface, and plays back based on the first target path information. Then, it generates shareable target tutorial information through step 707.

[0184] For example, referring to Figure 10, if the interface search condition is "view emergency contacts," the terminal determines the first target interface as the emergency contacts interface based on the interface search condition. It then locates the position of the emergency contacts interface in the historical operation trajectory, and finally determines and replays the first target path information based on its position in the historical operation trajectory. For example, referring to Figure 11, the first target operation indicated by the first target side information includes "return to desktop" - executing the second step "open social application" - executing the third step "click me" - executing the fourth step "click settings" - executing the fifth step "click account and security" - executing the sixth step "click emergency contacts," thus jumping to the emergency contacts interface.

[0185] In one possible implementation, after the terminal determines the first target path information, it displays the first target path information on the current interface and displays a schematic diagram of each interface involved in the first target path information; however, the first path information and the schematic diagram of each interface may not be very intuitive and lack detailed explanation; while a good tutorial can more clearly and thoroughly explain the detailed process from the initial interface to the first target interface; therefore, after executing step 706, step 707 is executed to generate tutorial information.

[0186] Step 707: Based on the first target edge information included in the first target path information, generate target tutorial information corresponding to the first target edge information. The target tutorial information is used to guide the completion of the first target operation behavior indicated by the first target edge information.

[0187] The terminal generates first tutorial information corresponding to the first target edge information based on the first target path information, including the first target edge information. The first tutorial information is used to guide the execution of the first target operation behavior indicated by the first target edge information using text. Based on the first tutorial information, the terminal determines the target tutorial information. Specifically, the terminal inputs the first target edge information into the large language model and outputs the first tutorial information corresponding to the first target edge information.

[0188] In one possible implementation, the step of generating first tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information can be as follows: The terminal generates the first tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information and the interface information of multiple interfaces included in the first target operation path to which the first target path information belongs. For example, if the first target function information is "Enable the care mode of the social application", then the interface information and target edge information of the multiple interfaces are as follows:

[0189] Interface information: Desktop; Target side information: Click "Social App"; Interface information: Main interface of Social App; Target side information: Click the text "Me" in the lower right corner of the screen; Interface information: Function interface of Social App; Target side information: Click the text "Settings" in the middle of the screen; Interface information: Settings interface of Social App; Target side information: Click the text "Care Mode" in the upper part of the screen; Interface information: Detailed explanation and activation interface of Care Mode.

[0190] The terminal inputs the above information into the large language model and outputs the first tutorial information; the first tutorial information can be as follows: Title: "Take You Through Social Apps: How to Enable the Care Mode in Social Apps" Content: Hello everyone! Today I'll guide you through a very useful feature of social apps—"Care Mode." Below, I will explain the detailed steps one by one: Step 1: Launch the social app. First, find the social app icon on your phone's home screen and tap to enter. You will see the chat history list interface of the social app. Tip: This list interface allows you to easily browse and manage all your social chat history, and also allows you to search and access more functions. Step 2: Enter the "Me" interface. In the bottom navigation bar of the list interface, slide to the right to the far right of a row of small text and tap the "Me" text to enter the personal page of the social app. Tip: This interface allows us to view personal information and switch between the list interface, contacts, discovery, and other function modules. Step 3: Tap "Settings". Find and tap the "Settings" item in the middle of the "Me" interface to enter the settings interface of the social app. Tip: The "Settings" section contains various settings options for social media apps. It can help you with any modifications you need. Step 4: Enable Care Mode. Swipe up in the Settings interface to find "Care Mode" at the top of the page. This will take you to the detailed Care Mode page. Tip: This is the detailed explanation page for the social media app's "Care Mode." You can learn more about "Care Mode" and enable it here. These are all the steps to enable Care Mode in a social media app. Following these steps should allow you to successfully enable this feature.

[0191] In one possible implementation, the target tutorial information is text information. The step of the terminal determining the target tutorial information based on the first tutorial information can be: the terminal determines the first tutorial information as the target tutorial information. In another possible implementation, the target tutorial information is graphic information. The step of the terminal determining the target tutorial information based on the first tutorial information can be: the terminal obtains second tutorial information based on at least one of the functional information, target edge information, and first tutorial information from the historical operation trajectory, and determines the second tutorial information as the target tutorial information. The target tutorial information is the use of graphic information to guide the execution of the first target operation behavior indicated by the first target edge. Here, the first tutorial information and functional information are text information, and the target edge information is image information; therefore, the target tutorial information is graphic information. For example, if the terminal obtains the second tutorial information based on three of the functional information, target edge information, and first tutorial information from the historical operation trajectory, the process can be: the terminal merges the functional information, target edge information, and first tutorial information from the historical operation trajectory to obtain the second tutorial information.

[0192] In another possible implementation, where the target tutorial information is video information, the steps for the terminal to determine the target tutorial information based on the first tutorial information can be as follows: the terminal obtains second tutorial information based on at least one of the functional information in the historical operation trajectory, the first target edge information, and the first tutorial information; the second tutorial information is converted into video format tutorial information to obtain the target tutorial information, which is video guidance for performing the first target operation behavior indicated by the first target edge. For example, the terminal uses slideshow technology and text-to-speech (TTS) capabilities to convert the second tutorial information into video format tutorial information to obtain the target tutorial information, thereby making the target tutorial information more intuitive.

[0193] In this embodiment, the target tutorial information includes detailed steps; therefore, the terminal provides the user with target tutorial information, enabling more detailed guidance for the user to perform the first target operation. Furthermore, the target tutorial information is generated based on a large language model, thus requiring no manual intervention and achieving an automated process. Moreover, the target tutorial information can be in at least one of text, image, and video formats; therefore, the formats of target tutorial information are enriched. For example, referring to Figure 12, the terminal generates target tutorial information in plain text format, image and text format, and video format.

[0194] In one possible implementation, the terminal can also allow users to learn about and use the functions on the terminal through dialogue; correspondingly, the process can be: receiving a first voice command, which is used to request the acquisition of second target function information; based on the first voice command, generating second tutorial information of the second target function information, which is used to guide the use of the second target function; wherein, the terminal inputs the first voice command into the large language model and outputs the second tutorial information of the second target function information.

[0195] In another possible implementation, the terminal can also utilize the flashback function of historical operation trajectories to allow users to complete complex tasks by issuing action commands via voice. Correspondingly, the process can be as follows: receiving a second voice command, which requests the display of a second target interface; determining the second target path information from the historical operation trajectory based on the second voice command, the historical operation trajectory also recording the edge information of the action edges corresponding to the operation behaviors within the interface during historical operations and the path information of the operation paths corresponding to the interfaces during historical operations, wherein the second target path information belongs to the second target operation path that leads to the second target interface; and executing the second target operation behavior indicated by the second target edge information based on the second target edge information included in the second target path information, the second target edge information including information representing the second target operation behavior and the interface transitions before and after the second target operation behavior. For example, referring to Figure 12, the terminal controls itself through human-computer interaction to understand its functions or control it to complete a task.

[0196] Step 708: Display the target tutorial information on the current screen.

[0197] The target tutorial information can also be shared via social media applications. For example, referring to Figure 3, during the positioning phase, the terminal generates target tutorial information and shares it via social media applications.

[0198] This application provides a unified solution for full-function search and device assistant. For example, referring to Figure 13, a historical operation trajectory is first generated, including the historical operation trajectory of a third-party application, a third-party application, and an application not installed. Then, a first feature library is generated based on the historical operation trajectory. Based on the first feature library and the functional information of at least one interface in the historical operation trajectory, the solution integrates function retrieval (retrieving the first target interface), automatic execution (automatically jumping to the first target interface), and tutorial generation through search, location, playback, large language model, and Retrieval-Augmented Generation (RAG) retrieval technologies, forming a complete technical approach. From the perspective of device manufacturers, this solution has significant value, providing users with search and usage instructions for any function across the entire device and even the entire mobile ecosystem.

[0199] Please refer to Figure 14, which shows a structural block diagram of an interface retrieval device provided in an exemplary embodiment of this application. The device includes: a first processing module 1401, configured to extract features from interface retrieval conditions indicated by a retrieval command in response to a retrieval command, thereby obtaining interface retrieval features; a first determining module 1402, configured to determine first target function information based on the interface retrieval features, a feature library corresponding to a historical operation trajectory, the interface retrieval conditions, and the historical operation trajectory, wherein the historical operation trajectory records function information of at least one interface during historical operations, the feature library stores features corresponding to the function information of the at least one interface, and the function information indicates the functions possessed by the interface; and a second determining module 1403, configured to determine a first target interface to which the first target function information belongs, wherein the first target interface is an interface retrieved based on the retrieval command.

[0200] In one possible implementation, the first determining module 1402 is configured to: determine at least one first functional information based on the interface retrieval features and the feature library, wherein the first functional information is information on matching functions retrieved based on the interface retrieval features and the feature library; determine at least one second functional information based on the interface retrieval conditions and the functional information of the at least one interface, wherein the second functional information is information on matching functions retrieved based on the interface retrieval conditions and the functional information of the at least one interface; and determine the first target functional information based on the at least one first functional information and the at least one second functional information.

[0201] In another possible implementation, the first determining module 1402 is used to search for features that match the interface retrieval features from a first feature library based on the interface retrieval features. The first feature library is used to store features corresponding to the functional information of at least one interface in the first historical operation trajectory. The first historical operation trajectory is the historical operation trajectory generated by the application installed on the terminal. If at least one first feature is found, at least one first functional information corresponding to the at least one first feature is determined.

[0202] In another possible implementation, the first determining module 1402 is configured to send a query request to the server, the query request carrying the interface retrieval features, so that the server searches for at least one first feature matching the interface retrieval features from a second feature library based on the interface retrieval features, and sends at least one first function information corresponding to the at least one first feature to the terminal, the second feature library including features corresponding to the function information of at least one interface in a second historical operation trajectory, the second historical operation trajectory being the historical operation trajectory generated by applications not installed on the terminal; and receives at least one first function information sent by the server.

[0203] In another possible implementation, the apparatus further includes: a third determining module, configured to determine a first target application to which the at least one first functional information belongs; a first generating module, configured to generate installation guidance information for the first target application based on the first target application if the first target application is an application not installed on the terminal, the installation guidance information for the first target application being used to instruct the process of installing the first target application; and a first display module, configured to display the installation guidance information for the first target application on the current interface.

[0204] In another possible implementation, the device further includes: a fourth determining module, configured to determine functional information of at least one interface in the first historical operation trajectory, the functional information including at least one of unit function information and interface function description information, the unit function information indicating the smallest unit function of the interface, and the interface function description information indicating the interface function of the interface; and a second processing module, configured to perform feature extraction on the functional information of the at least one interface to obtain the first feature library.

[0205] In another possible implementation, the apparatus further includes: setting a service to enable access permissions for the first feature library, allowing other services to access the first feature library to assist in performing tasks of the other services, wherein the other services are services in the terminal other than executing the search instruction, or the other services are services in a third-party system.

[0206] In another possible implementation, the first determining module 1402 is used to merge and sort the at least one first functional information and the at least one second functional information; and determine the first target functional information based on the sorted functional information sequence.

[0207] In another possible implementation, the at least one first functional information is ordered sequentially based on its matching degree with the interface retrieval features, and the at least one second functional information is ordered sequentially based on its matching degree with the interface retrieval conditions; the first determining module 1402 is used to determine the weight of any one of the at least one first functional information and the at least one second functional information based on the order of the functional information, wherein the weight of the functional information is negatively correlated with the order; and to sort the at least one first functional information and the at least one second functional information based on the weight of the at least one first functional information and the weight of the at least one second functional information.

[0208] In another possible implementation, the first determining module 1402 is used to segment the text information corresponding to the interface search conditions to obtain multiple keywords; determine the matching degree between the multiple keywords and the functional information of the at least one interface; and determine at least one second functional information from the functional information of the at least one interface based on the matching degree between the multiple keywords and the functional information of the at least one interface, wherein the matching degree between the at least one functional information and the multiple keywords satisfies the matching degree condition.

[0209] In another possible implementation, the device further includes: a fifth determining module, configured to determine first target path information from the historical operation trajectory based on the first target interface, wherein the historical operation trajectory is further configured to record edge information of the behavior edge corresponding to the operation behavior within the interface during the historical operation process and path information of the operation path corresponding to the interface during the historical operation process, and the first target operation path to which the first target path information belongs is the operation path leading to the first target interface; and a first execution module, configured to execute the first target operation behavior indicated by the first target edge information based on the first target edge information included in the first target path information, wherein the first target edge information includes information used to characterize the first target operation behavior and the interface jump situation before and after the first target operation behavior.

[0210] In another possible implementation, the device further includes: a second generation module, configured to generate target tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information, the target tutorial information being used to guide the completion of the first target operation behavior indicated by the first target edge information; and a second display module, configured to display the target tutorial information on the current interface.

[0211] In another possible implementation, the second generation module is used to generate first tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information. The first tutorial information is used to guide the execution of the first target operation behavior indicated by the first target edge information in text. Based on the first tutorial information, the target tutorial information is determined.

[0212] In another possible implementation, the second generation module is configured to determine the first tutorial information as the target tutorial information. In yet another possible implementation, the second generation module is configured to obtain second tutorial information based on at least one of the functional information in the historical operation trajectory, the target edge information, and the first tutorial information, and determine the second tutorial information as the target tutorial information, wherein the target tutorial information is a guide to perform the first target operation behavior indicated by the first target edge using graphic and textual information.

[0213] In another possible implementation, the second generation module is configured to obtain second tutorial information based on at least one of the functional information, the first target edge information, and the first tutorial information in the historical operation trajectory, convert the second tutorial information into video format tutorial information, and obtain the target tutorial information, wherein the target tutorial information is a video guide to perform the first target operation behavior indicated by the first target edge.

[0214] In another possible implementation, the device further includes: a receiving module for receiving a voice command, the voice command being used to request the display of a second target interface; a sixth determining module for determining second target path information from the historical operation trajectory based on the voice command, the historical operation trajectory also being used to record edge information of the behavior edge corresponding to the operation behavior within the interface during the historical operation process and path information of the operation path corresponding to the interface during the historical operation process, and the second target operation path to which the second target path information belongs is the operation path leading to the second target interface; and a first execution module for executing the second target operation behavior indicated by the second target edge information based on the second target edge information included in the second target path information, the second target edge information including information for characterizing the second target operation behavior and the interface jump situation before and after the second target operation behavior.

[0215] In this embodiment, a recall retrieval is performed based on interface retrieval features and feature library, and another recall retrieval is performed based on interface retrieval conditions and interface functional information. The first target interface that the user wants to retrieve is determined by comprehensively considering the results of the two recall retrievals, thereby realizing two-way recall retrieval to retrieve the first target interface and improving the accuracy of the retrieved first target interface.

[0216] It should be noted that the interface retrieval device provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal can be divided into different functional modules to complete all or part of the functions described above. In addition, the interface retrieval device and the interface retrieval method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0217] Please refer to Figure 15, which shows a block diagram of a terminal illustrated in an exemplary embodiment of this application. The terminal in this application may include one or more components such as a processor 1510, a memory 1520, and a display screen 1530. The processor 1510 may include one or more processing cores. The processor 1510 connects various parts within the terminal using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1520, and by calling data stored in the memory 1520. Optionally, the processor 1510 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1510 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content displayed on the screen 1530; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used for wireless communication. It is understood that the modem may also be implemented as a separate chip, rather than being integrated into the processor 1510.

[0218] The memory 1520 may include random access memory (RAM) or read-only memory. Optionally, the memory 1520 may include a non-transitory computer-readable storage medium. The memory 1520 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1520 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created according to the use of the terminal (such as audio data, phone book, etc.).

[0219] Display screen 1530 is a display component used to display a user interface. Optionally, the display screen 1530 is a touch-enabled display screen, through which users can use their fingers, styluses, or any suitable object to perform touch operations on the display screen 1530. Display screen 1530 is typically located on the front panel of the terminal. Display screen 1530 can be designed as a full-screen, curved screen, irregularly shaped screen, dual-sided screen, or foldable screen. Display screen 1530 can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen, etc., which are not limited in this embodiment.

[0220] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal may also include audio acquisition devices, speakers, radio frequency circuits, input units, sensors, audio circuits, Wireless Fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0221] This application provides a computer-readable storage medium storing at least one computer instruction, which is executed by a processor to implement the interface retrieval method as described in the above embodiments. Alternatively, this application provides a computer program product including computer instructions stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement the interface retrieval method as described in the above embodiments.

[0222] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0223] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

An interface retrieval method, the method comprising: In response to a search command, feature extraction is performed on the interface search conditions indicated by the search command to obtain interface search features; Based on the interface retrieval features, the feature library corresponding to the historical operation trajectory, the interface retrieval conditions, and the historical operation trajectory, the first target function information is determined. The historical operation trajectory is used to record the function information of at least one interface during the historical operation process. The feature library is used to store the features corresponding to the function information of the at least one interface. The function information is used to indicate the functions that the interface has. The first target interface to which the first target function information belongs is determined, and the first target interface is the interface retrieved based on the search instruction. According to the method of claim 1, determining the first target function information based on the interface retrieval features, the feature library corresponding to the historical operation trajectory, the interface retrieval conditions, and the historical operation trajectory includes: Based on the interface retrieval features and the feature library, at least one first functional information is determined, wherein the first functional information is information of the matching function retrieved based on the interface retrieval features and the feature library; Based on the interface search conditions and the functional information of the at least one interface, at least one second functional information is determined, wherein the second functional information is the information of the matching function retrieved based on the interface search conditions and the functional information of the at least one interface; The first target function information is determined based on the at least one first function information and the at least one second function information. According to the method of claim 2, determining at least one first functional information based on the interface retrieval features and the feature library includes: Based on the interface retrieval features, features matching the interface retrieval features are searched from the first feature library. The first feature library is used to store features corresponding to the functional information of at least one interface in the first historical operation trajectory. The first historical operation trajectory is the historical operation trajectory generated by the application installed on the terminal. If at least one first feature is found, at least one first functional information corresponding to the at least one first feature is determined. According to the method of claim 2, determining at least one first functional information based on the interface retrieval features and the feature library includes: A query request is sent to the server, the query request carrying the interface retrieval feature, so that the server searches for at least one first feature that matches the interface retrieval feature from the second feature library based on the interface retrieval feature, and sends at least one first function information corresponding to the at least one first feature to the terminal. The second feature library includes features corresponding to the function information of at least one interface in the second historical operation trajectory. The second historical operation trajectory is the historical operation trajectory generated by the terminal without installed applications. Receive at least one first functional information sent by the server. The method according to claim 4, further comprising: Determine the first target application to which the at least one first functional information belongs; If the first target application is an application that is not installed on the terminal, installation guidance information for the first target application is generated based on the first target application. The installation guidance information for the first target application is used to indicate the process of installing the first target application. The current screen displays the installation guide information for the first target application. The method according to claim 3, further comprising: Determine the functional information of at least one interface in the first historical operation trajectory. The functional information includes at least one of unit function information and interface function description information. The unit function information is used to indicate the smallest unit function of the interface, and the interface function description information is used to indicate the interface function of the interface. Feature extraction is performed on the functional information of the at least one interface to obtain the first feature library. The method according to claim 6, further comprising: The access permissions of the first feature library are set to be accessible, so that other services can access the first feature library to assist in the execution of the other service's tasks. The other service is a service in the terminal other than executing the search instruction, or the other service is a service in a third-party system. According to the method of claim 2, determining the first target function information based on the at least one first function information and the at least one second function information includes: The at least one first functional information and the at least one second functional information are merged and sorted; Based on the sequence of functional information obtained by sorting, the first target functional information is determined. According to the method of claim 8, the at least one first functional information is sorted sequentially based on the degree of matching with the interface retrieval features, and the at least one second functional information is sorted sequentially based on the degree of matching with the interface retrieval conditions. The merging and sorting of the at least one first functional information and the at least one second functional information includes: For any one of the at least one first functional information and the at least one second functional information, the weight of the functional information is determined based on the order of the functional information, and the weight of the functional information is negatively correlated with the order. The at least one first functional information and the at least one second functional information are sorted based on their respective weights. According to the method of claim 2, determining at least one second functional information based on the interface retrieval conditions and the functional information of the at least one interface includes: The text information corresponding to the search conditions on the interface is segmented into words to obtain multiple keywords; Determine the degree of matching between the plurality of keywords and the functional information of the at least one interface; Based on the matching degree between the plurality of keywords and the functional information of the at least one interface, at least one second functional information is determined from the functional information of the at least one interface, wherein the matching degree between the at least one functional information and the plurality of keywords satisfies the matching degree condition. The method according to any one of claims 1-10, the method further comprising: Based on the first target interface, the first target path information is determined from the historical operation trajectory. The historical operation trajectory is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation and the path information of the operation path corresponding to the interface during the historical operation. The first target operation path to which the first target path information belongs is the operation path to the first target interface. Based on the first target edge information included in the first target path information, the first target operation behavior indicated by the first target edge information is executed. The first target edge information includes information used to characterize the first target operation behavior and the interface jump situation before and after the first target operation behavior. The method according to claim 11, further comprising: Based on the first target edge information included in the first target path information, target tutorial information corresponding to the first target edge information is generated. The target tutorial information is used to guide the completion of the first target operation behavior indicated by the first target edge information. The target tutorial information is displayed on the current screen. According to the method of claim 12, generating target tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information includes: Based on the first target edge information included in the first target path information, first tutorial information corresponding to the first target edge information is generated. The first tutorial information is used to guide the execution of the first target operation behavior indicated by the first target edge information in text. Based on the first tutorial information, the target tutorial information is determined. According to the method of claim 13, determining the target tutorial information based on the first tutorial information includes: The first tutorial information is identified as the target tutorial information; or, Based on at least one of the functional information in the historical operation trajectory, the target edge information, and the first tutorial information, second tutorial information is obtained, and the second tutorial information is determined as the target tutorial information. The target tutorial information is a guide to perform the first target operation behavior indicated by the first target edge using graphic information. According to the method of claim 13, determining the target tutorial information based on the first tutorial information includes: Based on at least one of the functional information, the first target edge information, and the first tutorial information in the historical operation trajectory, second tutorial information is obtained. The second tutorial information is then converted into video format tutorial information to obtain the target tutorial information. The target tutorial information is a video guide for performing the first target operation behavior indicated by the first target edge. The method according to any one of claims 1-10, the method further comprising: Receive a voice command, the voice command being used to request the display of a second target interface; Based on the voice command, the second target path information is determined from the historical operation trajectory. The historical operation trajectory is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation and the path information of the operation path corresponding to the interface during the historical operation. The second target operation path to which the second target path information belongs is the operation path to reach the second target interface. Based on the second target edge information included in the second target path information, the second target operation behavior indicated by the second target edge information is executed. The second target edge information includes information used to characterize the second target operation behavior and the interface jump situation before and after the second target operation behavior. An interface retrieval device, the device comprising: The first processing module is used to extract features from the interface search conditions indicated by the search command in response to the search command, and obtain interface search features. The first determining module is used to determine first target function information based on the interface retrieval features, the feature library corresponding to the historical operation trajectory, the interface retrieval conditions, and the historical operation trajectory. The historical operation trajectory is used to record the function information of at least one interface during the historical operation process. The feature library is used to store the features corresponding to the function information of the at least one interface. The function information is used to indicate the functions that the interface has. The second determining module is used to determine the first target interface to which the first target function information belongs, wherein the first target interface is the interface retrieved based on the search instruction. A terminal includes a processor and a memory, the memory storing at least one computer instruction, the at least one computer instruction being loaded and executed by the processor to implement the interface retrieval method as described in any one of claims 1 to 16. A computer-readable storage medium storing at least one computer instruction for execution by a processor to implement the interface retrieval method as described in any one of claims 1 to 16. A computer program product includes computer instructions stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement the interface retrieval method as described in any one of claims 1 to 16.

Images