Interface retrieval method and device, terminal, storage medium and product

Abstract

The embodiment of the invention discloses an interface retrieval method and device, a terminal, a storage medium and a product, and belongs to the technical field of artificial intelligence. The method comprises the following steps: in response to a retrieval instruction, performing feature extraction on an interface retrieval condition indicated by the retrieval instruction to obtain an interface retrieval feature; based on the interface retrieval features, a feature library corresponding to historical operation tracks, the interface retrieval conditions and function information of at least one interface in the historical operation tracks, first target function information is determined, and the historical operation tracks are used for recording the function information of at least one interface in the historical operation process; the feature library is used for storing features corresponding to function information of the at least one interface, and the function information is used for indicating functions of the interface; and determining a first target interface to which the first target function information belongs, wherein the first target interface is an interface retrieved based on the retrieval instruction.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to an interface retrieval method, device, terminal, storage medium and product. Background Art

[0002] Global search, as a commonly used interface search method for users, is generally divided into in-application search and system-level search; among them, in-application search is implemented by different applications themselves, and different applications implement their own search capabilities; system-level search is implemented by system applications and provided by terminal manufacturers. Summary of the invention

[0003] The embodiment of the present application provides an interface search method, device, terminal, storage medium and product. The technical solution is as follows:

[0004] On the one hand, an embodiment of the present application provides an interface retrieval method, the method comprising:

[0005] In response to a retrieval instruction, feature extraction is performed on the interface retrieval condition indicated by the retrieval instruction to obtain an interface retrieval feature; based on the interface retrieval feature, a feature library corresponding to the historical operation track, the interface retrieval condition and the historical operation track, first target function information is determined, the historical operation track is used to record function information of at least one interface during the historical operation process, the feature library is used to store features corresponding to the function information of the at least one interface, and the function information is used to indicate the function of the interface; a first target interface to which the first target function information belongs is determined, the first target interface being the interface retrieved based on the retrieval instruction.

[0006] On the other hand, an embodiment of the present application provides an interface retrieval device, which includes: a first processing module, used to respond to a retrieval instruction, perform feature extraction on the interface retrieval condition indicated by the retrieval instruction, and obtain an interface retrieval feature; a first determination module, used to determine first target function information based on the interface retrieval feature, a feature library corresponding to the historical operation trajectory, the interface retrieval condition and the historical operation trajectory, the historical operation trajectory being used to record the function information of at least one interface during the historical operation process, the feature library being used to store features corresponding to the function information of the at least one interface, and the function information being used to indicate the function of the interface; a second determination module, used to determine a first target interface to which the first target function information belongs, the first target interface being the interface retrieved based on the retrieval instruction.

[0007] On the other hand, an embodiment of the present application provides a terminal, the terminal includes a processor and a memory, the memory stores at least one computer instruction, the at least one computer instruction is loaded and executed by the processor to implement the interface retrieval method as described in the above aspects. On the other hand, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores at least one computer instruction, the at least one computer instruction is used to be executed by the processor to implement the interface retrieval method as described in the above aspects. On the other hand, an embodiment of the present application provides a computer program product, the computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium; the processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the interface retrieval method as described in the above aspects.

[0008] In an embodiment of the present application, one-way recall retrieval is performed based on interface retrieval features and a feature library, and one-way recall retrieval is performed based on interface retrieval conditions and functional information of the interface. The first target interface that the user wants to retrieve is comprehensively determined based on the two-way recall retrieval results, thereby realizing two-way recall retrieval to retrieve the first target interface, thereby improving the accuracy of the retrieved first target interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of an operation process corresponding to a historical operation track shown in an exemplary embodiment of the present application;

[0010] Figure 2 is a flowchart of an interface retrieval method shown in an exemplary embodiment of the present application;

[0011] Figure 3 is a schematic diagram of an interface retrieval method shown in an exemplary embodiment of the present application;

[0012] Figure 4 is a schematic diagram of unit function information related to an application shown in an exemplary embodiment of the present application;

[0013] Figure 5 is a flowchart of an interface retrieval method shown in an exemplary embodiment of the present application;

[0014] Figure 6 is a schematic diagram of installation guide information shown in an exemplary embodiment of the present application;

[0015] Figure 7 is a flowchart of an interface retrieval method shown in an exemplary embodiment of the present application;

[0016] Figure 8 is an interface diagram of a process involved in an interface retrieval method shown in an exemplary embodiment of the present application;

[0017] Fig. 9 is a schematic diagram of first target function information shown in an exemplary embodiment of the present application;

[0018] Fig.10 is a schematic diagram of first target path information shown in an exemplary embodiment of the present application;

[0019] Fig.11 is an interface diagram of an interface related to first target path information shown in an exemplary embodiment of the present application;

[0020] Fig.12 is a schematic diagram of target tutorial information shown in an exemplary embodiment of the present application;

[0021] Fig.13 It is a structural block diagram of the overall solution shown in an exemplary embodiment of the present application;

[0022] Fig.14 is a structural block diagram of an exemplary embodiment of the present application showing an interface retrieval device;

[0023] Fig.15 It is a structural block diagram of a terminal shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0025] The "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the historical operation tracks and voice commands involved in this application are all obtained with full authorization.

[0026] Please refer to Figure 1In the embodiment of the present application, a historical operation trajectory (Trajectory Structure) is used to record the operation process. The historical operation trajectory 13 is composed of nodes 131, behavior edges 132 and operation paths 133. Among them, the node 131 corresponds to the interface 11 passed during the operation process; the behavior edge 132 corresponds to the operation behavior in the interface, which is an edge 12 used to connect different nodes (interface switching occurs after the operation) or connect the same node (no interface switching occurs after the operation); the operation path 133 corresponds to the operation behavior sequence. In addition, in order to ensure the accuracy of the operation process record, the node information 141 of the node 131 represents the interface features of the interface from the functional dimension to ensure the accuracy of subsequent interface retrieval and positioning; for example, the node information 141 can be functional information, and the functional information is used to indicate the functions of the interface. Accordingly, the historical operation trajectory 13 is used to record the functional information of the interface during the historical operation process. The edge information 142 of the behavior edge 132 indicates the operation behavior and the node jumps before and after the operation behavior, ensuring the accurate restoration of the operation in the subsequent interface; the path information 143 of the operation path 133 represents the execution sequence between different operation behaviors, ensuring that complex operations are executed in sequence.

[0027] Compared with screenshots and videos, historical operation trajectories have lower data volumes and can be automatically generated without manual triggering by the user. Moreover, they can be automatically backtracked and reproduced based on node information, edge information, and path information, improving the efficiency of operation backtracking and reproduction.

[0028] Functional information: The interface is a visual user interface, such as a desktop, an application interface, a system interface, etc. The interface in this embodiment is not limited to the visible range of the screen, but includes the completed interface within and outside the visible range of the screen. Accordingly, before and after the interface content is scrolled, even if the displayed content within the visible range of the screen changes significantly, the interface before and after the scrolling will not be judged as a different interface.

[0029] In the embodiments of the present application, different interfaces refer to interfaces with significant differences in interface layout, content, and style. Usually, when an interface switch occurs, the interfaces before and after the switch are determined to be different interfaces. For example, when text or images in an interface change dynamically, or when list elements in an interface change, they are not determined to be different interfaces; when clicking on a control in an interface to trigger an interface jump, the interfaces before and after the jump are determined to be different interfaces.

[0030] The function information includes at least one of unit function information and interface function description information. The unit function information is used to indicate the minimum unit function of the interface, and the interface function description information is used to indicate the interface function of the interface. In a possible implementation, at least one of the unit function information and the interface function description information of at least one interface is determined by a large language model.

[0031] Edge information: Edge information contains information used to characterize the operation behavior and the interface jump situation before and after the operation behavior. Among them, the behavior edge (Edge) corresponding to the operation behavior is a directed edge, and the direction of the behavior edge is the interface jump direction, that is, the behavior edge points from the interface before the jump to the interface after the jump. Optionally, the operation behavior refers to a manual operation behavior of the user, such as a trigger operation on a control in the interface; or, the operation behavior refers to a system behavior, such as an automatic interface jump operation after reaching a preset time. In some embodiments, a plurality of operation behavior types are pre-set, and the edge information of the behavior edge includes the operation behavior type to which the operation behavior belongs. Since it is necessary to know what operation is performed in the interface and the changes in the interface before and after the operation is performed when the operation behavior is reproduced later, in order to ensure the accuracy of the operation behavior reproduction, the edge information corresponding to the operation behavior has at least the ability to indicate the operation behavior and the interface before and after the operation behavior is performed.

[0032] In some embodiments, the edge information includes at least: behavior information of the operation behavior, a starting node identifier (fromNodeID) of the starting node corresponding to the starting interface before the operation behavior, a terminating node identifier (toNodeID) of the terminating node corresponding to the terminating interface after the operation behavior, and an edge identifier (similar to a unique identifier of a node identifier and a node element identifier). Accordingly, when determining the edge information of the behavior edge corresponding to the operation behavior in the interface during the operation process, the behavior information of the operation behavior in the interface during the operation process is determined, and the starting node identifier of the starting node corresponding to the starting interface before the operation behavior and the terminating node identifier of the terminating interface after the operation behavior are determined, so that the behavior information, the starting node identifier and the terminating node identifier are determined as the edge information of the behavior edge corresponding to the operation behavior.

[0033] Among them, when the interface before and after the operation behavior does not switch, the start node identifier and the end node identifier in the side information corresponding to the operation behavior are the same; when the interface before and after the operation behavior switches, the start node identifier and the end node identifier in the side information corresponding to the operation behavior are different. As an important content in the side information, the behavior information needs to be able to clearly indicate the operation behavior. In a possible implementation, the behavior information includes the action type (ActionType) of the operation behavior.

[0034] In some embodiments, a common action class (CommonAction) is defined, which includes all operation behaviors of users and systems to ensure the accuracy of recorded operation behaviors. In addition, the behavior type of each operation behavior belongs to the behavior type class. Optionally, the behavior type class is an enumeration class. In an illustrative example, taking the common operations in a mobile phone as an example, the behavior type class may include at least one of the following behavior types: GoHome (return to home page), RecentApps (recent applications), GoBack (return), StartApp (start application), Click (click), DoubleClick (double-click), LongClick (long press), InputText (input text), Scroll (scroll), Concurrent (cooperative operation, i.e., operation behavior occurring at the same time), Continuous (continuous operation, referring to operation behavior performed in sequence), Stop (termination), Wait (waiting, the scene where the interface changes automatically over time), Unknown (unknown behavior, as a bottom-up behavior).

[0035] Since some operation behaviors are directed to interface controls within the interface, in order to ensure that the subsequent operation can determine which interface control to operate on based on the behavior information, in some embodiments, when the behavior type of the operation behavior is determined as behavior information, if the operation behavior is related to the interface control within the interface, the node element identifier of the node element corresponding to the interface control is also determined as the behavior information.

[0036] For example, when the operation behavior is a click operation, the behavior information corresponding to the operation behavior includes the operation type "Click" and the node element identifier of the node element corresponding to the clicked button; for another example, when the operation behavior is a text input operation, the behavior information corresponding to the operation behavior includes the operation type "InputText" and the node element identifier of the node element corresponding to the text input box where the text is entered.

[0037] In addition, since the operation process may involve a series of continuous operation behaviors, in order to facilitate the tracing back of the previous operation behavior before the current operation behavior, in a possible implementation, when there is a previous operation in the operation behavior, the operation information of the previous operation is determined as the edge information of the behavior edge corresponding to the operation behavior, that is, the edge information also includes the previous operation information (preAction). Schematically, for the operation behavior of clicking the login button, its previous operation may include entering an account in the first text box and entering a password in the second text box. Optionally, the previous operation information can be identified in the form of a list, each item in the list corresponds to a previous operation, and the previous operation can be represented by the jump between nodes, such as the starting node identifier → the ending node identifier (the starting node identifier and the ending node identifier can be the same).

[0038] In this embodiment, the starting node identifier of the starting node before the operation behavior, the ending node identifier of the ending node after the operation behavior, and the behavior information of the operation behavior are used as side information, thereby improving the accuracy of the operation behavior record; and the behavior information includes the behavior type of the operation behavior, and contains the node element identifier of the targeted interface control, thereby ensuring the accuracy of subsequent restoration of the interface control operation.

[0039] Path information: Since a series of operations may need to be performed during the historical operation process, and jumps between multiple interfaces are required, it is necessary to sort the behavior edges corresponding to the operation behaviors according to the execution sequence between the operation behaviors to obtain the operation path (Path). Among them, the operation path is a directed path composed of multiple nodes and behavior edges between nodes, and different nodes correspond to different interfaces. Accordingly, the path information of the operation path is composed of edge information, and can indicate the order relationship between the edge information and the behavior edge.

[0040] In some embodiments, after determining the function information, edge information, and path information, a historical operation track is generated based on the function information, edge information, and path information. Based on the function information of the interface involved in the historical operation process, the edge information of the behavior edge involved, and the path information of the operation path involved, a historical operation track corresponding to the historical operation process is generated. In some embodiments, the generated historical operation track is stored so that the interface can be retrieved, located, and the operation can be reproduced based on the historical operation track later.

[0041] In some embodiments, the path information of the operation path includes at least a path identifier and a behavior edge list. In a possible implementation, when determining the path information of the operation path corresponding to the operation process based on the edge information, the path identifier of the operation path corresponding to the operation process is determined, and the behavior edge list is determined based on the edge information, so that the path identifier and the behavior edge list are determined as the path information of the operation path. Similar to the node identifier, the node element identifier and the behavior edge identifier, the path identifier is a unique identifier assigned to the operation path (with no practical meaning), which can be represented in the form of a string, wherein the path identifiers of different operation paths are different. The behavior edge list contains a series of behavior edges corresponding to the operation behaviors, and the behavior edges are arranged in the time sequence of the operation behaviors. For example, the behavior edge at the head of the list in the behavior edge list corresponds to the starting operation behavior, and the behavior edge at the end of the list corresponds to the ending operation behavior.

[0042] In some embodiments, the behavior edge list includes the behavior edge identifier in the edge information corresponding to the behavior edge. Since the edge information of the behavior edge can indicate the nodes (interfaces) involved before and after the operation behavior and the behavior information of the executed operation behavior, the edge information of the behavior edge arranged in time sequence in the behavior edge list can accurately describe the execution order of a series of operation behaviors and what kind of operation behavior is executed in the interface. In some embodiments, in order to facilitate the subsequent operation path retrieval based on the functions implemented by a series of operations, the path information also includes a path function (pathFunction). Accordingly, the path function of the operation path is determined, and the path function is determined as the path information of the operation path.

[0043] Regarding the method of determining the path function, in one possible implementation, the path function can be generated based on the interface elements in the interface (such as interface text, the package name of the application to which the interface belongs) and the text content corresponding to the interface controls triggered during the operation. For example, the application package name of the interface involved in the operation process is (com.android.setting), that is, the system setting application, and the text content corresponding to the interface controls triggered during the operation is "Display and Brightness" and "Screen Color Mode" respectively, and the generated path function is "Set Screen Color Mode in System Applications".

[0044] In another possible implementation, the interface image of the interface involved in the operation process can be input into the large language model, and the path function can be generated by the large language model. This embodiment does not limit the specific method of generating the path function. In this embodiment, a behavior edge list is generated based on the behavior edges corresponding to the operation behaviors involved in the operation process, and the behavior edge list is used as the path information of the operation path, which ensures the accuracy of subsequent restoration of a series of operations based on the path information; and the path function of the operation path is used as part of the path information, which is conducive to the subsequent operation path retrieval based on the functions required to be implemented by the operation, expanding the application scenarios of historical operation tracks.

[0045] In the embodiment of the present application, feature extraction is performed on the functional information of at least one interface in the historical operation track to obtain a feature library. In the embodiment of the present application, when searching for an interface, a one-way recall search is performed based on the feature library, and a one-way recall search is performed based on the functional information of at least one interface, thereby realizing the retrieval of the first target interface based on the two-way recall search.

[0046] In a possible implementation, the historical operation track is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation process and the path information of the operation path corresponding to the interface during the historical operation process. The edge information is used to characterize the operation behavior and the information of the interface jump before and after the operation behavior, and the path information is used to characterize the timing between the operation behaviors. After the first target interface is retrieved, the first target path information can be determined from the historical operation track. The first target operation path to which the first target path information belongs is the operation path to reach the first target interface. Based on the first target edge information included in the first target operation path, the first target edge information includes information used to characterize the first target operation behavior and the interface jump before and after the first target operation behavior, the first target operation behavior indicated by the first target edge information is executed, thereby jumping to the first target interface, and realizing the automatic execution and jump of the first target interface.

[0047] The interface search method provided in the embodiment of the present application can be executed by a terminal with a search function. The terminal can be a smart phone, a tablet computer, a smart wearable device, a personal computer, etc. Figure 2 , which shows a flowchart of an interface retrieval method provided by an exemplary embodiment of the present application. This embodiment is described by taking the method used in a terminal as an example, and the method may include the following steps:

[0048] Step 201: In response to a search instruction, feature extraction is performed on the interface search condition indicated by the search instruction to obtain interface search features.

[0049] The interface search feature may be any feature that can characterize the interface search condition; for example, the interface search feature is an interface search vector; then this step may be: in response to the search instruction, the interface search condition indicated by the search instruction is vectorized to obtain the interface search vector. The interface search vector is used to indicate the key information in the interface search condition; for example, if the interface search condition is "I want to open and view the chat records containing milk tea", the interface search vector may be "milk tea" and "chat records"; for another example, if the interface search condition is "I want to buy a skirt", the interface search vector may be "buy" and "skirt"; for another example, if the interface search condition is "I want to view the alarm clock at five o'clock", the interface search vector may be "five o'clock" and "alarm clock"; for another example, if the interface search condition is "I want to view the payment record of 500 yuan", the interface search vector may be "500 yuan" and "payment record", etc. The interface search condition indicated by the search instruction may be voice or text. For example, the interface search condition is a text describing the interface function; illustratively, the interface search condition is the text "I want to turn on the care mode". For another example, the interface search condition is a voice that describes the interface function; illustratively, the interface search condition is a voice that says "I want to turn on the care mode".

[0050] Step 202: Determine the first target function information based on the interface retrieval features, the feature library corresponding to the historical operation trajectory, the interface retrieval conditions and the historical operation trajectory. The historical operation trajectory is used to record the function information of at least one interface during the historical operation process. The feature library is used to store features corresponding to the function information of at least one interface. The function information is used to indicate the functions of the interface.

[0051] The embodiment of the present application adopts a multi-path search method to perform interface search, one of which is to perform interface search based on interface search features and a feature library, and the other is to perform interface search based on interface search conditions and functional information of at least one interface; accordingly, this step can be implemented by the following steps (1) to (3), including:

[0052] (1) Based on the interface retrieval feature and the feature library, at least one first functional information is determined. The first functional information is the information of the matching function retrieved based on the interface retrieval feature and the feature library. The feature library is used to store features corresponding to the functional information of at least one interface in the historical operation track. The historical operation track is used to record the functional information of the interface during the historical operation process. The functional information is used to indicate the functions of the interface.

[0053] The function vector obtained by vectorizing the function information of the interface using the features corresponding to the function information of the interface. The function vector of the interface is used to indicate the key information corresponding to the interface function; for example, if the interface is a payment interface, the function vector of the interface may be the amount information included in the payment interface; for another example, if the interface is an application interface, the function vector of the interface may be the application information of the application interface; for another example, if the interface is an alarm clock interface, the function vector of the interface may be the time information in the alarm clock interface; for another example, if the interface is a takeaway interface, the function vector of the interface may be the order status information in the takeaway interface; for another example, if the interface is a chat interface of a social application, the function vector of the interface may be the chat history in the chat interface, etc.

[0054] The degree of match between the feature corresponding to at least one first functional information and the interface retrieval feature satisfies the first matching condition, and the first matching condition may be to select a first preset number of features with the greatest matching degree or to select features with a matching degree exceeding the first preset matching degree. In one possible implementation, the terminal determines the degree of match between the interface retrieval feature and each feature in the feature library; based on the degree of match between the interface retrieval feature and each feature, determines from the feature library at least one feature whose matching degree satisfies the first matching condition, determines the functional information corresponding to at least one feature, and obtains at least one first functional information. For example, based on the degree of match between the interface retrieval feature and each feature, the terminal determines from the feature library the four features with the greatest matching degree, and determines the first functional information corresponding to these four features.

[0055] Function information includes at least one of unit function information and interface function description information; unit function information is used to indicate the minimum unit function of the interface, and the minimum unit function refers to an independent, basic, and combinable technology or functional module in the terminal application scenario, and the minimum unit function is used to assist users in completing one or more small tasks of complex tasks. In addition, the minimum unit function has the characteristics of being small, precise, efficient, and scalable. For example, turning on Bluetooth, turning off Bluetooth, clearing all notification messages, muting the phone, taking a photo, etc. on the terminal can all be considered as the minimum unit function. Among them, the minimum unit function is the basic unit that constitutes the application function of the terminal. By combining and expanding these minimum unit functions, a more complex, efficient, and user-friendly application experience can be formed. Exemplarily, a function that belongs to the minimum unit function needs to meet at least one of independence, basicity, combinability, scalability, accurate description, induction and summarization, and redundancy removal.

[0056] Independence means that the minimum unit function has strong autonomy and can independently realize a function without relying on other minimum unit functions. Basicity means that the minimum unit function is universal and can play a role in multiple scenarios or multiple applications, and many complex functions are built on the basis of these minimum unit functions. Combinability means that the minimum unit function can be combined with other minimum unit functions to jointly build more complex functions, which can meet the needs of users in different task scenarios. Easy scalability means that the minimum unit function can be expanded or upgraded to continuously improve its performance and characteristics to adapt to the ever-changing technical environment and user needs. Accurate description means that the description of the minimum unit function generally uses accurate descriptions; for example, open **, edit **, view **, enter **, adjust ** or change **. Inductive summary means that for some list items with similar content, such as contact lists or chat lists, each list has similar behaviors. Therefore, atomic capabilities can be summarized as descriptions such as "send a message to **" and "view chat records with **". De-redundancy means that functions such as returning to the previous level, going to the previous level, and returning to the homepage are not minimum unit functions.

[0057] The interface function description information is used to indicate the interface functions of the interface; illustratively, the interface function description information can summarize the functional role of the interface in one sentence, and the interface function description information is general and general; and the interface function description information will not change with the dynamic changes of the interface; for example, when the content in the input box in the interface changes dynamically, the interface function description information does not include the content in the input box.

[0058] In one possible implementation, after the terminal determines the first feature library, it can also open the first feature library for use by other services to meet the needs of other services; accordingly, the process can be: setting the access rights of the first feature library to accessible, so that other services can assist in performing tasks of other services by accessing the first feature library, and other services are services in the terminal other than executing retrieval instructions, or other services are services in a third-party system. For example, other services can be intelligent assistants of the terminal, so that other services can automatically complete tasks with the help of the first feature library. For another example, other services can be services in group buying applications or services in short video applications, so that group buying applications or short video applications can provide better services to users with the help of the first feature library.

[0059] In another possible implementation, the terminal may also generate an executable script corresponding to the features included in the first feature library based on the first feature library. For example, the first feature library includes features corresponding to unit function information (which may be referred to as features of the minimum unit function), and the terminal generates an executable script corresponding to the features of the minimum unit function. In addition, the terminal may also store the application to which the features of the minimum unit function belong and the application status of the application to which they belong, and the application status may be installed or not installed; for example, please refer to Table 1, for example, please refer to Table 1, the feature library includes features of three minimum unit functions, the numbers of the features of the three minimum unit functions may be F01, F02 and F03, and the unit function information corresponding to the features of the three minimum unit functions are: viewing device memory, turning on care mode and viewing current altitude; viewing device memory, turning on care mode and viewing current altitude respectively belong to system settings (one-party application), social applications (three-party application) and map applications (three-party application), and the application status of system settings, social applications and map applications are installed, installed and not installed respectively; the executable scripts corresponding to viewing device memory, turning on care mode and viewing current altitude are executable script 1, executable script 2 and executable script 3.

[0060] Table 1

[0061] serial number Unit function information Application Executable scripts Application Status F01 View device memory System Settings (One-Side Application) Executable script 1 Installed F02 Turn on caring mode Social applications (third-party applications) Executable script 2 Installed F03 View current altitude Map application (third-party application) Executable script 3 Not Installed

[0062] Before step 202, the terminal needs to generate a historical operation track first, and then generate a feature library based on the historical operation track. The process of the terminal generating a feature library based on the historical operation track can be: the terminal determines the functional information of at least one interface in the historical operation track, and the functional information includes at least one of the unit functional information and the interface functional description information; the functional information of at least one interface is subjected to feature extraction to obtain the feature library. In a possible implementation, the step of the terminal determining the functional information of at least one interface in the historical operation track can be: the terminal determines at least one of the unit functional information and the interface functional description information of at least one interface through a large language model.

[0063] It should be noted that, since the historical operation track includes the first historical operation track and the second historical operation track, the first historical operation track is the historical operation track corresponding to the historical operation record generated in the terminal installed application (one-party application and / or three-party application); the second historical operation track is the historical operation track corresponding to the historical operation record generated in the terminal not installed application (which can be the historical operation record of other terminals). Therefore, the feature library generated in this step can be the first feature library corresponding to the first historical operation track, and the server generates the second feature library corresponding to the second historical operation track. How to match the first function information through the first feature library and the second feature library will be described in detail in the subsequent embodiments.

[0064] Please refer to Figure 3 , in the preprocessing stage: the terminal generates a historical operation track, the historical operation track includes a first historical operation track and a second historical operation track, the first historical operation track is the historical operation track corresponding to the historical operation record generated in the terminal's installed applications (one-party applications and / or three-party applications), the second historical operation track is the historical operation track corresponding to the historical operation record generated in the terminal's non-installed applications (which can be the historical operation record of other terminals), the second historical operation track can be generated by the server and then deployed to the terminal; or, the second historical operation track can be directly deployed in the server. After the terminal generates the historical operation track, the terminal determines the interface function description information and unit function information of at least one interface in the historical operation track through a large language model; then the terminal extracts features from the interface function description information and the unit function information to obtain a feature library, so as to facilitate the subsequent matching of function information based on the feature library.

[0065] In a possible implementation, after the terminal determines the interface function description information and unit function information of at least one interface in the historical operation track, it generates a function list corresponding to each interface, and the function list corresponding to the interface includes the interface function description information and unit function information of the interface, so as to facilitate the subsequent determination of the interface to which the function information belongs based on the function list. For example, please refer to Figure 4, the terminal determines that the function list of the social application includes "view mobile phone number", "view login password", "view voice lock", "view emergency contacts", "view logged-in devices", "view more security settings", "view security center" and "view personal information and permissions". Among them, the unit function information includes "view care mode description", "view care mode function" and "turn on care mode". The terminal extracts the feature library obtained by the unit function information of the interface, including feature 1, feature 2, ... feature M, where M is an integer greater than 2.

[0066] (2) Based on the interface search condition and the functional information of at least one interface, at least one second functional information is determined, where the second functional information is information of a matching function retrieved based on the interface search condition and the functional information of at least one interface.

[0067] The matching degree between the feature corresponding to at least one second functional information and the functional information of at least one interface satisfies the second matching degree condition, and the second matching degree condition may be selecting the features of the second preset number with the largest matching degree or selecting the features with matching degree exceeding the second preset matching degree. An interface includes at least one functional information; for example, an interface includes multiple unit functional information and one interface functional description information.

[0068] In a possible implementation, the terminal performs word segmentation on the text information corresponding to the interface search condition to obtain multiple keywords; determines the matching degree between the multiple keywords and the functional information of at least one interface; and determines at least one second functional information from the functional information of at least one interface based on the matching degree between the multiple keywords and the functional information of at least one interface, and the matching degree between the at least one functional information and the multiple keywords satisfies the second matching degree condition. For example, the terminal determines the matching degree between the multiple keywords and the functional information of at least one interface through the bm25 function, and determines at least one second functional information from the functional information of at least one interface based on the matching degree between the multiple keywords and the functional information of at least one interface. The bm25 function is a ranking function in information retrieval, which is used to evaluate the relevance of text information under a given search query; before using the bm25 function, the text information needs to be word segmented to convert the interface search condition into multiple keywords. For example, if the interface search condition is "I want to turn on the Wifi switch", the terminal performs word segmentation on the interface search condition to obtain the word segmentation result of ["I", "want", "turn on", "Wifi", "switch"], and then uses the bm25 function to match the keywords in the interface search condition with the functional information of at least one interface to obtain at least one second functional information.

[0069] (3) Determine first target function information based on at least one first function information and at least one second function information.

[0070] This step can be achieved by following the steps (3-1) to (3-2), including:

[0071] (3-1) At least one first function information and at least one second function information are combined and sorted.

[0072] At least one first function information and at least one second function information are merged and sorted in descending order based on weight. In a possible implementation, at least one first function information is sorted in sequence based on the degree of matching with the interface retrieval feature, and at least one second function information is sorted in sequence based on the degree of matching with the interface retrieval condition; accordingly, the step of merging and sorting at least one first function information and at least one second function information by the terminal may be: for any one of the at least one first function information and at least one second function information, the terminal determines the weight of the function information based on the order of the function information, and the weight of the function information is negatively correlated with the order; based on the weight of the at least one first function information and the weight of the at least one second function information, the at least one first function information and the at least one second function information are sorted.

[0073] For the convenience of description, at least one first functional information is sequentially sorted based on the degree of matching with the interface search feature and is called the first search result, and at least one second functional information is sequentially sorted based on the degree of matching with the interface search condition and is called the second search result; accordingly, the step of determining the weight of the functional information based on the order of the functional information by the terminal may be: when the functional information is in the first search result but not in the second search result, the terminal determines the weight of the functional information based on the order of the functional information in the first search result; when the functional information is in the second search result but not in the first search result, the terminal determines the weight of the functional information based on the order of the functional information in the second search result; when the functional information is in the first search result and in the second search result, the terminal determines the weight of the functional information based on the order of the functional information in the first search result and the order of the functional information in the second search result. For example, the terminal determines the first weight of the functional information based on the order of the functional information in the first search result, determines the second weight of the functional information based on the order of the functional information in the second search result, and determines the weight of the functional information by weighted sum of the first weight and the second weight.

[0074] For example, the first search result is A(1 / 2), B(1 / 4), C(1 / 8), D(1 / 16); the second search result is D(1 / 2), B(1 / 4), A(1 / 8), C(1 / 16); then the weight of A is 1 / 2+1 / 8=5 / 8; the weight of B is 1 / 4+1 / 4=1 / 2; the weight of D is 1 / 16+1 / 2=7 / 16; the weight of C is 1 / 8+1 / 16=3 / 16; therefore, the terminal combines and sorts A, B, C and D to obtain the functional information sequence A, B, D, C.

[0075] (3-2) Based on the sorted function information sequence, determine the first target function information.

[0076] In one possible implementation, the terminal automatically determines the first target function information based on the weight of the function information. In response, step (3-2) may be: the terminal determines the function information with the largest weight in the function information sequence as the first target function information based on the sorted function information sequence; for example, the function information sequence is sorted in descending order based on the weight, and the terminal determines the first function information in the function information sequence as the first target function information. In another possible implementation, after the terminal sorts the function information sequence, the user selects the first target function information; accordingly, step (3-2) may be: the terminal displays the sorted function information sequence, determines the function information selected in the function information sequence, and determines the selected function information as the first target function information.

[0077] In the embodiment of the present application, since function retrieval is highly real-time, a combination strategy of two-way recall (recalling function information based on interface retrieval features and interface retrieval conditions respectively) + user selection (user manually selects function information) is adopted to determine the first target function information, thereby improving the accuracy of the determined first target function information.

[0078] Step 203: Determine the first target interface to which the first target function information belongs.

[0079] After the terminal determines the interface function description information and unit function information of at least one interface in the historical operation trajectory, it generates a function list corresponding to each interface, and the function list corresponding to the interface includes the interface function description information and unit function information of the interface; in this step, the terminal determines the function list where the first target function information is located, and determines the first target interface to which the function list belongs.

[0080] For example, please refer to Figure 3In the recall stage: the terminal extracts features from the interface retrieval conditions to obtain interface retrieval features (embedding), performs a recall based on the interface retrieval features and the feature library to obtain a first retrieval result; performs another recall (sql-like) based on the interface retrieval conditions and the functional information of the interface to obtain a second retrieval result, sorts the first retrieval result and the second retrieval result based on the sorting (bm25) function to obtain a functional information sequence (ReRank), and determines the first target interface based on the functional information sequence.

[0081] In the embodiment of the present application, a recall search is performed based on the interface search features and feature library, a recall search is performed based on the interface search conditions and the functional information of the interface, and the first target interface that the user wants to search is comprehensively determined based on the two-way recall search results, thereby realizing two-way recall search to search for the first target interface, thereby improving the accuracy of the retrieved first target interface. In addition, in the embodiment of the present application, interface search can be realized based on historical operation tracks, and the search engines for in-application search and system-level search need to be configured one by one, thereby reducing the cost of interface search.

[0082] Please refer to Figure 5 , which shows a flowchart of an interface retrieval method provided by an exemplary embodiment of the present application. This embodiment is described by taking the method used in a terminal as an example, and the method may include the following steps:

[0083] Step 501: In response to a search instruction, feature extraction is performed on the interface search condition indicated by the search instruction to obtain interface search features.

[0084] In some embodiments, this step is the same as step 201 and will not be repeated here.

[0085] In one possible implementation, after the terminal executes step 501, it may execute steps 502-503 or steps 504-505; the terminal may also execute step 502 first, and if the first feature is found based on step 502, execute step 503; if the first feature is not found, execute step 504; the terminal may also execute steps 502-503 first, search for the first feature on the terminal side, and then execute steps 504-505, search for the first feature on the service side.

[0086] Step 502: Based on the interface retrieval feature, search for a feature matching the interface retrieval feature from a first feature library, wherein the first function information is information of a matching function retrieved based on the interface retrieval feature and the feature library.

[0087] The first feature library is used to store features corresponding to the function information of at least one interface in the first historical operation track, and the first historical operation track is a historical operation track corresponding to the historical operation record generated in the terminal installed application. After receiving the search instruction, the terminal preferentially searches for features matching the interface search features from the first feature library; if at least one first feature is found, step 503 is executed; if no feature is found, step 504 is executed.

[0088] In a possible implementation, the terminal determines the degree of match between the interface retrieval feature and each feature in the first feature library; based on the degree of match between the interface retrieval feature and each feature in the first feature library, at least one first feature whose degree of match satisfies a third degree of match condition is determined from the first feature library. The third degree of match condition may be selecting a third preset number of features with the greatest degree of match or selecting features with a degree of match exceeding a third preset degree of match.

[0089] Step 503: When at least one first feature is found, determine at least one first functional information corresponding to the at least one first feature.

[0090] The terminal stores the correspondence between the features and the functional information in advance; accordingly, the terminal determines at least one functional information corresponding to the at least one first feature from the correspondence between the features and the functional information based on the at least one first feature.

[0091] Step 504: Send a query request to the server.

[0092] The query request carries the interface retrieval feature, so that the server searches for at least one first feature matching the interface retrieval feature from the second feature library based on the interface retrieval feature, and sends at least one first function information corresponding to the at least one first feature to the terminal, the second feature library includes features corresponding to the function information of at least one interface in the second historical operation track, and the second historical operation track is a historical operation track corresponding to the historical operation record generated in the terminal's non-installed application. The process of the server determining at least one first feature from the second feature library based on the interface retrieval feature is similar to the process of the terminal determining at least one first feature from the first feature library based on the interface retrieval feature, and will not be repeated here.

[0093] Step 505: Receive at least one first function information sent by the server.

[0094] In a possible implementation, the terminal may also obtain the second feature library and the correspondence between the features in the second feature library and the function information from the server in advance, and then store the second feature library and the correspondence between the features and the function information locally; accordingly, steps 504-505 may be replaced by: when no feature matching the interface retrieval feature is found, the terminal searches the second feature library for a feature matching the interface retrieval feature based on the interface retrieval feature to obtain at least one first feature, and then determines at least one function information corresponding to the at least one first feature from the correspondence between the feature and the function information based on the at least one first feature.

[0095] In the embodiment of the present application, since the second feature library is a feature corresponding to the function information of the interface in the terminal's non-installed application, searching for the function information based on the second feature library can realize retrieval of the target interface in the non-installed application.

[0096] In a possible implementation, after the terminal finds at least one first function information based on the second feature library, it can also guide the user to install at least one first target application to which the first function information belongs. The process can be: the terminal determines the first target application to which the at least one first function information belongs, and when the first target application is an application not installed by the terminal, based on the first target application, generates installation guidance information of the first target application, the installation guidance information of the first target application is used to indicate the process of installing the first target application; and displays the installation guidance information of the first target application on the current interface. Among them, the terminal can input the first target application into the large language model and output the installation guidance information of the first target application.

[0097] In another possible implementation, the installation guidance information of the first target application can be generated by a server; accordingly, after the server finds at least one first function information based on the second feature library, it determines the first target application to which the at least one first function information belongs, and based on the first target application, generates installation guidance information of the first target application, and sends the installation guidance information of the first target application to the terminal. The terminal receives the installation guidance information of the first target application sent by the server, and displays the installation guidance information of the first target application on the current interface, so that the user can install the first target application based on the installation guidance information of the first target application.

[0098] In an embodiment of the present application, for the first target application that is not installed on the terminal, after the user finds the matching first function information in the second feature library in the server through the interface retrieval conditions, the terminal can provide the user with "recommended installation applications" to meet the user's needs, thereby improving the installation rate of the application.

[0099] Step 506: Based on the interface search condition and the functional information of at least one interface, determine at least one second functional information, where the second functional information is information of a matching function retrieved based on the interface search condition and the functional information of at least one interface.

[0100] In some embodiments, this step is the same as step 203 and will not be repeated here.

[0101] Step 507: Based on at least one first function information and at least one second function information, determine the first target interface, the first target interface is the interface to which the first target function information among the at least one first function information and the at least one second function information belongs, and the first target interface is the interface retrieved based on the retrieval instruction.

[0102] In some embodiments, this step is the same as step 204 and will not be repeated here. In one possible implementation, after determining the first target interface and determining that the first target interface is a target interface where the terminal has no application installed, the terminal may guide the user to install the second target application to which the first target interface belongs; accordingly, the process may be: the terminal determines the second target application to which the first target interface belongs, and based on the second target application, generates installation guidance information for the second target application, the installation guidance information for the second target application is used to indicate the process of installing the second target application; the installation guidance information for the second target application is displayed on the current interface, so that the user can install the second target application based on the installation guidance information for the second target application. Among them, the terminal inputs the second target application into the large language model and outputs the installation guidance information for the second target application.

[0103] In an embodiment of the present application, for the second target application that is not installed on the terminal, after the user finds a matching first target interface in the second feature library in the server through the interface retrieval conditions, the terminal can provide the user with "recommended installation applications" to meet the user's needs, thereby improving the installation rate of the application.

[0104] For example, see Figure 6, the user searches for function information in the second feature library of the server through the terminal; when the first target function information is found, the installation guide information of the second target application to which the first target function information belongs is generated, thereby making function recommendations based on the installation guide information of the second target application. In one possible implementation, before the terminal generates the installation guide information of the second target application, the terminal may also generate third tutorial information for the first target function information, and the third tutorial information is used to guide the use of the first target function information. In one possible implementation, the step of the terminal generating the third tutorial information for the first target function information may be: the terminal generates the third tutorial information for the first target function information based on the first target function information; wherein the terminal inputs the first target function information into the large language model and outputs the third tutorial information for the first target function information.

[0105] In an embodiment of the present application, when determining at least one first function information, based on the interface retrieval feature, a feature matching the interface retrieval feature is searched from the first feature library (the feature library corresponding to the installed application) and the second feature library (the feature library corresponding to the non-installed application). Therefore, it is possible not only to retrieve the target interface of the installed application, but also to retrieve the target interface of the non-installed application, thereby improving the success rate of the interface retrieval.

[0106] Please refer to Figure 7 , which shows a flowchart of an interface retrieval method provided by an exemplary embodiment of the present application. This embodiment is described by taking the method used in a terminal as an example, and the method may include the following steps:

[0107] Step 701: In response to a search instruction, feature extraction is performed on the interface search condition indicated by the search instruction to obtain interface search features.

[0108] In some embodiments, this step is the same as step 201 and will not be repeated here.

[0109] Step 702: Determine at least one first functional information based on the interface retrieval features and the feature library. The feature library is used to store features corresponding to the functional information of at least one interface in the historical operation trajectory. The historical operation trajectory is a trajectory model corresponding to the historical operation process. The functional information is used to indicate the functions of the interface.

[0110] In some embodiments, this step is the same as step 202; alternatively, this step can be implemented through the above steps 502-505, which will not be repeated here.

[0111] Step 703: Determine at least one second function information based on the interface search condition and the function information of at least one interface.

[0112] In some embodiments, this step is the same as step 203 and will not be repeated here.

[0113] For example, see Fig. 9 Based on “I want to turn on care mode”, the terminal locates at least one first function information and at least one second function information, namely “View care mode”, “Turn on care mode”, “View care mode description”, “View care mode functions” and “View dark mode”.

[0114] Step 704: Based on at least one first function information and at least one second function information, determine a first target interface, where the first target interface is an interface retrieved based on the search instruction.

[0115] In some embodiments, this step is implemented together with step 204 or 507 and will not be described in detail here.

[0116] Step 705: Based on the first target interface, determine the first target path information from the historical operation trajectory. The historical operation trajectory is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation process and the path information of the operation path corresponding to the interface during the historical operation process. The first target operation path to which the first target path information belongs is the operation path to reach the first target interface.

[0117] This step can be achieved by following the steps (1) to (3), including:

[0118] (1) Based on the first target interface, determine the first target node corresponding to the first target interface from the historical operation trajectory.

[0119] Since the interface features of the interface are characterized by functional information in the historical operation track, and the interface feature matching method is usually used to identify whether two interfaces are the same interface, the terminal determines the first target node (recorded node) corresponding to the first target interface based on the functional information of the recorded node corresponding to the recorded interface in the historical operation track. In some embodiments, the terminal uses the same functional information generation method as the recorded interface to generate functional information corresponding to the first target interface, thereby determining the first target node corresponding to the first target interface by comparing the functional information of the recorded node corresponding to the recorded interface and the functional information corresponding to the first target interface.

[0120] In some embodiments, the functional information of a node includes a node identifier and a node factor of the node. Among them, the node identifier is a unique identifier assigned to the node (without actual meaning), which can be represented in the form of a string. Different nodes have different node identifiers, while the same node has the same node identifier. Optionally, a node factor (Node Factor) is composed of several fields that characterize interface features from a functional dimension, wherein the fields that characterize interface features can be referred to as node sub-factors. When determining whether two nodes are the same node, the terminal matches the field contents with the same field name in the two node factors, that is, matches the node sub-factors in different node factors, thereby determining whether the two nodes are the same node based on the matching results of the functional dimension. In some embodiments, the node factor may include at least one of a first node sub-factor and a second node sub-factor.

[0121] First node sub-factor: During the interface development phase, developers usually add interface function description information to the interface code. Even if the interface effect is iteratively updated later, the interface function description information usually does not change. Therefore, the terminal can determine whether two nodes are the same node from the dimension of interface function.

[0122] In a possible implementation, the terminal determines the first node sub-factor based on the interface function description information of the interface. The interface function description information can be extracted from the interface code. Further, when subsequently identifying whether two nodes are the same node, the terminal can compare whether the first node sub-factors in the node factors of the two nodes are the same. If they are the same, it is determined that the two nodes are the same node; if they are not the same, the terminal needs to make further determinations based on other node sub-factors.

[0123] Second node sub-factor: In a possible implementation, the terminal determines the second node sub-factor based on the unit function information of the interface. The unit function information is used to indicate the minimum unit function of the interface, and the minimum unit function is the minimum unit of the function, such as turning on function A, turning off function A, viewing B, adding C, uploading D, and so on. Regarding the method of generating unit function information, in some embodiments, the terminal generates the unit function information of the interface through a large language model. Optionally, the terminal inputs a screenshot of the interface and a prompt (prompt) of the interface into the large language model, instructing the large language model to extract the unit function information of the interface. The unit function information can be in the form of a list. In a possible implementation, the terminal can generate a node factor corresponding to the first target interface, and compare the node factor with the node factor of the recorded node to determine the first target node corresponding to the first target interface. This step may include the following sub-steps:

[0124] (1-1) Based on the interface features of the functional dimension of the first target interface, a node factor corresponding to the first target interface is generated.

[0125] In a possible implementation, the terminal generates corresponding node sub-factors based on the interface features of the functional dimension of the first target interface, and then determines the node factors corresponding to the first target interface. In some embodiments, the node factors corresponding to the first target interface may include all or part of the above node sub-factors. For example, the terminal determines the first to second node sub-factors corresponding to the first target interface.

[0126] (1-2) Based on the node factors of the recorded nodes corresponding to the recorded interfaces in the historical operation trace and the node factors corresponding to the first target interface, determine the recorded interface that matches the first target interface.

[0127] In a possible implementation, the terminal matches the node factor of the first target interface with the node factor corresponding to the recorded interface in the historical operation track one by one, thereby determining the recorded interface that matches the first target interface. Optionally, the terminal may determine the recorded interface that matches the first target interface based on a single node sub-factor in the node factor, or in combination with multiple node sub-factors in the node factor. In some embodiments, when the node factor includes the first to second node sub-factors mentioned above, the terminal may determine whether the first target interface matches the recorded interface in the following cases.

[0128] Case 1: When the first target interface and the first node sub-factor corresponding to the recorded interface are the same, it is determined that the first target interface matches the recorded interface.

[0129] If the functions of the two interfaces are completely the same, then the two interfaces should belong to the same interface. Therefore, in some embodiments, when the first node sub-factors of the first target interface and the recorded interface are both not empty, the terminal compares whether the first node sub-factors of the two are the same, and when the first node sub-factors of the two are the same, determines that the first target interface matches the recorded interface.

[0130] Case 2: When the second node sub-factors corresponding to the first target interface and the recorded interface are the same, it is determined that the first target interface matches the recorded interface.

[0131] (1-3) Determine the recorded node corresponding to the matched recorded interface as the first target node corresponding to the first target interface.

[0132] In some embodiments, when a single recorded interface is matched, the terminal determines the recorded node corresponding to the recorded interface as the first target node corresponding to the first target interface; when multiple recorded interfaces are matched, the terminal determines the recorded node corresponding to the recorded interface with the highest matching degree as the first target node corresponding to the first target interface.

[0133] (2) Determine, from the historical operation trajectory, a first target behavior edge corresponding to the first target operation behavior involved in the first target operation path including the first target node.

[0134] In a possible implementation, this step can be implemented by following the steps (2-1) to (2-3), including:

[0135] (2-1) Determine, from the historical operation trajectory, a starting node corresponding to the starting interface before the first target operation behavior, and an ending node corresponding to the ending interface after the first target operation behavior.

[0136] In one possible implementation, the terminal determines the starting node corresponding to the starting interface based on the node information corresponding to the starting interface, and obtains the starting node identifier of the starting node; determines the terminating node corresponding to the terminating interface based on the node information corresponding to the terminating interface, and obtains the terminating node identifier of the terminating node.

[0137] In some embodiments, in order to ensure the accuracy of the determined termination node, when the duration of the received first target operation behavior reaches a preset duration (because there may be a response delay in the interface jump), the terminal determines the current interface as the termination interface, and then determines the termination node corresponding to the termination interface.

[0138] (2-2) When the first target operation behavior is related to the first target interface control, determine the node element corresponding to the first target interface control from the historical operation track.

[0139] In some embodiments, the terminal determines whether the first target operation behavior is related to the first target interface control. If the first target operation behavior is related to the first target interface control (i.e., the first target operation behavior acts on the first target interface control), the terminal determines the node element corresponding to the first target interface control. If the first target operation behavior is not related to the first target interface control (for example, the first target operation behavior is the system automatic interface jump behavior after reaching a specified time length), the terminal does not need to determine the node element.

[0140] In a possible implementation, the historical operation track includes node element information of node elements corresponding to each recorded interface control in the recorded interface. Accordingly, the terminal determines the node element information corresponding to the first target interface control, and matches the node element information with the node element information of the node element corresponding to the recorded interface control, thereby determining the matched recorded node element as the node element corresponding to the first target interface control. In some embodiments, the terminal determines the node element factor corresponding to the first target interface control (which is part of the node element information and is used to characterize the characteristics of the node element from multiple dimensions), thereby determining the node element corresponding to the first target interface control based on the degree of matching between the node element factor and the node element factor of the node element corresponding to the recorded interface control.

[0141] (2-3) Determine a first target behavior edge corresponding to the first target operation behavior based on the start node, the end node, the node elements, and the first target behavior type of the first target operation behavior.

[0142] In one possible implementation, the terminal matches the starting node identifier of the starting node, the ending node identifier of the ending node, the node element identifier of the node element, and the first target behavior type of the first target operation behavior with the edge information of the recorded behavior edge corresponding to the recorded operation behavior in the historical operation trajectory, thereby determining the matched recorded behavior edge as the first target behavior edge corresponding to the first target operation behavior.

[0143] (3) Based on the function information of the first target node and the first target edge information of the first target behavior edge, first target path information of the first target operation path is generated.

[0144] After determining each first target node passed through on the first target operation path and the first target behavior edge between each first target node from the historical operation trajectory, the terminal can generate the first target path information of the first target operation path. Based on the first target path information, the terminal can determine the nodes on the first target operation path and the behavior edges between the nodes.

[0145] In a possible implementation, a historical operation track containing a target node corresponding to the first target interface is found, and then in the historical operation track, the target node corresponding to the first target interface is taken as the end point, and the first target operation path to reach the target node is determined from the historical operation track through a breadth-first traversal algorithm. For example, please continue to refer to Figure 3 ,In the positioning stage, the terminal locates the first target path information from the historical operation trajectory based on the first target interface.

[0146] The essence of the terminal determining the first target path information from the historical operation trajectory is to find a first target operation path that can reach the first target interface from the starting interface. For example, the first target operation path from the starting interface to the first target interface is path 1 to path N, where N is an integer greater than 1. Among them, the terminal determines the complete path of the first target operation path through the breadth-first traversal algorithm as: path 1 to path 2 to path 3 to path 4 to path N.

[0147] Step 706: Based on the first target side information included in the first target path information, execute the first target operation behavior indicated by the first target side information, the first target side information including information used to characterize the first target operation behavior and the interface jump situation before and after the first target operation behavior.

[0148] After the terminal of the embodiment of the present application determines the first target path information, it can execute the first target operation behavior indicated by the first target edge information included in the first target roadbed information, thereby automatically executing the first target operation behavior, that is, automatically jumping from the starting interface to the first target interface. For example, please continue to refer to Figure 3 ,In the flashback phase, the terminal automatically jumps from the starting interface to the ,first target interface based on the playback function.

[0149] For example, from a product perspective, the process of the embodiment of the present application is as follows: Figure 8 As shown, the terminal generates a historical operation trajectory in advance and generates a feature library based on the historical operation trajectory; the interface retrieval condition can be "I want to turn on the care mode"; the terminal locates the first target interface based on the interface retrieval condition, determines the first target path information based on the first target interface, and plays back based on the first target path information, and then generates shareable target tutorial information through step 707.

[0150] For example, see Fig.10 , the interface search condition is "view emergency contacts", then the terminal determines the first target interface as the emergency contact interface based on the interface search condition, locates the position of the emergency contact interface in the historical operation track based on the emergency contact interface, and then determines the first target path information and plays it back based on the position of the emergency contact interface in the historical operation track. For example, please refer to Fig.11 The first target operation behavior indicated by the first target side information includes "return to the desktop" - execute the second step "open the social application" - execute the third step "click me" - execute the fourth step "click settings" - execute the fifth step "click account and security" - execute the sixth step "click emergency contact", thereby jumping to the emergency contact interface.

[0151] In one possible implementation, after the terminal determines the first target path information, the first target path information is displayed on the current interface, and a schematic diagram of each interface involved in the first target path information is displayed; however, the first path information and the schematic diagram of each interface may not be very intuitive and there is no detailed explanation; and a good tutorial can more clearly and in detail introduce the detailed process of reaching the first target interface from the initial interface; therefore, after executing step 706, execute step 707 to generate tutorial information.

[0152] Step 707: Based on the first target side information included in the first target path information, generate target tutorial information corresponding to the first target side information, where the target tutorial information is used to guide the completion of the first target operation behavior indicated by the first target side information.

[0153] The terminal generates first tutorial information corresponding to the first target side information based on the first target side information included in the first target path information, wherein the first tutorial information is used to guide the execution of the first target operation behavior indicated by the first target side information in words; and determines the target tutorial information based on the first tutorial information. The terminal inputs the first target side information into the large language model and outputs the first tutorial information corresponding to the first target side information.

[0154] In a possible implementation, the step of generating the first tutorial information corresponding to the first target side information based on the first target side information included in the first target path information may be as follows: the terminal generates the first tutorial information corresponding to the first target side information based on the first target side information included in the first target path information and the interface information of multiple interfaces included in the first target operation path to which the first target path information belongs. For example, if the first target function information is "enable the care mode of the social application", the interface information and target side information of the multiple interfaces are respectively:

[0155] Interface information: desktop; target side information: click on "social application"; interface information: main interface of social application; target side information: click on the text "me" in the lower right corner of the screen; interface information: functional interface of social application; target side information: click on the text "settings" in the middle of the screen; interface information: setting interface of social application; target side information: click on the text "care mode" in the upper part of the screen; interface information: detailed description and activation interface of care mode.

[0156] The terminal inputs the above information into the large language model and outputs the first tutorial information; the first tutorial information can be as follows: Title "Take you to play social applications: How to turn on the care mode of social applications" Content: Hello, students! Today I will take you to understand a very practical small function of social applications - "care mode". Below, I will explain the detailed steps to you one by one: Step 1: Start the social application. First, find the social application App icon on your mobile phone desktop, click to enter, and you will see the chat history list interface of the social application. Tip: This list interface allows you to browse and manage all social chat records, and can also search and access more functions. Step 2: Enter the "Me" interface. In the bottom navigation bar of the list interface, slide to the right to the far right of a row of small characters, click the "Me" text, and enter the personal page of the social application. Tip: This interface allows us to view personal information, and also switch between the list interface, address book, discovery and me and other functional modules. Step 3: Click "Settings". Find and click to open the "Settings" item in the middle of the "Me" interface to enter the settings interface of the social application. Tip: Under "Settings", various setting options of social applications are gathered. As long as you have something to modify, here can help you. Step 4: Turn on the caring mode. Slide up on the settings interface, find the top of the page, and click "Caring Mode" to enter the detailed page of the caring mode. Tip: This is a detailed description page of the "Caring Mode" of the social application. You can learn more about the "Caring Mode" and turn it on here. The above are all the steps on how to enable the caring mode of the social application. I believe that through the above steps, you can definitely turn on this function smoothly.

[0157] In a possible implementation, the target tutorial information is text information, and the terminal determines the target tutorial information based on the first tutorial information in the step that the terminal determines the first tutorial information as the target tutorial information. In another possible implementation, the target tutorial information is graphic information, and the terminal determines the target tutorial information based on the first tutorial information in the step that the terminal determines the target tutorial information as the target tutorial information. The target tutorial information is to use graphic information to guide the execution of the first target operation behavior indicated by the first target edge. Among them, the first tutorial information and the function information are text information, and the target edge information is picture information; therefore, the target tutorial information is graphic information. For example, the terminal obtains the second tutorial information based on the function information, the target edge information and the first tutorial information in the historical operation track. The process can be: the terminal merges the function information, the target edge information and the first tutorial information in the historical operation track to obtain the second tutorial information.

[0158] In another possible implementation, the target tutorial information is video information, and the terminal determines the target tutorial information based on the first tutorial information in the following steps: the terminal obtains the second tutorial information based on at least one of the function information in the historical operation track, the first target edge information, and the first tutorial information, converts the second tutorial information into tutorial information in video format, and obtains the target tutorial information, where the target tutorial information is a video guide to perform the first target operation behavior indicated by the first target edge. For example, the terminal uses slide technology and TTS capabilities to convert the second tutorial information into tutorial information in video format to obtain the target tutorial information, thereby making the target tutorial information more intuitive.

[0159] In the embodiment of the present application, the target tutorial information includes detailed steps; therefore, the terminal provides the user with the target tutorial information, which can guide the user to perform the first target operation behavior in more detail. In addition, the target tutorial information is generated based on a large language model, so no human intervention is required, and an automated process is realized. In addition, the target tutorial information can be in at least one of a text format, a picture format, and a video format; therefore, the format of the target tutorial information is enriched. For example, please refer to Fig.12 , the terminal generates target tutorial information in plain text format, target tutorial information in graphic and text format, and target tutorial information in video format.

[0160] In one possible implementation, the terminal may also allow the user to understand and use the functions on the terminal through a dialogue; accordingly, the process may be: receiving a first voice command, the first voice command is used to request to obtain second target function information; based on the first voice command, generating second tutorial information of the second target function information, the second tutorial information is used to guide the use of the second target function; wherein the terminal inputs the first voice command into the large language model, and outputs the second tutorial information of the second target function information.

[0161] In another possible implementation, the terminal can also use the flashback function of the historical operation track to allow the user to complete complex tasks by issuing action instructions through voice; accordingly, the process can be: receiving a second voice instruction, the second voice instruction is used to request the display of the second target interface; based on the second voice instruction, determining the second target path information from the historical operation track, the historical operation track is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation process and the path information of the interface corresponding to the operation path during the historical operation process, and the second target operation path to which the second target path information belongs is the operation path to reach the second target interface; based on the second target edge information included in the second target path information, executing the second target operation behavior indicated by the second target edge information, the second target edge information includes information used to characterize the second target operation behavior and the interface jump situation before and after the second target operation behavior. For example, please continue to refer to Fig.12, the terminal controls the terminal through human-computer communication to understand the function of the terminal or control the terminal to complete a task.

[0162] Step 708: Display the target tutorial information on the current interface.

[0163] The target tutorial information can also be shared, and the process can be to share the target tutorial information through social applications. For example, please continue to refer to Figure 3 ,In the positioning stage, the terminal generates target tutorial information and ,shares the target tutorial information through social applications.

[0164] The present application embodiment provides a unified function full search and machine assistant solution. For example, please refer to Fig.13 First, generate a historical operation track, which includes the historical operation track of the terminal's first-party application, the historical operation track of the third-party application, and the historical operation track of the uninstalled application; then generate a first feature library based on the historical operation track, and based on the first feature library and the functional information of at least one interface in the historical operation track, through the retrieval function, positioning function, playback function, large language model and RAG retrieval technology, connect the function retrieval (retrieval of the first target interface), automatic execution (automatic jump to the first target interface) and tutorial generation to form a complete technical route. From the perspective of terminal manufacturers, this solution has great value and can provide users with: the entire machine and even the entire mobile terminal ecosystem, the retrieval of any function and the instructions for using the machine.

[0165] Please refer to Fig.14 , which shows a structural block diagram of an interface retrieval device provided by an exemplary embodiment of the present application. The device includes: a first processing module 1401, which is used to extract features of the interface retrieval conditions indicated by the retrieval instruction in response to a retrieval instruction to obtain an interface retrieval feature; a first determination module 1402, which is used to determine the first target function information based on the interface retrieval feature, the feature library corresponding to the historical operation track, the interface retrieval condition and the historical operation track, the historical operation track is used to record the function information of at least one interface during the historical operation process, the feature library is used to store the features corresponding to the function information of the at least one interface, and the function information is used to indicate the function of the interface; a second determination module 1403, which is used to determine the first target interface to which the first target function information belongs, and the first target interface is the interface retrieved based on the retrieval instruction.

[0166] In a possible implementation, the first determination module 1402 is used to determine at least one first function information based on the interface retrieval feature and the feature library, the first function information being information of matching functions retrieved based on the interface retrieval feature and the feature library; determine at least one second function information based on the interface retrieval condition and the function information of the at least one interface, the second function information being information of matching functions retrieved based on the interface retrieval condition and the function information of the at least one interface; and determine the first target function information based on the at least one first function information and the at least one second function information.

[0167] In another possible implementation, the first determination module 1402 is used to search for features matching the interface retrieval features from a first feature library based on the interface retrieval features, the first feature library being used to store features corresponding to function information of at least one interface in a first historical operation trajectory, the first historical operation trajectory being a historical operation trajectory generated in an application installed on the terminal; when at least one first feature is found, determine at least one first function information corresponding to the at least one first feature.

[0168] In another possible implementation, the first determination module 1402 is used to send a query request to the server, the query request carries the interface retrieval feature, so that the server searches for at least one first feature matching the interface retrieval feature from a second feature library based on the interface retrieval feature, and sends at least one first function information corresponding to the at least one first feature to the terminal, the second feature library includes features corresponding to the function information of at least one interface in a second historical operation track, the second historical operation track is a historical operation track generated in an application not installed on the terminal; and receives at least one first function information sent by the server.

[0169] In another possible implementation, the device also includes: a third determination module, used to determine the first target application to which the at least one first function information belongs; a first generation module, used to generate installation guidance information of the first target application based on the first target application when the first target application is an application not installed on the terminal, the installation guidance information of the first target application is used to indicate the process of installing the first target application; and a first display module, used to display the installation guidance information of the first target application on the current interface.

[0170] In another possible implementation, the device also includes: a fourth determination module, used to determine the functional information of at least one interface in the first historical operation trajectory, the functional information includes at least one item of unit functional information and interface function description information, the unit functional information is used to indicate the minimum unit function of the interface, and the interface function description information is used to indicate the interface function of the interface; a second processing module, used to extract features from the functional information of the at least one interface to obtain the first feature library.

[0171] In another possible implementation, the device also includes: a setting service, which is used to set the access permission of the first feature library to accessible, so that other services can assist in executing the tasks of the other services by accessing the first feature library, and the other services are services in the terminal other than executing the retrieval instruction, or the other services are services in a third-party system.

[0172] In another possible implementation, the first determining module 1402 is used to merge and sort the at least one first function information and the at least one second function information; and determine the first target function information based on the function information sequence obtained by sorting.

[0173] In another possible implementation, the at least one first function information is sorted in sequence based on the degree of matching with the interface retrieval feature, and the at least one second function information is sorted in sequence based on the degree of matching with the interface retrieval condition; the first determination module 1402 is used to determine the weight of the at least one first function information and the at least one second function information based on the order of the function information, and the weight of the function information is negatively correlated with the order; based on the weight of the at least one first function information and the weight of the at least one second function information, sort the at least one first function information and the at least one second function information.

[0174] In another possible implementation, the first determination module 1402 is used to segment the text information corresponding to the interface search condition to obtain multiple keywords; determine the matching degree between the multiple keywords and the functional information of the at least one interface; based on the matching degree between the multiple keywords and the functional information of the at least one interface, determine at least one second functional information from the functional information of the at least one interface, and the matching degree between the at least one functional information and the multiple keywords satisfies the matching condition.

[0175] In another possible implementation, the device also includes: a fifth determination module, used to determine first target path information from the historical operation trajectory based on the first target interface, the historical operation trajectory is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation process and the path information of the interface corresponding to the operation path during the historical operation process, and the first target operation path to which the first target path information belongs is the operation path to reach the first target interface; a first execution module, used to execute the first target operation behavior indicated by the first target edge information based on the first target edge information included in the first target path information, the first target edge information including information used to characterize the first target operation behavior and the interface jump situation before and after the first target operation behavior.

[0176] In another possible implementation, the device also includes: a second generation module, used to generate target tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information, and the target tutorial information is used to guide the completion of the first target operation behavior indicated by the first target edge information; a second display module, used to display the target tutorial information on the current interface.

[0177] In another possible implementation, the second generation module is used to generate first tutorial information corresponding to the first target edge information based on the first target edge information included in the first target path information, wherein the first tutorial information is used to provide textual guidance for executing the first target operation behavior indicated by the first target edge information; and determine the target tutorial information based on the first tutorial information.

[0178] In another possible implementation, the second generating module is used to determine the first tutorial information as the target tutorial information.

[0179] In another possible implementation, the second generation module is used to obtain second tutorial information based on at least one of the function information in the historical operation trajectory, the target edge information and the first tutorial information, and determine the second tutorial information as the target tutorial information, wherein the target tutorial information uses graphic information to guide the execution of the first target operation behavior indicated by the first target edge.

[0180] In another possible implementation, the second generation module is used to obtain second tutorial information based on at least one of the function information in the historical operation trajectory, the first target edge information and the first tutorial information, convert the second tutorial information into tutorial information in a video format, and obtain the target tutorial information, wherein the target tutorial information is a video guide for executing the first target operation behavior indicated by the first target edge.

[0181] In another possible implementation, the device also includes: a receiving module, which is used to receive a voice instruction, and the voice instruction is used to request to display a second target interface; a sixth determination module, which is used to determine the second target path information from the historical operation trajectory based on the voice instruction, and the historical operation trajectory is also used to record the edge information of the behavior edge corresponding to the operation behavior in the interface during the historical operation process and the path information of the interface corresponding to the operation path during the historical operation process, and the second target operation path to which the second target path information belongs is the operation path to reach the second target interface; a first execution module, which is used to execute the second target operation behavior indicated by the second target edge information based on the second target edge information included in the second target path information, and the second target edge information includes information used to characterize the second target operation behavior and the interface jump situation before and after the second target operation behavior.

[0182] In an embodiment of the present application, one-way recall retrieval is performed based on interface retrieval features and a feature library, and one-way recall retrieval is performed based on interface retrieval conditions and functional information of the interface. The first target interface that the user wants to retrieve is comprehensively determined based on the two-way recall retrieval results, thereby realizing two-way recall retrieval to retrieve the first target interface, thereby improving the accuracy of the retrieved first target interface.

[0183] It should be noted that the interface retrieval device provided in the above embodiment only uses the division of the above functional modules as an example when performing retrieval. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal is divided into different functional modules to complete all or part of the functions described above. In addition, the interface retrieval device provided in the above embodiment and the interface retrieval method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0184] Please refer to Fig.15, which shows a block diagram of a terminal shown in an exemplary embodiment of the present application. The terminal in the present application may include one or more of the following components: a processor 1510, a memory 1520, and a display screen 1530. The processor 1510 may include one or more processing cores. The processor 1510 uses various interfaces and lines to connect various parts in the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1520, and calling data stored in the memory 1520. Optionally, the processor 1510 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-ProgrammableGate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 1510 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a time-frequency resource allocation method device (Graphics Processing Unit, GPU), a neural network processor (Neural-networkProcessing Unit, NPU) and a modem. The CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen 1530; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1510, but may be implemented by a separate chip.

[0185] The memory 1520 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1520 includes a non-transitory computer-readable storage medium. The memory 1520 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1520 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data created according to the use of the terminal (such as audio data, a phone book), etc.

[0186] The display screen 1530 is a display component for displaying a user interface. Optionally, the display screen 1530 is a display screen with a touch function, through which the user can use any suitable object such as a finger or a touch pen to perform touch operations on the display screen 1530. The display screen 1530 is usually arranged on the front panel of the terminal. The display screen 1530 can be designed as a full screen, a curved screen, a special-shaped screen, a double-sided screen or a folding screen. The display screen 1530 can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., which is not limited in this embodiment.

[0187] In addition, those skilled in the art will appreciate that the structure of the terminal shown in the above drawings does not constitute a limitation on the terminal, and the terminal may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the terminal also includes an audio acquisition device, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0188] The embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores at least one computer instruction, the at least one computer instruction is used to be executed by a processor to implement the interface retrieval method as described in the above embodiment. On the other hand, the embodiment of the present application provides a computer program product, the computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium; the processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the interface retrieval method as described in the above embodiment.

[0189] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0190] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An interface retrieval method, characterized in that: The method comprises: In response to the search instruction, extracting features of the interface search condition indicated by the search instruction to obtain interface search features; Determine first target functional information based on the interface retrieval feature, a feature library corresponding to the historical operation track, the interface retrieval condition, and functional information of at least one interface in the historical operation track, wherein the historical operation track is used to record functional information of at least one interface during the historical operation process, the feature library is used to store features corresponding to the functional information of the at least one interface, and the functional information is used to indicate the function of the interface; A first target interface to which the first target function information belongs is determined, where the first target interface is an interface retrieved based on the search instruction.

2. The method according to claim 1, characterized in that The determining of the first target function information based on the interface retrieval feature, the feature library corresponding to the historical operation track, the interface retrieval condition and the historical operation track includes: Based on the interface retrieval feature and the feature library, determining at least one first function information, wherein the first function information is information of a matching function retrieved based on the interface retrieval feature and the feature library; Based on the interface search condition and the functional information of the at least one interface, determining at least one second functional information, where the second functional information is information of a matching function retrieved based on the interface search condition and the functional information of the at least one interface; The first target function information is determined based on the at least one first function information and the at least one second function information.

3. The method according to claim 2, characterized in that The determining at least one first functional information based on the interface retrieval feature and the feature library includes: Based on the interface retrieval feature, searching a feature matching the interface retrieval feature from a first feature library, the first feature library being used to store features corresponding to function information of at least one interface in a first historical operation track, the first historical operation track being a historical operation track generated in an application installed on the terminal; When at least one first feature is found, at least one first function information corresponding to the at least one first feature is determined.

4. The method according to claim 2, characterized in that: The determining at least one first functional information based on the interface retrieval feature and the feature library includes: Sending a query request to a server, the query request carrying the interface retrieval feature, so that the server searches for at least one first feature matching the interface retrieval feature from a second feature library based on the interface retrieval feature, and sends at least one first function information corresponding to the at least one first feature to the terminal, the second feature library including a feature corresponding to at least one interface function information in a second historical operation track, the second historical operation track being a historical operation track generated in an application not installed by the terminal; Receive at least one first function information sent by the server.

5. The method according to claim 4, characterized in that The method further comprises: Determining a first target application to which the at least one first function information belongs; In a case where the first target application is an application not installed on the terminal, generating installation guide information of the first target application based on the first target application, wherein the installation guide information of the first target application is used to indicate a process of installing the first target application; The installation guide information of the first target application is displayed on the current interface.

6. The method according to claim 3, characterized in that The method further comprises: Determine function information of at least one interface in the first historical operation track, the function information including at least one of unit function information and interface function description information, the unit function information is used to indicate the minimum unit function of the interface, and the interface function description information is used to indicate the interface function of the interface; Feature extraction is performed on the functional information of the at least one interface to obtain the first feature library.

7. The method according to claim 6, characterized in that The method further comprises: The access right of the first feature library is set to accessible, so that other services can assist in executing the tasks of the other services by accessing the first feature library. The other services are services in the terminal other than executing the retrieval instruction, or the other services are services in a third-party system.

8. The method according to claim 2, characterized in that: The determining the first target function information based on the at least one first function information and the at least one second function information includes: Merging and sorting the at least one first function information and the at least one second function information; Based on the sorted function information sequence, the first target function information is determined.

9. The method according to claim 8, characterized in that The at least one first function information is sequentially sorted based on the degree of matching with the interface search feature, and the at least one second function information is sequentially sorted based on the degree of matching with the interface search condition; The merging and sorting the at least one first function information and the at least one second function information comprises: For any one of the at least one first functional information and the at least one second functional information, determining a weight of the functional information based on an order of the functional information, wherein the weight of the functional information is negatively correlated with the order; The at least one first function information and the at least one second function information are ranked based on the weight of the at least one first function information and the weight of the at least one second function information.

10. The method according to claim 2, characterized in that The determining at least one second functional information based on the interface search condition and the functional information of the at least one interface includes: Segmenting the text information corresponding to the interface search condition to obtain multiple keywords; Determining a degree of match between the plurality of keywords and the functional information of the at least one interface; Based on the matching degree between the multiple keywords and the functional information of the at least one interface, at least one second functional information is determined from the functional information of the at least one interface, and the matching degree between the at least one functional information and the multiple keywords satisfies the matching condition.

11. The method according to claim 1, characterized in that: After determining the first target interface to which the first target function information belongs, the method further includes: Based on the first target interface, determining first target path information from the historical operation track, the historical operation track is further used to record edge information of behavior edges corresponding to operation behaviors in the interface during historical operations and path information of operation paths corresponding to the interface during historical operations, and the first target operation path to which the first target path information belongs is an operation path to reach the first target interface; Based on the first target side information included in the first target path information, a first target operation behavior indicated by the first target side information is executed, wherein the first target side information includes information used to characterize the first target operation behavior and interface jump conditions before and after the first target operation behavior.

12. The method according to claim 11, characterized in that The method further comprises: Based on the first target side information included in the first target path information, generating target tutorial information corresponding to the first target side information, wherein the target tutorial information is used to guide the completion of the first target operation behavior indicated by the first target side information; The target tutorial information is displayed on the current interface.

13. The method according to claim 12, characterized in that The step of generating target tutorial information corresponding to the first target side information based on the first target side information included in the first target path information includes: Based on the first target side information included in the first target path information, first tutorial information corresponding to the first target side information is generated, where the first tutorial information is used to guide the execution of the first target operation behavior indicated by the first target side information in words; Based on the first tutorial information, the target tutorial information is determined.

14. The method according to claim 13, characterized in that The determining the target tutorial information based on the first tutorial information includes: The first tutorial information is determined as the target tutorial information.

15. The method according to claim 13, characterized in that The determining the target tutorial information based on the first tutorial information includes: According to at least one of the function information in the historical operation trajectory, the target edge information and the first tutorial information, second tutorial information is obtained, and the second tutorial information is determined as the target tutorial information, where the target tutorial information uses graphic information to guide the execution of the first target operation behavior indicated by the first target edge.

16. The method according to claim 13, characterized in that The determining the target tutorial information based on the first tutorial information includes: According to at least one of the function information in the historical operation trajectory, the first target edge information and the first tutorial information, second tutorial information is obtained, and the second tutorial information is converted into tutorial information in a video format to obtain the target tutorial information, wherein the target tutorial information is a video guide for executing the first target operation behavior indicated by the first target edge.

17. The method according to claim 1, characterized in that The method further comprises: receiving a voice command, wherein the voice command is used to request display of a second target interface; Based on the voice command, second target path information is determined from the historical operation track, the historical operation track is further used to record edge information of behavior edges corresponding to operation behaviors in the interface during the historical operation process and path information of the interface corresponding to the operation path during the historical operation process, and the second target operation path to which the second target path information belongs is an operation path to reach the second target interface; Based on the second target side information included in the second target path information, a second target operation behavior indicated by the second target side information is executed, wherein the second target side information includes information for characterizing the second target operation behavior and interface jump conditions before and after the second target operation behavior.

18. An interface search device, characterized in that: The device comprises: A first processing module is used to extract features of the interface search conditions indicated by the search instruction in response to the search instruction to obtain interface search features; A first determination module is used to determine first target function information based on the interface retrieval feature, a feature library corresponding to a historical operation track, the interface retrieval condition and the historical operation track, wherein the historical operation track is used to record function information of at least one interface during a historical operation, the feature library is used to store features corresponding to the function information of the at least one interface, and the function information is used to indicate a function of the interface; The second determination module is used to determine a first target interface to which the first target function information belongs, where the first target interface is an interface retrieved based on the search instruction.

19. A terminal, characterized in that: The terminal includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the interface retrieval method as described in any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the interface retrieval method as described in any one of claims 1 to 17.

21. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the interface retrieval method as described in any one of claims 1 to 17.

Images