Instruction recommendation method and apparatus, electronic device, storage medium and program product

Abstract

Provided are an instruction recommendation method and apparatus, an electronic device, a storage medium and a program product. The method comprises: displaying an application page (S110); if a selected object in an application page is acquired from an application, acquiring at least one recommendation instruction corresponding to the selected object, wherein the recommendation instruction is used for indicating a processing operation on the selected object (S120); and displaying the at least one recommendation instruction (S130). A selected object in an application page is acquired from an application for instruction recommendation, so that excessive consideration factors are prevented from making it difficult to accurately pinpoint the specific intents of users, and the effectiveness of instruction recommendation is improved.

Description

Recommended methods, apparatus, electronic devices, storage media, and program products for the instructions.

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese application No. CN202510347534.X, filed on March 21, 2025, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0003] This application relates to the field of electronic equipment technology, and more specifically, to a recommended method, apparatus, electronic device, storage medium, and program product for providing instructions. Background Technology

[0004] With the development of science and technology, electronic devices are becoming increasingly widespread and feature-rich, becoming an essential part of people's daily lives. Currently, electronic devices can recommend commands based on users' interests within application pages. Summary of the Invention

[0005] In view of the above problems, this application proposes a recommended method, apparatus, electronic device, storage medium, and program product for instructions to solve the above problems.

[0006] In a first aspect, embodiments of this application provide a method for recommending instructions, the method comprising: displaying an application page; if a selected object is found in the application page, obtaining at least one recommended instruction corresponding to the selected object, wherein the recommended instruction is used to indicate a processing operation on the selected object; and displaying the at least one recommended instruction.

[0007] Secondly, embodiments of this application provide a recommendation device for instructions, the device comprising: an application page display module for displaying an application page; a recommendation instruction acquisition module for acquiring at least one recommendation instruction corresponding to the selected object if a selected object is found in the application page, wherein the recommendation instruction is used to instruct processing operations on the selected object; and a recommendation instruction display module for displaying the at least one recommendation instruction.

[0008] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory is coupled to the processor, the memory stores instructions, and when the instructions are executed by the processor, the processor performs the above-described method.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be invoked by a processor to execute the above-described method.

[0010] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the above-described method. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 shows a flowchart illustrating the recommended method of the instructions provided in an embodiment of this application;

[0013] Figure 2 shows a flowchart illustrating a recommended method for instructions provided in an embodiment of this application;

[0014] Figure 3 shows a flowchart illustrating step S240 of the recommended method of the instructions shown in Figure 2 of this application;

[0015] Figure 4 shows a flowchart illustrating step S250 of the recommended method of the instructions shown in Figure 2 of this application;

[0016] Figure 5 shows a flowchart illustrating a recommended method for instructions provided in an embodiment of this application;

[0017] Figure 6 shows a flowchart illustrating a recommended method for instructions provided in an embodiment of this application;

[0018] Figure 7 shows a schematic diagram of a first interface of the electronic device provided in an embodiment of this application;

[0019] Figure 8 shows a schematic diagram of a first interface of the electronic device provided in an embodiment of this application;

[0020] Figure 9 shows a schematic diagram of a first interface of the electronic device provided in an embodiment of this application;

[0021] Figure 10 shows a schematic diagram of a first interface of the electronic device provided in an embodiment of this application;

[0022] Figure 11 shows a schematic diagram of a first interface of the electronic device provided in an embodiment of this application;

[0023] Figure 12 shows a schematic diagram of a first interface of the electronic device provided in an embodiment of this application;

[0024] Figure 13 shows a schematic diagram of a first interface of the electronic device provided in an embodiment of this application;

[0025] Figure 14 shows a timing diagram of the recommended instructions provided in the embodiments of this application;

[0026] Figure 15 shows a timing diagram of the acquisition and processing of the selected object provided in an embodiment of this application;

[0027] Figure 16 shows a block diagram of a recommended apparatus for the instructions provided in an embodiment of this application;

[0028] Figure 17 shows a block diagram of an electronic device according to an embodiment of the present application for executing a recommended method according to the instructions of an embodiment of the present application;

[0029] Figure 18 illustrates a storage unit according to an embodiment of the present application for storing or carrying program code that implements the recommended method according to the instructions of the present application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0031] Currently, electronic devices can recommend commands based on users' interests on application pages. Specifically, electronic devices can recommend commands by combining a wide range of contexts such as the application page the user is currently on, device status, and geographical information. However, this approach considers too many factors and makes it difficult to accurately target the user's specific intent, thus reducing the effectiveness of command recommendations.

[0032] To address the aforementioned problems, the inventors, through extensive research, discovered and proposed the instruction recommendation method, apparatus, electronic device, storage medium, and program product provided in the embodiments of this application. By establishing a communication channel with the application, the method obtains the selected object selected through native interaction on the application page and recommends instructions based on the selected object. This avoids the difficulty of accurately targeting the user's specific intent due to excessive consideration of factors, thus improving the effectiveness of instruction recommendations. The specific instruction recommendation method will be described in detail in subsequent embodiments.

[0033] This application provides a method for recommending instructions, wherein the method includes: displaying an application page; if a selected object is found on the application page, obtaining at least one recommended instruction corresponding to the selected object, wherein the recommended instruction is used to indicate a processing operation on the selected object; and displaying the at least one recommended instruction.

[0034] According to an embodiment of this application, after displaying the at least one recommendation instruction, the method further includes: after receiving a target instruction for instructing processing of the selected object, obtaining the selected object from the application; and processing the selected object based on the target instruction.

[0035] According to an embodiment of this application, before obtaining the selected object from the application after receiving a target instruction for instructing processing of the selected object, the method further includes: if a touch operation is detected acting on any of the at least one recommendation instruction, then generating the target instruction based on the touch operation; or if target voice information is received, then generating the target instruction based on the target voice information; or if target text information is received through an input box, then generating the target instruction based on the target text information.

[0036] According to an embodiment of this application, the target instruction is generated based on the target text information, and the method further includes: obtaining the input text information during the process of receiving input text information through the input box; and updating the at least one recommended instruction based on the input text information.

[0037] According to an embodiment of this application, the method further includes: displaying a dialog page and displaying the processing result for the selected object on the dialog page.

[0038] According to an embodiment of this application, displaying the dialog page includes: overlaying the dialog page onto the application page.

[0039] According to an embodiment of this application, obtaining the selected object from the application includes: obtaining the page type of the application page from the application; and obtaining the selected object from the application based on the page type.

[0040] According to an embodiment of this application, obtaining the selected object from the application based on the page type includes: determining an object acquisition method based on the page type; and obtaining the selected object from the application according to the object acquisition method.

[0041] According to an embodiment of this application, obtaining the selected object from the application based on the page type includes: if the page type is a list page, obtaining the object selected by checking a box on the application page as the selected object; if the page type is a details page, obtaining the object selected by selecting a box on the application page as the selected object.

[0042] According to an embodiment of this application, if the page type is a details page type, then obtaining the object selected by box selection in the application page from the application as the selected object includes: if the page type is a details page type and the target instruction does not declare the processing scope for the application page, then obtaining the object selected by box selection in the application page from the application as the selected object.

[0043] According to an embodiment of this application, the method further includes: if the page type is a details page type and the target instruction declaration processing scope includes the entire application page, then obtain the page content of the application page from the application.

[0044] According to an embodiment of this application, processing the selected object based on the target instruction includes: performing intent analysis on the target instruction to obtain the processing intent corresponding to the target instruction; and processing the selected object according to the processing intent.

[0045] According to an embodiment of this application, obtaining at least one recommendation instruction corresponding to the selected object includes: obtaining the object type of the selected object from the application; and determining the at least one recommendation instruction based on the object type.

[0046] According to an embodiment of this application, before obtaining at least one recommendation instruction related to the selected object if the selected object is found to exist on the application page from the application, the method further includes: after receiving a wake-up instruction for instructing the smart assistant to wake up, waking up the smart assistant, and determining from the application page through the smart assistant whether the selected object exists on the application page; the step of obtaining at least one recommendation instruction corresponding to the selected object if the selected object is found to exist on the application page from the application includes: if the selected object is found to exist on the application page from the application, obtaining at least one recommendation instruction corresponding to the selected object through the smart assistant.

[0047] According to an embodiment of this application, displaying the at least one recommendation instruction includes: displaying the at least one recommendation instruction in a floating position on the application page.

[0048] According to an embodiment of this application, displaying the at least one recommendation instruction includes: determining the number of recommendation instructions; if the number of recommendation instructions exceeds the upper limit, then selecting a portion of the recommendation instructions from all recommendation instructions in descending order of relevance to the selected object, with a corresponding number not exceeding the upper limit, for display; or if the number of recommendation instructions exceeds the upper limit, then selecting a portion of the recommendation instructions from all recommendation instructions in descending order of usage rate, with a corresponding number not exceeding the upper limit, for display.

[0049] The following will explain the terms used in the embodiments of this application:

[0050] Intelligent Assistant: An "intelligent assistant" refers to a computer program that uses artificial intelligence technology to interact with users through natural language processing, machine learning, and multimodal interaction technologies (such as voice, text, and images).

[0051] Screen Understanding: "Screen understanding" refers to the ability to analyze and understand the content on a display screen using artificial intelligence technology. This includes recognizing and interpreting various elements such as text, images, and icons, and then making corresponding responses or actions based on this.

[0052] Screen Referring: "Screen referring" refers to the ability to identify and process specific elements (such as text, images, icons, etc.) pointed to by a user on a screen using artificial intelligence technology. This technology can understand the user's intent and perform corresponding actions, such as selecting, clicking, or providing relevant information.

[0053] Points of Interest (POIs): “POIs” refer to specific objects on the screen that the user wants the intelligent assistant to focus on, understand, or process according to their intentions. These objects can be text, images, icons, or other interface elements.

[0054] App Native Interaction: App native interaction is an interaction method designed within a specific application itself. It is different from operating system-level gestures and global interactions with an unpredictable GUI. It focuses on direct interaction with application functions and content.

[0055] Please refer to Figure 1, which shows a flowchart of the instruction recommendation method provided in this application embodiment. This method is used to establish a communication channel with the application, obtain the selected object selected through native interaction on the application page, and recommend instructions based on the selected object. This avoids the difficulty of accurately hitting the user's specific intent due to too many factors, thus improving the effectiveness of instruction recommendations. In a specific embodiment, this instruction recommendation method is applied to the instruction recommendation device 200 shown in Figure 16 and the electronic device 100 (Figure 17) configured with the instruction recommendation device 200. The specific process of this embodiment will be described below using an electronic device as an example. Of course, it is understood that the electronic device provided in this embodiment may include smartphones, tablets, in-vehicle devices, wearable devices, etc., and is not limited here. The process shown in Figure 1 will be described in detail below. The instruction recommendation method specifically includes the following steps:

[0056] Step S110: Display the application page.

[0057] In this embodiment, the electronic device may display application pages. An application page can be understood as a page of a specific application that appears when the electronic device is unlocked. It typically refers to a specific page within a software or application that the user interacts with. These pages are important components of the software's functionality, used to display information, collect user input, provide navigation, and execute various operations requested by the user. The design of application pages directly affects the user experience and the ease of use of the software.

[0058] Optionally, the displayed application page can be the application page of the electronic device's system application, such as the application page of the camera application or the application page of the telephone application; the displayed application page can also be the application page of the electronic device's third-party application, such as the application page of the instant messaging application, the application page of the email application, the application page of the video application, the application page of the music application, etc., without limitation.

[0059] Optionally, the electronic device can display the application page in full screen or partially screen. If the electronic device displays the application page partially screen, it can display half the screen, a quarter of the screen, etc., without limitation.

[0060] In some implementations, application icons for various applications can be displayed on the desktop of an electronic device. During the display of these icons, if a touch operation is detected on the icon, the application page can be displayed in response to that touch operation.

[0061] In some implementations, the electronic device may display a search box in the main screen pull-down menu or in the negative one screen. By entering the application name in the search box, the application icon of the searched application can be obtained. If a touch operation is detected on the application icon during the display of the application icon, the application page can be displayed in response to the touch operation.

[0062] In some implementations, the electronic device may display an application drawer through which application icons of applications can be displayed. During the display of application icons, if a touch operation is detected on the application icon, the application page can be displayed in response to the touch operation.

[0063] In some implementations, the electronic device can display the application's application page when it receives a voice command instructing the application to be opened.

[0064] Of course, in this embodiment, there are many other ways to trigger the display of the application page, which will not be described in detail here.

[0065] Step S120: If a selected object is found in the application page, then at least one recommendation instruction corresponding to the selected object is obtained, wherein the recommendation instruction is used to indicate the processing operation for the selected object.

[0066] Optionally, the smart assistant of the electronic device can establish a communication channel with the application page to which the application belongs. Based on this communication channel, data transmission between the smart assistant and the application can be realized. For example, the smart assistant can send data requests to the application, and the application can provide corresponding data based on the data requests.

[0067] As a method, an Application Programming Interface (API) is an interface that allows different applications to interact and share data. Through APIs, a smart assistant can request information from applications about the user's current actions or interface state to determine the user's focus of attention. Simultaneously, applications can also send relevant data and event notifications to the smart assistant via APIs. For example, in application development, many applications provide specific APIs to obtain information such as the content currently displayed on the screen, the text or element selected by the user, etc. The smart assistant can call these APIs to obtain the user's focus of action within the application in order to provide corresponding help and suggestions.

[0068] As another method, WebSocket is a protocol for enabling bidirectional real-time communication in web applications. It allows smart assistants and applications to exchange data and information in real time after establishing a connection. Through a WebSocket connection, the application can push information such as user actions on the page and focus changes to the smart assistant in real time, and the smart assistant can also respond instantly and send relevant feedback or suggestions.

[0069] In this embodiment, a query message can be sent to the application via a communication channel. This query message inquires whether a selected object exists on the application page. Upon confirming the existence of a selected object, feedback is provided. Therefore, the application can provide feedback via this communication channel upon confirming the existence of a selected object. Based on this, if the application confirms the existence of a selected object on its page, at least one recommendation instruction corresponding to the selected object can be obtained. This recommendation instruction instructs on the processing operation for the selected object.

[0070] In some implementations, during the display of an application page, users can select objects from the application page using native in-app interactions. For example, if the application page includes text content, users can select a text paragraph using native in-app text selection functionality (such as drag-and-drop) and designate the selected text paragraph as the selected object. Similarly, if the application page includes multiple lists, users can select one or more lists using native checkbox interaction functionality and designate the selected list as the selected object.

[0071] As an feasible approach, if the application determines that a selected object exists on the application page, it can send feedback of the selected object to the smart assistant through a communication channel. Accordingly, the smart assistant can obtain at least one recommended instruction corresponding to the selected object.

[0072] As another feasible approach, when the application determines that a selected object exists on the application page, it can feed back the identification information of the selected object to the smart assistant through a communication channel. Accordingly, the smart assistant can obtain at least one recommendation instruction corresponding to the selected object based on the identification information of the selected object.

[0073] As another feasible approach, when the application determines that a selected object exists on the application page, it can feed back the object type of the selected object to the smart assistant through a communication channel. Accordingly, the smart assistant can obtain at least one recommended instruction corresponding to the selected object based on the object type of the selected object.

[0074] Optionally, recommended commands may include "translate", "save to notes", "summarize", "batch export", "summarize one by one", "send via email", etc., without limitation.

[0075] In this embodiment, during the display of the application page, the user can select an object as the selected object from the application page through native in-app interactions. When the smart assistant is activated, the application can send information back to the smart assistant through a communication channel, so that the smart assistant can make corresponding instruction recommendations and processing. Currently, the smart assistant obtains information from the application in the following ways: the user extracts content from the application page by long-pressing the screen, drags it to a fixed area, and releases it to send it to the smart assistant; the user enters the screen information collection state of the smart assistant through gestures or the GUI, and then uses gestures such as circling, swiping, and drawing to indicate the location of the point of interest on the screen; the smart assistant simultaneously understands the user's natural language instructions and screen content through a multimodal big data model, and identifies and locates the user's point of interest. Therefore, it can be understood that the way the smart assistant obtains information from the application in this embodiment has at least the following advantages compared to the current way the smart assistant obtains information from the application:

[0076] First, it reduces the cognitive burden and operational costs for users: the proactive perception method that eliminates the need for circles and drags saves users from cumbersome operation steps, while also continuing the original interactions and user habits within the application, allowing users to interact with the intelligent assistant more naturally and smoothly, thus lowering the operational threshold and learning costs.

[0077] Secondly, the accuracy of referential reference has been improved: a communication channel for "user attention focus" has been successfully built between the intelligent assistant and the application, enabling fast, error-free and lossless data transmission between the two, which lays a solid foundation for subsequent accurate intent matching and efficient interaction.

[0078] Third, faster intent execution response: Once the intelligent system completes intent recognition, the interaction between the application and the intelligent assistant does not rely on large models or other algorithms, and the waiting time before intent execution can be greatly shortened.

[0079] Fourth, fewer privacy risks: The main data exchange occurs between the application and the local intelligent system. By leveraging user focus obtained from the application, the effective context is narrowed, and the minimal reliance on screen content means that the intelligent assistant does not need to redundantly collect and over-analyze any content on the screen that is irrelevant to the user's intent, thus minimizing the uploading and cloud processing of user data.

[0080] Step S130: Display the at least one recommended instruction.

[0081] In this embodiment, if at least one recommended instruction is determined to correspond to the selected object, at least one recommended instruction can be displayed so that the user can refer to at least one recommended instruction to issue the corresponding control instruction.

[0082] In some implementations, if at least one recommendation command is determined to correspond to the selected object, at least one recommendation command can be displayed floating on the application page. For example, at least one recommendation command can be displayed on the application page in the form of a "bubble".

[0083] In some implementations, when at least one recommended instruction corresponding to the selected object is determined, at least one recommended instruction can be displayed. In this case, the display position of the recommended instruction remains fixed. Alternatively, at least one recommended instruction can be displayed dynamically. In this case, the display position of the recommended instruction changes automatically or dynamically based on user operation, which is not limited here.

[0084] In some implementations, when at least one recommended instruction corresponding to the selected object is obtained, at least one recommended instruction can be displayed and restricted to not supporting touch-based navigation. In this case, the recommended instruction is only used as a prompt, and the user can refer to the recommended instruction to input more detailed control instructions. Alternatively, at least one recommended instruction can be displayed and restricted to supporting touch-based navigation. In this case, the recommended instruction can be used as a prompt, and the user can refer to the recommended instruction to input more detailed control instructions. It can also be used as a navigation control, and the user can trigger the corresponding control instructions by touching the recommended instruction.

[0085] In some implementations, when at least one recommendation instruction is obtained, all recommendation instructions in the at least one recommendation instruction may be displayed, or only some recommendation instructions in the at least one recommendation instruction may be displayed; there is no limitation on this.

[0086] As one feasible approach, the electronic device can pre-set and store an upper limit (e.g., 3) for the number of recommended instructions that can be displayed. If at least one recommended instruction corresponds to the selected object, the number of recommended instructions can be determined, and this number can be compared with the upper limit to determine whether the number of recommended instructions exceeds the upper limit.

[0087] If the number of recommended commands does not exceed the display limit, then all recommended commands can be displayed.

[0088] If the number of recommended commands exceeds the display limit, then a portion of the recommended commands, not exceeding the display limit, can be selected from all recommended commands for display. Optionally, the selection can be based on the relevance to the selected object from highest to lowest, or on usage rate from highest to lowest, etc., without limitation.

[0089] An embodiment of this application provides a method for recommending instructions. It displays an application page, and if a selected object is found on the application page, it retrieves at least one recommended instruction corresponding to the selected object to instruct on processing the selected object, and displays at least one recommended instruction. By establishing a communication channel with the application, it retrieves the selected object selected through native interaction on the application page from the application, and recommends instructions based on the selected object. This avoids considering too many factors, making it difficult to accurately target the user's specific intent, and improves the effectiveness of instruction recommendations.

[0090] Please refer to Figure 2, which shows a flowchart illustrating a method for recommending instructions according to an embodiment of this application. The flowchart shown in Figure 2 will be described in detail below. Specifically, the method for recommending instructions may include the following steps:

[0091] Step S210: Display the application page.

[0092] Step S220: If a selected object is found in the application page, then at least one recommendation instruction corresponding to the selected object is obtained, wherein the recommendation instruction is used to indicate the processing operation for the selected object.

[0093] Step S230: Display the at least one recommended instruction.

[0094] For a detailed description of steps S210-S230, please refer to steps S110-S1630, which will not be repeated here.

[0095] Step S240: After receiving a target instruction to process the selected object, the selected object is obtained from the application.

[0096] In this embodiment, during the display of at least one recommendation instruction, it is possible to detect whether an input instruction (user-input instruction) has been received. If an input instruction instructing the processing of a selected object is detected, the selected object can be retrieved from the application in response to the input instruction. If no user-input instruction instructing the processing of a selected object is detected, the display of at least one recommendation instruction can continue, while the detection of whether an input instruction has been received continues.

[0097] In some implementations, during the display of at least one recommendation instruction, it can be detected whether an input instruction has been received. If it is determined that an input instruction has been received, it can be determined whether the input instruction is related to the selected object. Specifically, if it is determined that the input instruction is related to the selected object, it is determined that a target instruction instructing the selected object to be processed has been received, and the selected object can be obtained from the application; if it is determined that the input instruction is not related to the selected object, it is determined that no target instruction instructing the selected object to be processed has been received, and the selected object does not need to be obtained from the application.

[0098] One approach is to segment user input commands to extract key information, such as "summarize" or "English," clarifying the meaning and focus of these keywords. "Summary" typically implies a condensed summary of a text, while "English" specifies the output language. The extracted keywords are then compared to the application's main functions and common content types. If the application is a text editor, document reader, or translation software, the presence of keywords like "summarize" or "English" is likely related to the current application's content. For example, in a document reader, users frequently summarize document content, making the command highly likely to be relevant to the application's content—that is, confirming that the input command is related to the selected object.

[0099] As another approach, natural language processing (NLP) techniques can be used to calculate the semantic similarity between user-inputted commands and the text content displayed on the application page. For example, word vector models and semantic role labeling (SLS) can be used to convert commands and text content into vector representations, and then a similarity score can be calculated between them. If the score exceeds a certain threshold, the command can be considered relevant to the current content; that is, the input command is determined to be relevant to the selected object.

[0100] Of course, in this embodiment, there are many other ways to determine whether the input instruction is related to the selected object, which will not be elaborated here.

[0101] In some implementations, during the display of at least one recommended instruction, if a touch operation is detected acting on any of the at least one recommended instruction, a target instruction is generated based on that touch operation. Optionally, the recommended instructions are limited to supporting touch navigation. Based on this, during the display of at least one recommended instruction, touch operations acting on any of the at least one recommended instruction can be detected. If a touch operation is detected acting on a certain recommended instruction among the at least one recommended instructions, it can be determined that the user expects to trigger processing for the selected object by touching that recommended instruction. Therefore, the recommended instruction corresponding to the touch operation among the at least one recommended instructions can be determined as the target instruction.

[0102] As an example, suppose there are three recommended commands displayed: "Translate", "Save to Notes", and "Summarize". If a touch operation is detected on the recommended command "Translate", then "Translate" can be identified as the target command; if a touch operation is detected on the recommended command "Save to Notes", then "Save to Notes" can be identified as the target command.

[0103] Optionally, the touch operations applied to the recommendation command may include: a click operation applied to the recommendation command, a press operation applied to the recommendation command, a swipe operation applied to the recommendation command, etc., without limitation. Specifically, the click operation applied to the recommendation command may include: a single-finger click operation applied to the recommendation command, a multi-finger click operation applied to the recommendation command, a single click operation applied to the recommendation command, a series of consecutive clicks applied to the recommendation command, etc.; the press operation applied to the recommendation command may include: a single-finger press operation applied to the recommendation command, a multi-finger press operation applied to the recommendation command, a short press operation applied to the recommendation command, a long press operation applied to the recommendation command, etc.; the swipe operation applied to the recommendation command may include: a swipe operation applied to the recommendation command along a specified direction applied to the recommendation command, a swipe operation applied to the recommendation command along a specified path applied to the recommendation command, a swipe operation applied to the recommendation command reaching a specified distance applied to the recommendation command, etc., without limitation.

[0104] In some implementations, if target voice information is received during the display of at least one recommended instruction, a target instruction is generated based on the target voice information. During the display of at least one recommended instruction, the user can also input instructions via voice. Therefore, during the display of at least one recommended instruction, the system can detect whether the user has input voice information. If target voice information is received, a target instruction can be generated based on that target voice information. If no target voice information is received, the system can continue displaying at least one recommended instruction while continuing to detect whether the user has input voice information.

[0105] As an example, suppose the display shows three recommended commands: "Translate", "Save to Notes", and "Summarize". If the target voice information of "Translate" is received from the user, the target command can be determined to be "Translate" based on the target voice information; if the target voice information of "Send to XX" is received from the user, the target command can be determined to be "Send to XX" based on the target voice information.

[0106] Optionally, the target instruction generated based on the user's input target speech information may or may not be related to the displayed recommended instruction. Specifically, if the displayed recommended instruction is the instruction the user expects, the user can refer to the recommended instruction to input the target speech information. In this case, the target instruction generated based on the target speech information is related to (consistent with) the recommended instruction. For example, if the recommended instruction is "Translate," the target instruction is "Translate." If the displayed recommended instruction is similar to but not detailed enough for the user's expected instruction, the user can refer to the recommended instruction to input the target speech information. In this case, the target instruction generated based on the target speech information is related to (similar to) the recommended instruction. For example, if the recommended instruction is "Translate," the target instruction is "Translate into English." If the displayed recommended instruction is different from the user's expected instruction, the user can input the target speech information. In this case, the target instruction generated based on the target speech information is unrelated to the recommended instruction. For example, if the recommended instruction is "Translate," the target instruction is "Send to XX."

[0107] In some implementations, during the display of at least one recommendation instruction, if target text information is received through an input box, a target instruction is generated based on the target text information. Optionally, when displaying at least one recommendation instruction, an input box may also be displayed simultaneously. This input box is used for users to input text information. Therefore, during the display of at least one recommendation instruction and the input box, it is possible to detect whether the user has entered text information through the input box. If target text information is received through the input box, a target instruction can be generated based on the target text information. If no text information is received through the input box, the display of at least one recommendation instruction and the input box can continue, while continuing to detect whether the user has entered text information through the input box. Optionally, the input box may be displayed floating on the application page.

[0108] As an example, suppose there are three recommended instructions displayed: "Translate", "Save to Notes", and "Summarize". If the target text information "Translate" is received from the user through the input box, the target instruction can be determined to be "Translate" based on the target text information. If the target text information "Send to XX" is received from the user through the input box, the target instruction can be determined to be "Send to XX" based on the target text information.

[0109] Optionally, the target instruction generated based on the user-input target text information may or may not be related to the displayed recommended instructions. Specifically, if the displayed recommended instructions are the instructions the user expects, the user can refer to the recommended instructions to input the target text information. In this case, the target instruction generated based on the target text information is related to (consistent with) the recommended instructions. For example, if the recommended instructions are "Translate it," the target instruction is "Translate it." If the displayed recommended instructions are similar to but not detailed enough for the user's expected instructions, the user can refer to the recommended instructions to input the target text information. In this case, the target instruction generated based on the target text information is related to (similar to) the recommended instructions. For example, if the recommended instructions are "Translate it," the target instruction is "Translate into English." If the displayed recommended instructions are different from the user's expected instructions, the user can input the target text information. In this case, the target instruction generated based on the target text information is unrelated to the recommended instructions. For example, if the recommended instructions are "Translate it," the target instruction is "Send to XX."

[0110] In some implementations, if the target instruction is generated based on target text information—that is, the user triggers the generated target instruction by inputting target text information through an input box—then, during the process of receiving the input text information through the input box, the input text information can be determined, and at least one recommended instruction can be updated based on the input text information. Since it takes a certain amount of time for the user to input the desired target text information through the input box, the input text information can be determined during the user's text input process, and at least one recommended instruction can be updated based on the input text information. The updated recommended instruction is related to the input text information, ensuring that the displayed recommended instruction matches the user's input text information, thus improving the accuracy and adaptability of the recommended instruction.

[0111] As an example, when the user does not enter text information through the input box, three recommended instructions are displayed: "Translate", Instruction A, and Instruction B. After the user enters the text information "Translate" through the input box, it means that the intent has been locked to translation-related, and Instruction A and Instruction B are no longer appropriate. At this time, "Translate this passage" can be further subdivided into three recommended instructions: "Translate to Chinese", "Translate to French", and "Translate to Japanese" to further simplify the user's subsequent operations.

[0112] Please refer to Figure 3, which shows a flowchart of step S240 of the recommended method of the instructions shown in Figure 2 of this application. The flowchart shown in Figure 3 will be described in detail below, and the method specifically includes the following steps:

[0113] Step S241: Obtain the page type of the application page from the application.

[0114] In some implementations, the smart assistant can query the application for the page type of the application page through a communication channel with the application. Accordingly, the application can determine the page type of the application page and send it back to the smart assistant through the communication channel. That is, the smart assistant can obtain the page type of the application page from the application.

[0115] As an feasible approach, when an application receives a query from a smart assistant regarding the page type of an application page, it can determine the page type through a routing system and then report this page type back to the smart assistant. In web front-end frameworks (such as React, Angular, and Vue), a routing system is typically used to manage navigation between pages. Each route can be associated with a specific component or page, and by checking the currently active route, the application can determine the page type of the currently displayed application page. In mobile applications (such as iOS and Android applications), page navigation is usually managed by navigation controllers or view controllers. Each view controller can represent a page type, and the application can determine the page type by checking the currently displayed view controller.

[0116] As another feasible approach, when the application receives a query from the intelligent assistant regarding the page type of an application page, it can determine the page type by examining page components or views, and then report the page type back to the intelligent assistant. In a component-based development model, a page consists of multiple components or views, and the application can determine the page type by inspecting the currently rendered components or views.

[0117] Of course, in this embodiment, the application can also determine the page type of the application page in other ways, which will not be elaborated here.

[0118] Optionally, the page type may include list page type, detail page type, video page type, etc., and there is no limitation here.

[0119] Step S242: Obtain the selected object from the application based on the page type.

[0120] In some implementations, if the page type of the application page is obtained, the selected object can be obtained from the application based on the page type of the application page.

[0121] As one feasible approach, upon obtaining the page type of the application page, an object acquisition method can be determined based on that page type, and the selected object can be retrieved from the application using that object acquisition method. Optionally, the electronic device can pre-set and store a first mapping relationship, which may include multiple page types, multiple acquisition methods, and a correspondence between multiple page types and multiple acquisition methods. The correspondence between multiple page types and multiple acquisition methods may include: one page type corresponding to one acquisition method, multiple page types corresponding to one acquisition method, etc., and is not limited here. Therefore, in this embodiment, upon obtaining the page type of the application page, an acquisition method corresponding to the page type of the application page can be determined based on the first mapping relationship, and the selected object can be retrieved from the application using that acquisition method.

[0122] In some implementations, if the page type is a list page, the selected object can be obtained from the application as the selected object. As one approach, if the smart assistant obtains the page type from the application and determines it to be a list page, it can request the selected object from the application via a communication channel. The application can then determine if the selected object exists on the page. If it does, it can provide the selected object to the smart assistant via the communication channel, allowing the smart assistant to obtain it as the selected object. If it does not exist, an error message can be displayed. For example, the application can report to the smart assistant via the communication channel that the selected object does not exist on the page, and the smart assistant can output an error message to improve the user's object selection process.

[0123] In some implementations, if the page type is a detail page, the object selected by box selection on the application page can be obtained from the application as the selected object. As one approach, if the smart assistant obtains the page type from the application and determines that it is a detail page, it can request the object selected by box selection from the application through a communication channel. Correspondingly, the application can determine whether the object selected by box selection exists on the application page. If it does, it can provide the object selected by box selection on the application page to the smart assistant through the communication channel, and the smart assistant can then obtain the object selected by box selection from the application as the selected object.

[0124] As one feasible approach, the intelligent assistant obtains the page type of the application page from the application and, if it determines that the page type is a details page, it can determine whether the target instruction declares a processing scope for the application page. If the target instruction does not declare a processing scope for the application page, the object selected by a selection box on the application page can be obtained from the application as the selected object; if the target instruction declares that the processing scope includes the entire application page, the page content of the application page can be obtained from the application for subsequent processing.

[0125] In some implementations, if the target instruction does not declare the processing scope for the application page, the object selected by the box selection method can be requested from the application through a communication channel. Accordingly, the application can determine whether the object selected by the box selection method exists on the application page. If it exists, the object selected by the box selection method on the application page can be provided to the intelligent assistant through the communication channel, and the intelligent assistant can obtain the object selected by the box selection method from the application as the selected object. If it does not exist, the entire page content of the application page can be provided to the intelligent assistant through the communication channel, and the intelligent assistant can obtain the entire page content from the application for processing.

[0126] Step S250: Process the selected object based on the target instruction.

[0127] In this embodiment, when the selected object is obtained from the application, it can be processed based on the target instruction to obtain the processing result for the selected object. Optionally, if the target instruction is "translate to Chinese", the selected object can be translated into Chinese; if the target instruction is "save as a note", the selected object can be saved as a note, and this is not limited here.

[0128] In some implementations, during the processing of the selected object based on the target instruction, a dialog page can be displayed, showing the processing result for the selected object. As another possible approach, while displaying the processing result for the selected object on the dialog page, the selected object and the target instruction instructing the processing of the selected object can also be displayed. Here, the selected object and the target instruction instructing the processing of the selected object constitute one side of the dialog, while the processing result for the selected object is displayed as the other side.

[0129] As one feasible approach, the chat page can be overlaid on the application page. In this case, the chat page can completely cover the application page, or it can partially cover the application page; this is not a definitive answer.

[0130] As another feasible approach, one could switch from displaying an application page to displaying a dialog page.

[0131] Please refer to Figure 4, which shows a flowchart of step S250 of the recommended method of the instructions shown in Figure 2 of this application. The flowchart shown in Figure 4 will be described in detail below, and the method specifically includes the following steps:

[0132] Step S251: Perform intent analysis on the target instruction to obtain the processing intent corresponding to the target instruction.

[0133] In some implementations, upon receiving an input target instruction, intent analysis can be performed on the target instruction to obtain the processing intent corresponding to the target instruction.

[0134] As a feasible approach, upon receiving a target instruction, it can be parsed to extract key information such as operation type, target, and parameters. Building upon this parsing, Natural Language Processing (NLP) techniques, such as lexical analysis, syntactic analysis, and semantic role labeling, can be used to identify entities, actions, and relationships within the target instruction, thereby understanding its semantic content. Based on the semantic understanding results, the processing intent behind the target instruction can be identified by matching predefined intent templates and using machine learning models for intent classification. Of course, when processing complex instructions, the context in which the target instruction appears can be considered, such as the user's historical input and the current application state, to more accurately understand the processing intent corresponding to the target instruction.

[0135] Step S252: Process the selected object according to the processing intention.

[0136] In this embodiment, if the processing intent corresponding to the target instruction is obtained, the selected object can be processed according to the processing intent.

[0137] As an feasible approach, once the processing intent corresponding to the target instruction is obtained, the processing intent can be mapped to a specific processing operation or processing function based on a mapping table or rule set. Then, based on the parameters and context information provided in the target instruction, the parameters required by the processing operation or processing function can be configured through steps such as data type conversion, default value filling, and parameter validation. Finally, the selected object can be processed through the processing operation or processing function with the configured parameters.

[0138] An embodiment of this application provides a method for recommending instructions. It displays an application page, and if a selected object is found on the application page, it retrieves at least one recommended instruction corresponding to the selected object. The recommended instruction instructs on processing the selected object. At least one recommended instruction is displayed. Upon receiving a target instruction instructing processing of the selected object, the selected object is retrieved from the application, and processing is performed on the selected object based on the target instruction. Compared to the instruction recommendation method shown in Figure 1, this embodiment also establishes a communication channel with the application to retrieve and process the selected object from the application page, achieving fast, error-free, and lossless data transmission and improving data processing efficiency.

[0139] Please refer to Figure 5, which shows a flowchart illustrating a method for recommending instructions according to an embodiment of this application. The flowchart shown in Figure 5 will be described in detail below. Specifically, the method for recommending instructions may include the following steps:

[0140] Step S310: Display the application page.

[0141] For a detailed description of step S310, please refer to step S110, which will not be repeated here.

[0142] Step S320: If a selected object is found in the application page, then the object type of the selected object is obtained from the application.

[0143] In this embodiment, a query message can be sent to the application via a communication channel. This query message inquires whether a selected object exists on the application page. Upon confirming the existence of a selected object, the application provides feedback on the object type of the selected object. Therefore, the application can provide feedback on the object type of the selected object via this communication channel when it determines that a selected object exists on the application page. Based on this, if the application confirms the existence of a selected object on the application page, the application can then retrieve the object type of that selected object.

[0144] Optionally, if the selected object is a piece of text, the object type of the selected object can be determined as "text type"; if the selected object is a list, the object type of the selected object can be determined as "list type"; if the selected object is an image, the object type of the selected object can be determined as "image type", etc., without limitation.

[0145] Step S330: Determine the at least one recommendation instruction based on the object type, wherein the recommendation instruction is used to indicate the processing operation for the selected object.

[0146] In this embodiment, if the object type of the selected object is obtained, at least one recommended instruction can be determined based on the object type of the selected object.

[0147] In some implementations, the electronic device may pre-set and store a second mapping relationship, which may include multiple object types, multiple recommendation instructions, and a correspondence between multiple object types and multiple recommendation instructions. The correspondence between multiple object types and multiple recommendation instructions may include: one object type corresponding to one recommendation instruction, multiple object types corresponding to one recommendation instruction, one object type corresponding to multiple recommendation instructions, etc., and is not limited thereto. Therefore, in this embodiment, when the object type of the selected object is obtained, at least one recommendation instruction corresponding to the object type of the selected object can be determined based on the second mapping relationship.

[0148] Optionally, if the selected object is a text type, the recommended instructions for the selected object may include "Translate", "Save to Notes", and "Summarize"; if the selected object is a list type, the recommended instructions for the selected object may include "Batch Export", "Summarize One by One", and "Send via Email", without limitation.

[0149] Step S340: Display the at least one recommended instruction.

[0150] For a detailed description of step S340, please refer to step S130, which will not be repeated here.

[0151] An embodiment of this application provides a method for recommending instructions. It displays an application page, and if a selected object is found on the application page, it retrieves the object type of the selected object from the application and determines at least one recommended instruction based on the object type. The recommended instruction instructs on processing operations for the selected object, and at least one recommended instruction is displayed. Compared to the instruction recommendation method shown in Figure 1, this embodiment further retrieves the object type of the selected object from the application and determines at least one recommended instruction based on the object type. This allows the recommended instruction to be compatible with the object type of the selected object, improving the accuracy of the recommended instructions.

[0152] Please refer to Figure 6, which shows a flowchart illustrating a method for recommending instructions according to an embodiment of this application. The flowchart shown in Figure 6 will be described in detail below. Specifically, the method for recommending instructions may include the following steps:

[0153] Step S410: Display the application page.

[0154] For a detailed description of step S410, please refer to step S110, which will not be repeated here.

[0155] Step S420: After receiving a wake-up command to instruct the smart assistant to be woken up, wake up the smart assistant and determine from the application page whether there is a selected object.

[0156] Optionally, the smart assistant of an electronic device can be in a closed or dormant state by default. That is, while the electronic device is displaying an application page and the user is selecting an object from the application page through native in-app interactions, the smart assistant may still be in a closed or dormant state. During this process, the user can wake up the smart assistant via a wake-up command. At this time, the electronic device can respond to the smart assistant wake-up command to wake up the smart assistant and determine from the application page whether the selected object exists, so that the user can interact with the smart assistant more naturally and smoothly, reducing the operation threshold and learning cost.

[0157] In some implementations, when an electronic device receives a wake-up command to instruct the smart assistant to be woken up, it can wake up the smart assistant in response to the wake-up command, triggering the smart assistant to send a query message to the application to inquire whether there is a selected object on the application page, thereby determining from the application whether there is a selected object on the application page.

[0158] In some implementations, users can trigger the smart assistant to wake up via voice, touch operation, or shaking operation, etc., without limitation.

[0159] Step S430: If the selected object is found to exist on the application page from the application, then at least one recommendation instruction corresponding to the selected object is obtained through the smart assistant, wherein the recommendation instruction is used to indicate the processing operation of the selected object.

[0160] Step S440: Display the at least one recommended instruction.

[0161] For a detailed description of steps S430-S440, please refer to steps S120-S130, which will not be repeated here.

[0162] This application provides an embodiment of a method for recommending instructions. Upon receiving a wake-up instruction to activate a smart assistant, the method activates the smart assistant and uses the smart assistant to determine whether a selected object exists on the application page. If a selected object is found, the smart assistant determines at least one recommended instruction corresponding to the selected object. This recommended instruction instructs on processing operations for the selected object, and at least one recommended instruction is displayed. Compared to the instruction recommendation method shown in Figure 1, this embodiment also retrieves and processes the selected object from the application page when the smart assistant is activated, allowing for smoother and more natural interaction between the user and the smart assistant, thus improving the user experience.

[0163] The embodiments of this application will be described below using two specific scenarios as examples.

[0164] Please refer to Figures 7-9, which illustrate three interface diagrams in a scenario where the application page type of the electronic device provided in this application embodiment is a details page. First, during the display of the application page, the user selects a text paragraph in the application page using the application's native text selection function (such as drag-and-drop) (as shown in Figure 7). Then, when the smart assistant is activated by the user, it learns from the application that the user has selected a text paragraph, so functions related to "translate it," "save it as a note," and "summarize it" are recommended as high-priority instructions (as shown in Figure 8). Afterward, the electronic device receives the user's target instruction "summarize this paragraph in English," determines that it is related to the current application content, queries the application for the data of the object referred to by the user, and prioritizes extracting the selected content (user focus) as the data input for the intended execution (as shown in Figure 9).

[0165] Please refer to Figures 10-13, which illustrate four interface diagrams in a scenario where the application page type of the electronic device provided in this application embodiment is a list page. First, during the display of the application page, the user selects one or more items from the list through the native checkbox interaction within the application (as shown in Figure 10); then, when the smart assistant is activated by the user, it learns from the application that the user has selected multiple rich text objects (such as documents), so functions related to "batch export", "summarize one by one", and "send via email" are recommended as high-priority instructions (as shown in Figure 11); after that, the electronic device receives the target instruction "summarize these files" input by the user, determines that it is related to the current application content, queries the application for the data of the object referred to by the user, and prioritizes extracting the selected content (user focus) as the data input for the intended execution (as shown in Figures 12 and 13).

[0166] The following section will further describe the "instruction recommendation" scenario involved in the aforementioned embodiments, in conjunction with the timing shown in Figure 14.

[0167] As shown in Figure 14, when the user wakes up the smart assistant, it triggers a query to the application to see if there is a selected object in the currently displayed application page. The application responds by determining whether there is a selected object, and if it is determined that there is a selected object, it returns the object type of the selected object, translates the object type of the selected object into recommendation parameters, combines the parameters to sort and match instructions, and outputs recommendation instructions to the smart assistant, so that the user can browse the recommendation instructions and perform subsequent interactions.

[0168] The following section will describe the scenario of acquiring and processing the "selected object" involved in the aforementioned embodiments, in conjunction with the timing shown in Figure 15.

[0169] As shown in Figure 15, user input triggers intent analysis to determine if the target instruction contains a reference (explicit / implicit) to process the application page. If intent analysis determines that a reference exists, it can detect whether the user selected an object by swiping or manually adding it. If the user selected an object by swiping or manually adding it, it can determine whether the selected object is an object contained in the application page. If so, it can query the application for the currently displayed page type. The application responds and returns the page type of the currently displayed application page, and then determines whether the currently displayed application page is a list page type or a details page type. If the currently displayed application page is determined to be a list page type, it can request the object selected by swiping from the application. The application responds and determines whether the object selected by swiping exists in the application page. If it exists, it provides the object selected by swiping for subsequent processing; if it does not exist, it provides an error message. If the currently displayed application page is determined to be a details page, it can be determined whether the target instruction declares a processing scope. If the target instruction declares the processing scope as the entire file, the entire application page can be requested from the application. The application responds by providing the page content for subsequent processing. If the target instruction does not declare a processing scope, the object selected by the selection box method is requested from the application. The application responds by determining whether the object selected by the selection box method exists on the application page. If it exists, the object selected by the selection box method is provided for subsequent processing; if it does not exist, the page content is provided for subsequent processing. Then, during processing, necessary format conversions can be performed on the data returned by the application, returning object data, the entire object data, and the intent. Finally, the intent is understood, analyzed, and implemented to obtain the final processing result, which is then displayed.

[0170] Please refer to Figure 16, which shows a block diagram of an instruction recommendation device according to an embodiment of this application. The block diagram shown in Figure 16 will be described below. The instruction recommendation device 200 includes: an application page display module 210, a recommendation instruction acquisition module 220, and a recommendation instruction display module 230, wherein:

[0171] Application page display module 210 is used to display the application page.

[0172] The recommendation instruction acquisition module 220 is used to acquire at least one recommendation instruction corresponding to the selected object if the selected object is found to exist on the application page from the application. The recommendation instruction is used to indicate the processing operation for the selected object.

[0173] Furthermore, the recommendation instruction acquisition module includes: an object type acquisition submodule and a first recommendation instruction acquisition submodule, wherein:

[0174] The object type acquisition submodule is used to obtain the object type of the selected object from the application.

[0175] The first recommendation instruction acquisition submodule is used to determine the at least one recommendation instruction based on the object type.

[0176] Furthermore, the recommendation instruction acquisition module includes: a second recommendation instruction acquisition submodule, wherein:

[0177] The second recommendation instruction acquisition submodule is used to acquire at least one recommendation instruction corresponding to the selected object through the smart assistant if the selected object is found to exist on the application page from the application.

[0178] The recommendation instruction display module 230 is used to display the at least one recommendation instruction.

[0179] Furthermore, the recommendation device 200 for the instruction also includes: a selected object acquisition module and an object content processing module, wherein:

[0180] The selected object acquisition module is used to acquire the selected object from the application after receiving a target instruction that instructs the selected object to be processed.

[0181] Furthermore, the selected object acquisition module includes: a page type acquisition submodule and a selected object acquisition submodule, wherein:

[0182] The page type acquisition submodule is used to obtain the page type of the application page from the application.

[0183] The selected object retrieval submodule is used to retrieve the selected object from the application based on the page type.

[0184] Further, the selected object acquisition submodule includes: a first object acquisition unit and a second object acquisition unit, wherein:

[0185] The first object acquisition unit is used to acquire, if the page type is a list page type, the object selected by a check mark in the application page as the selected object.

[0186] The second object acquisition unit is used to acquire, if the page type is a details page type, the object selected by box selection in the application page as the selected object.

[0187] Further, the second object acquisition unit includes: an object acquisition subunit, wherein:

[0188] The object acquisition subunit is used to acquire, if the page type is a details page type and the target instruction does not declare the processing scope for the application page, the object selected by box selection in the application page as the selected object.

[0189] Furthermore, the second object acquisition unit includes: a page content acquisition subunit, wherein:

[0190] The page acquisition subunit is used to acquire the page content of the application page from the application if the page type is a details page type and the target instruction declaration processing scope includes the entire application page.

[0191] The selected object processing module is used to process the selected object based on the target instruction.

[0192] Furthermore, the selected object processing module includes: a processing intent acquisition submodule and a selected object processing submodule, wherein:

[0193] The processing intent acquisition submodule is used to perform intent analysis on the target instruction and obtain the processing intent corresponding to the target instruction.

[0194] The selected object processing submodule is used to process the selected object according to the processing intention.

[0195] Furthermore, the instruction recommendation device 200 further includes: a first target instruction generation module, a second target instruction generation module, and a third target instruction generation module, wherein:

[0196] The first target instruction generation module is configured to generate the target instruction based on the touch operation if a touch operation is detected acting on any of the at least one recommended instruction.

[0197] The second target instruction generation module is used to generate the target instruction based on the target voice information if the input target voice information is received.

[0198] The third target instruction generation module is used to generate the target instruction based on the target text information if target text information is received through the input box.

[0199] Furthermore, the target instruction is generated based on the text information, and the instruction recommendation device 200 further includes: an input text information determination module and a recommendation instruction update module, wherein:

[0200] The module for determining entered text information is used to determine the entered text information during the process of receiving entered text information through the input box.

[0201] The recommendation instruction update module is used to update the at least one recommendation instruction based on the input text information.

[0202] Furthermore, the instruction recommendation device 200 further includes: a dialog page display submodule, wherein:

[0203] The dialog page display submodule is used to display the dialog page and show the processing results for the selected object on the dialog page.

[0204] Furthermore, the dialog page display submodule includes: a dialog page display unit, wherein:

[0205] A dialog page display unit is used to overlay the dialog page on the application page.

[0206] Furthermore, the recommendation device 200 for the instruction also includes: a selection judgment module, wherein:

[0207] The selection judgment module is used to wake up the smart assistant after receiving a wake-up command that instructs the smart assistant to wake up, and to determine from the application page whether the selected object exists through the smart assistant.

[0208] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0209] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0210] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0211] Please refer to Figure 17, which shows a structural block diagram of an electronic device 100 provided in an embodiment of this application. The electronic device 100 can be a smartphone, tablet computer, e-reader, or other electronic device capable of running applications. The electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications can be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0212] The processor 110 may include one or more processing cores. The processor 110 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and by calling data stored in the memory 120. Optionally, the processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 110 and may be implemented separately using a communication chip.

[0213] The memory 120 may include random access memory (RAM) or read-only memory (ROM). The memory 120 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing functions (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0214] In some embodiments, the electronic device 100 may further include a touch screen, which is used to display information input by the user, information provided to the user, and various graphical user interfaces of the electronic device 100. These graphical user interfaces may be composed of graphics, text, icons, numbers, video, and any combination thereof. In one example, the touch screen may be a liquid crystal display (LCD) or an organic light-emitting diode (OLED), without limitation.

[0215] In some embodiments, the electronic device 100 may further include a camera for collecting user motion data. Optionally, the camera may include a front-facing camera, a rear-facing camera, a telescopic camera, a rotating camera, etc., and is not limited thereto.

[0216] In some embodiments, the electronic device 100 may further include sensors, including a light sensor that can be used to turn off the touchscreen display when an object approaches the touchscreen, such as when the body of the electronic device is moved to the ear. The sensor may also include a pressure sensor that can detect pressure generated by pressing on the electronic device; that is, the pressure sensor can detect pressure generated by contact or pressing between the user and the electronic device, such as pressure generated by contact or pressing between the user's ear or finger and the electronic device. The sensor may also include a gravity sensor that can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. This can be used for applications that identify the posture of the electronic device (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. Additionally, the electronic device may also be equipped with other sensors such as gyroscopes, barometers, and hygrometers, which are not limited here.

[0217] In some embodiments, the electronic device 100 may also include an artificial intelligence module, which may be integrated into the processor 110 of the electronic device 100 to improve the intelligence level and performance of the electronic device.

[0218] Of course, electronic devices may include many other components, which will not be elaborated here.

[0219] Please refer to Figure 18, which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 300 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0220] The computer-readable storage medium 300 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 300 has storage space for program code 310 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 310 may be compressed, for example, in a suitable form.

[0221] In some embodiments, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the recommended method of the instructions in the embodiments of this application.

[0222] In summary, the instruction recommendation method, apparatus, electronic device, storage medium, and program product provided in this application embodiment display an application page. If a selected object is obtained from the application page, at least one recommendation instruction corresponding to the selected object for instructing processing operations on the selected object is obtained and displayed. The selected object obtained from the application can be selected through native interaction within the application, thus accurately determining the recommendation instruction based on the selected object without needing to refer to the context of the application page. This avoids considering too many factors, making it difficult to accurately hit the user's specific intent, and improves the effectiveness of instruction recommendation.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for recommending instructions, wherein, The method includes: Display the application page; If a selected object is found in the application page, at least one recommendation instruction corresponding to the selected object is obtained, wherein the recommendation instruction is used to indicate the processing operation for the selected object; Display the at least one recommendation instruction.

2. The method according to claim 1, wherein, Following the display of the at least one recommendation instruction, the method further includes: Upon receiving a target instruction instructing the selected object to be processed, the selected object is retrieved from the application. The selected object is processed based on the target instruction.

3. The method according to claim 2, wherein, Before retrieving the selected object from the application after receiving a target instruction instructing processing of the selected object, the method further includes: If a touch operation is detected acting on any of the at least one recommended instruction, then the target instruction is generated based on the touch operation; or If the target voice information is received, the target instruction is generated based on the target voice information; or If target text information is received through the input box, the target instruction is generated based on the target text information.

4. The method according to claim 3, wherein, The target instruction is generated based on the target text information, and the method further includes: During the process of receiving input text information through the input box, the already input text information is obtained; The at least one recommendation instruction is updated based on the input text information.

5. The method according to claim 2, wherein, The method further includes: Display a dialog page and show the processing results for the selected object on the dialog page.

6. The method according to claim 5, wherein, The dialog page includes: The dialog page is overlaid on the application page.

7. The method according to claim 2, wherein, Retrieving the selected object from the application includes: Obtain the page type of the application page from the application; The selected object is obtained from the application based on the page type.

8. The method according to claim 7, wherein, The step of retrieving the selected object from the application based on the page type includes: Based on the page type, determine the object acquisition method; The selected object is obtained from the application according to the object acquisition method described above.

9. The method according to claim 7, wherein, The step of retrieving the selected object from the application based on the page type includes: If the page type is a list page type, then the object selected by checking the box on the application page is obtained from the application as the selected object; If the page type is a details page, then the object selected by box selection on the application page is obtained from the application as the selected object.

10. The method according to claim 9, wherein, If the page type is a details page, then obtaining the object selected by a selection box on the application page from the application as the selected object includes: If the page type is a details page and the target instruction does not declare the processing scope for the application page, then the object selected by box selection in the application page is obtained from the application as the selected object.

11. The method according to claim 10, wherein, The method further includes: If the page type is a details page and the target instruction declares that the processing scope includes the entire application page, then the page content of the application page is obtained from the application.

12. The method according to any one of claims 2-11, wherein, The process of processing the selected object based on the target instruction includes: The target instruction is subjected to intent analysis to obtain the processing intent corresponding to the target instruction; The selected object is processed according to the stated processing intent.

13. The method according to any one of claims 1-11, wherein, The step of obtaining at least one recommendation instruction corresponding to the selected object includes: Obtain the object type of the selected object from the application; The at least one recommendation instruction is determined based on the object type.

14. The method according to any one of claims 1-11, wherein, Before obtaining at least one recommendation instruction related to the selected object if the selected object is found to exist on the application page, the method further includes: Upon receiving a wake-up command instructing the smart assistant to be activated, the smart assistant is activated, and the smart assistant determines from the application whether a selected object exists on the application page. If a selected object is found on the application page, then at least one recommendation instruction corresponding to the selected object is obtained, including: If the selected object is found to exist on the application page, then at least one recommendation instruction corresponding to the selected object is obtained through the smart assistant.

15. The method according to any one of claims 1-11, wherein, The display of the at least one recommendation instruction includes: The at least one recommendation instruction is displayed in a floating position on the application page.

16. The method according to any one of claims 1-11, wherein, The display of the at least one recommendation instruction includes: Determine the number of recommended instructions; If the number of recommended instructions exceeds the upper limit, then a portion of the recommended instructions, not exceeding the upper limit, are selected from all recommended instructions in descending order of relevance to the selected object for display; or If the number of recommended instructions exceeds the upper limit, then a portion of the recommended instructions, whose number does not exceed the upper limit, will be selected from all recommended instructions in descending order of usage rate for display.

17. A device for recommending instructions, wherein, The device includes: The application page display module is used to display the application page; The recommendation instruction acquisition module is used to acquire at least one recommendation instruction corresponding to the selected object if the selected object is found to exist on the application page from the application. The recommendation instruction is used to indicate the processing operation for the selected object. A recommendation instruction display module is used to display the at least one recommendation instruction.

18. An electronic device, wherein, The method includes a memory and a processor, the memory being coupled to the processor, the memory storing instructions, and when the instructions are executed by the processor, the processor performing the method as described in any one of claims 1-16.

19. A computer-readable storage medium, wherein, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-16.

20. A computer program product, wherein, The computer program product includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-16.

Images