Information processing method and apparatus, electronic device, and medium

Abstract

The application discloses an information processing method and device, electronic equipment and medium, and relates to the technical field of terminals. The method comprises the following steps: in the case of displaying a reading interface, a first input is received, the first input being used for determining first interaction information; wherein the reading interface comprises first reading information, and the first reading information has an association relationship with the first interaction information; based on the first interaction information and context information corresponding to the first reading object, first reply information is displayed on the reading interface.

Description

Technical Field

[0001] This application belongs to the field of terminal technology, specifically relating to an information processing method, apparatus, electronic device, and medium. Background Technology

[0002] With the development of terminal technology, electronic devices are becoming increasingly feature-rich. For example, users can use electronic devices to read articles, novels, information, news, and share content on social media platforms.

[0003] When reading on an electronic device, if a user encounters content they don't understand, they can trigger the device to call up an AI (Artificial Intelligence) model. The user then manually inputs the content of the currently read document or article, along with any questions they wish to ask, into the AI ​​model. The AI ​​model then combines this information with the user's input to provide an answer. After reviewing the answer, the user manually switches back to the original reading interface. However, this process is cumbersome and results in a poor reading experience because users must switch between the current reading interface and the AI ​​model interface, and manually input the content and questions. Summary of the Invention

[0004] The purpose of this application is to provide an information processing method, apparatus, electronic device, and medium to improve the convenience and accuracy of searching for information during the reading process.

[0005] In a first aspect, embodiments of this application provide an information processing method, the method comprising: receiving a first input when a reading interface is displayed, the first input being used to determine first interactive information; wherein the reading interface includes first reading information, the first reading information being associated with the first interactive information; and displaying first response information on the reading interface based on the first interactive information and context information corresponding to the first reading information.

[0006] Secondly, embodiments of this application provide an information processing apparatus, including: a receiving module and a display module; the receiving module is configured to receive a first input when a reading interface is displayed, the first input being used to determine first interactive information; wherein, the reading interface includes first reading information, the first reading information being associated with the first interactive information; the display module is configured to display first response information on the reading interface based on the first interactive information received by the receiving module and context information corresponding to the first reading information.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0011] In this embodiment of the application, when the reading interface is displayed, after the user determines the first interactive information through the first input, the first reading information associated with the first interactive information in the reading interface can be automatically locked, and the first reply information can be displayed (such as overlay display or floating display) on the reading interface based on the first interactive information and the context information corresponding to the first reading information. That is, the user does not need to manually lock the first reading information and its corresponding context information, thus simplifying the convenience of information query, retrieval and extension operations during the reading process, and improving the convenience and experience of reading. Attached Figure Description

[0012] Figure 1 This is a flowchart of the information processing method provided in the embodiments of this application;

[0013] Figures 2A-2D This is a schematic diagram of the interface for applying the information processing method provided in the embodiments of this application;

[0014] Figures 3A-3D This is a schematic diagram of the interface for applying the information processing method provided in the embodiments of this application;

[0015] Figures 4A-4B This is a schematic diagram of the interface for applying the information processing method provided in the embodiments of this application;

[0016] Figures 5A-5B This is a schematic diagram of the interface for applying the information processing method provided in the embodiments of this application;

[0017] Figures 6A-6B This is a schematic diagram of the interface for applying the information processing method provided in the embodiments of this application;

[0018] Figures 7A-7B This is a schematic diagram of the interface for applying the information processing method provided in the embodiments of this application;

[0019] Figure 8 This is a schematic diagram of the structure of the information processing device provided in the embodiments of this application;

[0020] Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0021] Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in this application's specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0024] The terms "at least one," "at least one," etc., used in this application's specification refer to any one, any two, or a combination of two or more of the included objects. For example, "at least one of a, b, and c" can mean "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more, and its meaning is similar to that of "at least one."

[0025] The information processing provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings, through specific examples and application scenarios.

[0026] The information processing method, apparatus, electronic device, and medium provided in this application can be applied to reading scenarios. These reading scenarios may include, but are not limited to, at least one of the following:

[0027] Document reading scenarios: PPT, PDF, WORD, etc.;

[0028] Information-related scenarios: financial news, public opinion news, public accounts, blogs, etc.;

[0029] Content sharing communities: Dating communities, travel sharing communities, shopping sharing communities, etc. within dating apps;

[0030] Reading-related scenarios include: popular novels, classic works, biographies, essays, and ancient poems.

[0031] The information processing method, apparatus, electronic device, and medium provided in this application embodiment, when displaying a reading interface, can automatically lock the first reading information associated with the first interactive information in the reading interface after the user determines the first interactive information through the first input. Based on the first interactive information and the context information corresponding to the first reading information, the first reply information is displayed on the reading interface (such as overlay display or floating display). That is, the user does not need to manually lock the first reading information and its corresponding context information. Therefore, it can simplify the convenience of information query, retrieval, and extension operations during the reading process, and improve the reading convenience and experience.

[0032] The information processing method, apparatus, electronic device, and medium provided in this application embodiment can enable users to ask questions and obtain answers at any time during the reading process, thereby facilitating the associative reading and learning of the content in the reading interface.

[0033] The information processing method provided in this application is executed by an information processing device, which can be an electronic device, or a functional module or entity within an electronic device. This application does not limit the specific implementation of this method. The following will use an information processing device as an example to illustrate the information processing method provided in this application.

[0034] This application provides an information processing method, which may include the following steps 101 and 102.

[0035] Step 101: The information processing device receives the user's first input while displaying the reading interface.

[0036] The first input mentioned above is used to determine the first interactive information. The reading interface mentioned above may include the first reading information, and the first reading information is related to the first interactive information.

[0037] In some embodiments of this application, the aforementioned "the first reading information and the first interaction information are related" includes at least one of the following:

[0038] The first reading information is the reading information associated with the first interactive information in the reading interface;

[0039] The first reading information is the same as the first interactive information in the reading interface;

[0040] The first reading information is the reading information summarized from the first interaction information in the reading interface.

[0041] In some embodiments of this application, the aforementioned "reading interface includes first reading information, which can be understood as at least one of the following:"

[0042] The reading information currently displayed on the reading interface includes the first reading information;

[0043] The reading history displayed on the reading interface includes the first reading information;

[0044] The reading information preloaded in the reading interface includes the first reading information.

[0045] In some embodiments of this application, the aforementioned reading interface can be an interface that displays reading information from a first reading object. The first reading object can be any type of reading object, such as a file, image, document, e-book, post, news, or article.

[0046] For example, the primary reading material is PPT, PDF, or WORD documents.

[0047] For example, the primary reading material can be information, such as financial news, public opinion news, WeChat official accounts, blogs, etc.

[0048] For example, the primary reading target could be a content sharing community: a dating app's social community, a travel sharing community, a shopping sharing community, etc.

[0049] For example, the first reading material can be any possible e-book, such as popular novels, classics, biographies, essays, or ancient poems.

[0050] For example, the first reading material can be academic papers, journals, magazines, textbooks, etc.

[0051] In some embodiments of this application, the first input in mode 1 can be the input of the first interactive information by the user through the input method; or, the first input in mode 2 can be the input of the user's selection of reading information in the reading interface; the first input in mode 3 can be the input of the user's input of the recommendation icon displayed on the reading interface, such as the first input of the user's input of the first recommendation icon as described below.

[0052] The aforementioned input methods may include voice input, text input, or handwriting input.

[0053] For example, the user can first trigger the light and shadow keyboard to call out the AI assistant, and then manually input the first interaction information through the light and shadow keyboard. Among them, the light and shadow keyboard is shaped like light and shadow and has a certain transparency, that is, the light and shadow keyboard does not block the content below it. It can be understood that in places where it is not convenient to have a conversation, such as on the subway, the user can use the text input method to input the first interaction information.

[0054] For another example, the user can Figure 3A long press on the voice listening status identifier 34 shown in

[0055] In some embodiments of the present application, the AI assistant can listen to the user's voice when it has the permission of sound collection or voice listening, so as to improve the information security while realizing convenient input of interaction information.

[0056] In some embodiments of the present application, when the first input is the input of reading information selection, the first reading information is the same as the first interaction information, both of which are the reading information selected by the user.

[0057] In some embodiments of the present application, the selection input of the above reading information can be any one of the following:

[0058] (1) The user long presses on the reading interface to select the input of the first reading information, that is, the reading information selection input that is not circled.

[0059] (2) The user uses the circle selection function to perform a circle selection input on the reading interface, where the reading information within the circle selection range is the first interaction information, and the first reading information is the same as the first interaction information.

[0060] Exemplarily, taking the reading information selection input that is not circled as an example, as Figure 2A shown, during the process of reading "Chu Shi Biao", the user can long press on "sù yè yōu tàn" displayed in the reading interface 20 of "Chu Shi Biao", that is, receive the first input; thus as Figure 2B shown, the mobile phone can display the "Ask AI" identifier 21 and the "Copy" identifier 22; where the "Copy" identifier 22 is used to indicate copying "sù yè yōu tàn"; the "Ask AI" identifier 21 is used to indicate asking the AI assistant a question with "sù yè yōu tàn" as the interaction information, so that the AI assistant can deeply think and analyze the meaning expressed by "sù yè yōu tàn" in "Chu Shi Biao" in combination with the corresponding reading context information.

[0061] For example, taking "circle search input" as the first input, after activating the circle search function, a circle search mask can be displayed on the reading interface. The user can perform circle selection input on the mask to select the first reading information in the reading interface. For example, if the user selects "sighing day and night" in "Memorial to the Throne", the electronic device can directly use "sighing day and night" as the interactive information and conduct in-depth thinking and analysis.

[0062] In some embodiments of this application, in a selection scenario, when a user performs selection input on a selection mask, the information processing device selects a display area in the reading interface each time, obtains a screenshot image within that display area, and uses the screenshot image as the first interactive information.

[0063] It is understandable that, compared to non-circle search scenarios, users do not need to manually or verbally input the first interaction information in circle search scenarios, thus improving the convenience of determining the first interaction information.

[0064] In some embodiments of this application, the "receiving the user's first input" in step 101 above can be implemented by step 201 below.

[0065] Step 201: When the information processing device displays an AI reading companion icon on the reading interface, it receives the user's first input, which indicates that the reading interface is in AI reading companion mode.

[0066] In some embodiments of this application, when the reading interface is in AI-assisted reading mode, the AI ​​assistant can obtain reading information in the reading interface and / or listen to user-defined interaction information, such as voice interaction information; and can generate or determine response information based on the user-defined interaction information and the context information corresponding to the reading information in the reading interface that is key to the interaction information.

[0067] In some embodiments of this application, the information processing device can obtain reading information from the reading interface through an AI assistant, perform semantic understanding on the obtained reading information, and cache the semantic understanding information. In this way, upon receiving the first input, the device can quickly locate the context information corresponding to the first reading information directly from the cached information, and respond by relating the context based on the first interaction information and the context information corresponding to the first reading information, without needing to perform semantic understanding on the original text in the reading interface again, thereby further improving response speed.

[0068] In some embodiments of this application, when the reading interface is in AI-assisted reading mode, the user can continue reading through the reading interface without being affected.

[0069] For example, such as Figure 2AAs shown, after the reading interface enters AI-assisted reading mode, the information processing device can display the AI-assisted reading icon 23 in the top area of ​​the screen; then as... Figure 2C As shown, users can trigger updates to reading information, such as page turning, by swiping input on the reading interface 20, such as swiping up or down.

[0070] In some embodiments of this application, such as Figure 3A As shown, the screen displays a reading interface 30, and the AI ​​reading companion icon 31 can be displayed in the status bar 32, thereby avoiding or reducing the obstruction of reading information on the reading interface by the AI ​​reading companion icon.

[0071] In some embodiments of this application, the aforementioned AI reading companion icon may be displayed in the form of an atom island or an atom island card, or it may be displayed in the form of a floating icon.

[0072] For example, such as Figure 3A As shown, the AI ​​companion reading icon 31 can be displayed in the form of atomic islands by default. When the user inputs information into these atomic islands or during the user's voice input interaction, such as... Figure 3D As shown, the AI-assisted reading indicator automatically switches to the display format of Atomic Island Card 39. It can be understood that the display size of the atom-to-card format is larger than the display size of the atom island format.

[0073] In some embodiments of this application, after the user triggers the wake-up of the AI ​​assistant, such as Figure 3B As shown, the phone can display a light and shadow card 35 at the top edge of the screen. This card 35 includes the prompt message, "I'm here, how can I help you?", a "Clean Phone" icon 36, and an "Enter AI Reading Mode" icon 37. After the user clicks the "AI Reading Mode" icon 37, the reading interface enters the AI ​​assistant's AI reading mode, at which point the user can continue reading unaffected. The AI ​​assistant's prompts only retain the light and shadow at the screen edge, indicating to the user that the AI ​​assistant is continuously providing services in the background. There are no other operation buttons for the AI ​​assistant on the screen. Furthermore, the AI ​​assistant's background activities or conversation status can be indicated through the AI ​​reading icons.

[0074] When the AI ​​assistant is performing background AI-assisted reading activities, the edge lighting can be switched to an atom island-style AI-assisted reading icon, such as... Figure 3A The identifier 31 shown; if the user is interacting with the AI ​​reading companion function, the Atomic Island can be switched to an Atomic Island card, such as Figure 3D The identifier 39 is shown.

[0075] In some embodiments of this application, the AI ​​reading aid icon in the form of an atom island card supports shortcut button operations, such as mute and turn off.

[0076] For example, the AI-assisted reading identifier is displayed in the form of atomic islands when: such as Figure 3B As shown, when a user clicks the "Enter AI Reading Mode" recommendation icon, triggering the AI ​​reading mode to take effect, the AI ​​assistant's dialogue state will automatically change. Figure 3A The AI ​​reading companion icon (31) is represented by atomic islands, indicating that the AI ​​assistant is constantly accompanying the user during reading and can ask questions at any time. The AI ​​reading companion icon is displayed in atomic island form when: [e.g.] Figure 3A As shown, when the user clicks on the AI ​​companion reading icon 31 in the form of an atom island, it triggers a display. Figure 3D The atomic island card shown is 39.

[0077] In some embodiments of this application, in AI-assisted reading mode, a light and shadow icon may be displayed on the reading interface to indicate that the AI ​​assistant is providing reading assistance in the background.

[0078] In some embodiments of this application, in AI-assisted reading mode, the clipboard function built into the electronic device and the clipboard of the application to which the reading interface belongs can be disabled, while the clipboard of the AI ​​assistant's AI-assisted reading mode, such as the "Ask AI" and "Copy" capabilities, can be retained. See [link to relevant documentation]. Figure 2B This ensures that users can trigger questions to the AI ​​assistant through touch input on the reading interface, such as long-press input, to enter initial interaction information, or to copy content from the reading interface. Even if the reading interface itself does not support content copying, content can be copied and extracted through the AI ​​assistant's clipboard. For example, the AI ​​assistant can use a screenshot tool to capture the interface content, then use OCR to recognize the text content and copy it.

[0079] In some embodiments of this application, such as Figure 3A As shown, the AI ​​companion reading identifier 31 may include at least one of the following: an interactive information input identifier 33 and a voice monitoring status identifier 34. The interactive information input identifier 33 is used to input interactive information based on user operation, and the voice monitoring status identifier 34 is used to monitor user voice control information.

[0080] In some embodiments of this application, the voice control information may include at least one of the following: voice information for determining first interactive information, voice information for turning AI reading companion mode on or off, voice information for stopping listening to voice input, voice information for waking up or turning off the circle search function, etc.

[0081] In some embodiments of this application, users can trigger an AI assistant to listen to the user's voice control information by long-pressing the voice monitoring status indicator.

[0082] In some embodiments of this application, the aforementioned voice monitoring status identifier can also be used to indicate whether the AI ​​reading companion function is in voice monitoring status.

[0083] For example, such as Figure 3D As shown, users can input the voice monitoring status indicator in the Atomic Island card 39 to trigger the AI ​​assistant to be prohibited from listening to voice control information, that is, to control the AI ​​assistant to stop taking sound and to change the display form of the voice monitoring status indicator; the AI ​​assistant can only take sound and answer questions normally after the user unmutes it.

[0084] In some embodiments of this application, the aforementioned voice monitoring status indicator includes two display states, namely a first state and a second state. Users can trigger the display state of the voice monitoring status indicator through input, such as voice input, touch input, or gesture input.

[0085] When the voice monitoring status indicator is displayed in the first state, it means that the AI ​​companion reading function is in voice monitoring mode, that is, it can collect and recognize voice information, such as listening to whether the user has entered the first interactive information via voice. When the voice monitoring status indicator is displayed in the second state, it means that the AI ​​companion reading function is in a muted voice state, that is, the electronic device cannot collect the user's voice. If the user wants to ask a question via voice, they can first switch the display state of the voice monitoring status indicator back to the first state.

[0086] In some embodiments of this application, at least one of the color, shape, and pattern of the voice monitoring status indicator is different in different states.

[0087] For example, the voice monitoring status indicator can be a speaker icon. When voice monitoring is enabled, the speaker is green; when recording is disabled, the speaker is gray.

[0088] For example, in voice monitoring mode, the voice monitoring status is indicated by a speaker icon; when sound is not allowed, the voice monitoring status is indicated by a speaker icon plus a slash icon, with the slash indicating that sound is prohibited.

[0089] In some embodiments of this application, by inputting an AI-assisted reading identifier, a first input method interface can be brought up while maintaining the displayed reading interface, and first interactive information can be input through this first input method interface. The first input method interface can be the AI ​​assistant's light and shadow keyboard. Furthermore, the user can also trigger a switch of the input method interface through input, such as switching the Pinyin input method keyboard to a handwriting input method panel.

[0090] Furthermore, users can also input the first interactive information by voice through the voice input control in the first input method interface. This application is not limited to this.

[0091] In some embodiments of this application, such as Figure 3D As shown, the AI ​​reading companion icon displayed on the Atomic Island card 39 also includes a close button (i.e., button "×"). If the user presses the close button, the AI ​​assistant will exit the AI ​​reading companion mode, and the screen light and shadow and the Atomic Island status will disappear.

[0092] Thus, since the AI-assisted reading identifier includes an interactive information input identifier based on user operation input and a voice monitoring status identifier for listening to user voice control information, users can quickly trigger corresponding processing through these two identifiers during the reading process, thereby further improving reading convenience.

[0093] It should be noted that the AI ​​reading companion mode in this application is a system-level function, meaning that the AI ​​reading companion mode can be activated in any reading interface display state to perform intelligent reading companion on the reading interface through the AI ​​assistant.

[0094] Thus, when the reading interface is in AI-assisted reading mode, it can receive the user's first input and display the first response information on the reading interface based on the context information corresponding to the first interaction information and the first reading information. This can improve the ease of operation for users to interact with electronic devices, as well as the intelligent reading and companionship effects.

[0095] Step 102: The information processing device displays the first reply information on the reading interface based on the context information corresponding to the first interactive information and the first reading information.

[0096] In some embodiments of this application, the information processing device displays first response information on the reading interface based on context information corresponding to the first interaction information and the first reading information, including any one of the following:

[0097] In response to the first input, based on the context information corresponding to the first interaction information and the first reading information, the first reply information is displayed on the reading interface;

[0098] In response to the user's selection input of reading information in the reading interface, that is, receiving the first input, displaying "Ask AI", and receiving the user's input on "Ask AI", based on the context information corresponding to the first interaction information and the first reading information, displaying the first reply information on the reading interface.

[0099] In response to the user's first input of the first interactive information, and after receiving confirmation input confirming the completion of the input of the interactive information, the first reply information is displayed on the reading interface based on the context information corresponding to the first interactive information and the first reading information.

[0100] In some embodiments of this application, the context information corresponding to the first reading information may include at least one of the following:

[0101] The reading information currently displayed on the reading interface is semantically related to the first reading information;

[0102] The reading history displayed on the reading interface contains reading information that is semantically related to the first reading information;

[0103] The first screen content obtained after the most recent activation of AI reading mode, such as when AI reading mode is activated while reading a second reading object, and the user reads the first reading object after finishing the second reading object, then: the first screen content may include at least part of the content of the second reading object and the content that has already been read or displayed in the first reading object.

[0104] In some embodiments of this application, the information processing device can acquire and cache the content displayed on the screen, such as reading information in a reading interface, text or image interaction information entered by the user, and response information corresponding to the interaction information.

[0105] In some embodiments of this application, when the first reading object corresponding to the reading interface is a publicly available reading object, the information processing device can request access to or obtain the complete content in the first reading object based on the identifier or index of the first reading object.

[0106] In some embodiments of this application, the above-mentioned display of the first response information on the reading interface based on the context information corresponding to the first interaction information and the first reading information includes:

[0107] Based on the context information and target information corresponding to the first interaction information and the first reading information, the first response information is displayed on the reading interface.

[0108] The target information may include at least one of the following:

[0109] The user's historical interaction information, as well as the corresponding response information;

[0110] In some embodiments of this application, the above-mentioned "displaying first response information on the reading interface based on the context information corresponding to the first interaction information and the first reading information" may include: determining the user's interaction intent based on the context information corresponding to the first interaction information and the first reading information; and performing information understanding, reasoning, information retrieval, and information generation based on the interaction intent and the context information corresponding to the first reading information, thereby obtaining the first response information.

[0111] For example, the information processing device performs an associated reply to the interaction information based on the context information corresponding to the first reading information associated with the first interaction information, and the historical reading information (such as historical articles, web pages, etc. browsed, and historical Q&A information) in the AI companion reading mode, so as to obtain the first reply information.

[0112] In some embodiments of the present application, before displaying the first reply information on the reading interface based on the context information corresponding to the first interaction information and the first reading information, the information processing device can reply to the query intention based on the context information corresponding to the first interaction information and the first reading information, in combination with the knowledge base of the AI assistant itself, and can also refer to the materials in the public knowledge base to perform a wider range of information retrieval on the interaction intention, and generate the first reply information based on the retrieval results.

[0113] Exemplarily, in the AI companion reading mode, when the user selects a certain sentence / paragraph of text in the reading interface and asks a question to the AI assistant, the AI assistant can default to answer in combination with the content of the entire article / file corresponding to the reading interface, rather than only reply to the currently selected text, and remind the user when replying that the generated reply information is based on the reply to all the content in the first reading object.

[0114] For example, as Figure 2A shown, when browsing the web text content, the user can long-press the text content "sù yè yōu tàn", so as to carry the currently selected text content as the input query (i.e., the first interaction information), and at the same time quote the current document to understand the text content selected by the user, that is, perform Q&A in combination with the context, rather than simply explaining the words of "sù yè yōu tàn". Based on the first interaction information and the full text of "Chu Shi Biao", generate the reply information, and as Figure 2D shown, display the reply information 24 corresponding to "sù yè yōu tàn" on the reading interface 20. From Figure 2D the reply information shown, it can be seen that the reply information 24 is the understanding conclusion of "sù yè yōu tàn" in combination with the full text content of "Chu Shi Biao". And, it is stated in the reply information that the reply to "sù yè yōu tàn" is "based on the understanding of the current document", and the reference documents or reference materials for the reply information are also stated.

[0115] In some embodiments of the present application, when presenting the first reply information to the user, the identification of the reference materials in the public knowledge base cited and referred to, such as the reference material name, the number of reference materials, can be presented to the user; for example, Figure 5B the first reply information 56 shown includes the reference description information of "found 4 reference materials"; and for another example, Figure 7B the first reply information 76 shown includes the reference description information of "found X reference materials".

[0116] Understandably, users can... Figure 5B or Figure 7B Entering the reference information will display a list of references, which includes entries for the cited references. Clicking on an entry will take you to the original webpage corresponding to that reference.

[0117] In some embodiments of this application, the user can also trigger an AI assistant to search for other related materials by inputting a command, and then compare and answer the first interactive information. Specifically, the AI ​​assistant can search for related materials online, and then combine this information with reading information from the currently viewed first reading object, such as the full text or content already viewed on the screen, to provide a combined answer to the first interactive information and obtain the first response information.

[0118] In some embodiments of this application, the first response information may include multiple information items, which can be output in a Markdown table format for easy comparison and reading by the user.

[0119] In some embodiments of this application, multiple response messages can be generated based on the context information corresponding to multiple reference documents, first interaction information, and first reading information, with each reference document corresponding to one response message. These multiple response messages can be output for comparison in Markdown table format for easy comparison and reading by the user.

[0120] In some embodiments of this application, in method 3 above, the first input may include input of a first recommendation identifier among at least one recommendation identifier displayed on the reading interface; each recommendation identifier is generated based on different reading information in the reading interface, and the first response information corresponding to each recommendation identifier is different.

[0121] In some embodiments of this application, the above-mentioned at least one recommendation identifier may include at least one of the following: highlight identifier, key concept identifier, background interpretation identifier, webpage link identifier, and key information interpretation identifier.

[0122] In some embodiments of this application, each recommendation identifier corresponds to a first response message, and different recommendation identifiers correspond to different first response messages.

[0123] For example, the first response information corresponding to the highlight identifier is the highlight content in the first reading object.

[0124] For example, the first response information corresponding to the key concept identifier is the key concept information in the first reading object.

[0125] For example, the first response information corresponding to the background interpretation identifier is the background interpretation information of the first reading object.

[0126] For example, the first response information corresponding to the webpage link identifier is the webpage indicated by the URL or webpage name in the reading interface. Clicking the webpage link identifier will jump to the corresponding webpage.

[0127] For example, if the reading interface includes "For details, please visit the official website of XX", the information processing device can find the link to the official website of XX based on the official website of XX, and bind the link to the generated "Go to the official website" icon. In this way, when the user clicks the "Go to the official website" icon, the webpage of the official website of XX will be displayed on the reading interface.

[0128] For example, if the reading interface includes "The interview series sponsored by XX, 'Zhang San's Dialogue: Exploring Artificial Intelligence,' is officially launched online...", the information processing device can find the playback link of "Zhang San's Dialogue: Exploring Artificial Intelligence" based on the reading information, and bind the playback link to the generated "Interview Playback Link" identifier. In this way, when the user clicks the "Interview Playback Link" identifier, the playback page of "Zhang San's Dialogue: Exploring Artificial Intelligence" will be displayed on the reading interface.

[0129] For example, the first response information corresponding to the key information interpretation identifier is the interpretation, extended understanding, or summary information of the key reading information in the reading interface.

[0130] For example, if a term is mentioned in the reading interface, the information processing device can interpret the data based on the context information corresponding to the term to obtain term interpretation information, generate a term interpretation identifier, and associate the term interpretation identifier with the term interpretation information. In this way, clicking the term interpretation identifier will bring up the corresponding term interpretation information.

[0131] For example, if an event is mentioned in the reading interface, the information processing device can summarize and extract the background, details, and impact of the event based on the context information corresponding to the event, and obtain event interpretation information and an event interpretation identifier. The event interpretation identifier is then associated with the event interpretation information, so that clicking the event interpretation identifier will bring up the corresponding time interpretation information.

[0132] For example, if a name is mentioned in the reading interface, such as Company X, the information processing device can generate a first response information about the causes and consequences of Company X's acquisition and a company event interpretation mark based on recent events (such as Company X being acquired) of Company X. The interpretation mark is then associated with the first response information. In this way, clicking the event interpretation mark will display the causes and consequences of Company X's acquisition.

[0133] It should be noted that, for key information interpretation labels, the information processing device can generate key information interpretation information and corresponding key information interpretation labels based on the key reading information displayed on the reading interface and the context information corresponding to the key reading information.

[0134] For "web link identifiers", you can directly find the link to the webpage indicated by the URL or webpage name in the reading interface and a web link identifier, and then bind the link to a web link identifier.

[0135] In some embodiments of this application, the recommendation identifiers are different, and the reading information and first response information corresponding to the recommendation identifiers are also different.

[0136] For example, the reading information corresponding to the highlight identifier is the introduction or summary of the first reading object, and the highlight identifier corresponds to the highlight response information of the first reading object.

[0137] For example, the reading information corresponding to the key concept identifier is the full text content and chapter structure information of the reading interface, and the key concept identifier corresponds to the key concept response information of the first reading object.

[0138] For example, the reading information corresponding to the background interpretation identifier is the full text content corresponding to the reading interface, and the background interpretation identifier corresponds to the background answer information of the first reading object.

[0139] It should be noted that when the reading content (such as an article) on the reading interface is from publicly available media or is an authorized reading content for the AI ​​assistant, the AI ​​assistant can access the full text content corresponding to the reading interface. This means obtaining the full text content through compliant channels.

[0140] It can be understood that when the first input is an input to the first recommendation identifier, the first interaction information can be considered as: the first reading information used when generating the first response information corresponding to the first recommendation identifier. That is, the first interaction information is the same as the first reading information.

[0141] In some embodiments of this application, background interpretation markers, highlight markers, and key concept markers can be collectively referred to as global recommendation markers corresponding to the reading interface. Webpage link markers and key information interpretation markers can be collectively referred to as personalized associative recommendation markers. It can be seen that the first response information corresponding to the associative recommendation marker is an extension and interpretation of the local reading information in the reading interface. The first response information corresponding to the global recommendation marker is a macro-level summary, interpretation, or extension of the content of the first reading object corresponding to the reading interface.

[0142] In some embodiments of this application, the first recommendation identifier is any one of at least one recommendation identifier.

[0143] Thus, since the reading interface displays at least one of the following: highlight markers, key concept markers, background explanation markers, webpage link markers, and key information explanation markers, the information processing device can flexibly and diversely recommend markers related to the reading information on the reading interface to the user, thereby improving reading convenience and intelligence.

[0144] In some embodiments of this application, each recommendation identifier is generated based on "simulated" interaction information corresponding to different reading information in the reading interface. That is, the AI ​​assistant simulates the user asking questions about the reading information in the reading interface, and generates corresponding response information and recommendation identifiers based on the question information (i.e., simulated interaction information) and the context information corresponding to the reading information. In essence, each recommendation identifier corresponds to one "simulated" interaction information.

[0145] For example, such as Figure 4A As shown, the recommendation icon 44, "Talk about lunar tides," is generated based on the "lunar tides" information in the reading interface. The text "Talk about lunar tides" corresponds to the "interactive information" associated with recommendation icon 44.

[0146] For example, such as Figure 5A As shown, the recommendation tag 54, "Tell us about the acquisition of Company X," is generated based on "Company X" in the reading interface. The text "Tell us about the acquisition of Company X" is equivalent to the "interactive information" corresponding to recommendation tag 54.

[0147] For example, such as Figure 6A As shown, the "post highlight" identifier 61 is generated based on the reading highlight information in the reading interface. The text "post highlight" is equivalent to the "interactive information" corresponding to the recommendation identifier 61, that is, equivalent to the first interactive information corresponding to "post highlight," which is equivalent to the response information generated by the information processing device when the user inputs: "What is the highlight of this post?"

[0148] In some embodiments of this application, when the first input is an input for a first recommendation identifier, the information processing device selects the first response information corresponding to the first recommendation identifier from at least one pre-generated first response information and displays the selected first response information on the reading interface.

[0149] It is understandable that, when displaying a reading interface, at least one recommendation icon can be shown on the interface. Each recommendation icon is generated based on different reading information within the interface, and each recommendation icon corresponds to a different first response. This means that the system can anticipate potential user interactions during reading based on the reading information and generate responses that relate to the reading context in advance. This approach guides users to gain a deeper understanding of the reading information, preventing them from overlooking certain technical details. Furthermore, since corresponding responses are generated in advance based on different reading information, when a user confirms input for a recommendation icon, the corresponding response can be displayed directly. Compared to real-time response generation after user input, this reduces response generation latency, shortens user wait times, and improves the user's reading experience.

[0150] Thus, during the reading process, since the user can directly input the first recommendation icon from at least one recommendation icon displayed on the reading interface, the information processing device can directly display the first response information pre-generated based on a reading message on the reading interface, without the user having to manually input interactive information. Therefore, compared with the solution of generating the response information in real time after the user manually inputs interactive information, the response information generation delay can be reduced, the user's waiting time for the response can be reduced, and the user's reading experience can be improved.

[0151] It is understood that, in this embodiment of the application, when using an electronic device, such as a mobile phone, for reading, there is no need to manually switch to a third-party APP to ask questions and search. Instead, in the AI-assisted reading mode, the AI ​​assistant can be invoked to complete the questioning, searching, and answering, thereby making it more convenient and faster to answer the questions encountered by the user during the reading process.

[0152] Furthermore, since users can ask questions at any time during the reading process and quickly obtain answers (such as the first reply), they can maintain the continuity of knowledge acquisition and information reading even during fragmented reading. This is very convenient, and with the assistance of the AI ​​assistant, they can more accurately and specifically understand and read texts or other multimodal content more deeply.

[0153] In the information processing method provided in this application embodiment, users can view and ask questions at any time, which is very convenient. It features an AI assistant to assist reading, providing companionship everywhere and offering Q&A services at any time, which is equivalent to having a "mentor" who can be consulted at any time.

[0154] In the information processing apparatus provided in this application embodiment, when a reading interface is displayed, after the user determines the first interactive information through the first input, the first reading information associated with the first interactive information in the reading interface can be automatically locked, and the first reply information can be displayed (such as overlay display or floating display) on the reading interface based on the first interactive information and the context information corresponding to the first reading information. That is, the user does not need to manually lock the first reading information and its corresponding context information, thus simplifying the convenience of information query, retrieval and extension operations during the reading process, and improving reading convenience and experience.

[0155] In some embodiments of this application, the AI ​​reading companion mode will not exit when the user triggers the activation of circle search or uses any other APP capabilities.

[0156] In some embodiments of this application, after the AI ​​reading companion mode is turned on, the user can manually close the mode using the close button on the Atomic Island card, or tell the AI ​​assistant to exit the AI ​​reading companion mode via voice / text.

[0157] In some embodiments of this application, after the AI ​​reading companion mode is enabled, the display scene can be continuously monitored. If it is detected that the reading scene has been exited, the AI ​​reading companion mode is exited and a prompt message indicating that the AI ​​reading companion mode has been exited is output.

[0158] In some embodiments of this application, before step 101 above, the information processing method provided in the embodiments of this application may further include the following step 103.

[0159] Step 103: When the information processing device displays the reading interface, it displays the corresponding recommendation icon at the corresponding position on the reading interface based on the reading information in the reading interface. The recommendation icon displayed at different positions is different.

[0160] In some embodiments of this application, when a reading interface is displayed, the information processing device can generate at least one "simulated" interaction information based on the reading information in the reading interface, that is, predict the user's possible interactive questions. Then, based on each of the above-mentioned "simulated" interaction information and the preceding and following questions corresponding to the reading information associated with each simulated interaction information in the reading interface, the information processing device generates a first response information and a recommendation identifier, and establishes an association between the first response information and the recommendation identifier.

[0161] For example, taking highlight identifiers as an example, the information processing device can summarize the core highlights and features of the reading interface based on the brief content of the first reading object, and associate the summarized highlights with highlight identifiers. In this way, after the user inputs the highlight identifiers, the content highlights of the documents, articles, posts, and books can be displayed in a personalized way on the reading interface, so as to tell the reader in a relatively concise and easy-to-understand way what the attractive points of the content in the reading interface are.

[0162] For example, taking highlight identifiers as an example, the information processing device generates structured key concept information corresponding to the reading interface based on the full text content and chapter structure information of the reading interface, and then associates this key concept information with key concept identifiers.

[0163] For example, taking a text or book as the primary reading material, the most crucial and core concepts of the entire text or book can be output more completely and specifically based on the entire content and chapter structure of the primary reading material. It should be noted that, compared to the first "core highlights," the structured key concept information will be more systematic, complete, and comprehensive.

[0164] For example, taking background resolution identifier as an example, the information processing device can read the full text content corresponding to the interface, determine a background interpretation intention, and conduct a network search for the background interpretation intention to mine background information, such as the creative background, historical background, cultural background, author's life, other related achievements of the author, and related authors of the same era, to help users supplement the information and better understand the text content.

[0165] In some embodiments of this application, the information processing device can centrally display a global recommendation icon. That is, the global recommendation icon can be located in a fixed area on the reading interface.

[0166] The aforementioned fixed area can be the top area, bottom area, left area, or right area of ​​the reading interface; this application does not impose any limitation on this.

[0167] For example, such as Figure 4A , Figure 4B , Figure 5A , Figure 5B , Figure 6A , Figure 6B and Figure 7A As shown, when displaying the reading interface, the "Highlights", "Key Concepts", and "Background Explanation" icons can be displayed in the bottom area of ​​the screen (equivalent to the bottom area of ​​the reading interface).

[0168] In some embodiments of this application, the Lenovo recommendation icon can be displayed following relevant reading information in the reading interface, so that users can intuitively understand the correspondence between the recommendation icon and specific reading information in the reading interface. That is, they know which reading information in the reading interface the recommendation icon is generated based on.

[0169] In some embodiments of this application, after the user enables the "circle search" function, the information processing device can display a corresponding recommendation icon at a corresponding position on the reading interface based on the reading information in the reading interface.

[0170] In some embodiments of this application, after triggering the circle search, because there are more precise and personalized service recommendation slots, such as at least one recommendation icon mentioned above, the user's reading experience can be more interesting and in-depth, and it is convenient to call AI tools to make connections and associations, making reading and learning more efficient.

[0171] In some embodiments of this application, when at least one recommendation icon is displayed, the user can also trigger the information processing device to determine the content selected by the user as the first interactive information by circling, and generate and display the corresponding first reply information on the reading interface based on the first interactive information determined by the user's circling input and the corresponding context information through the AI ​​assistant.

[0172] In some embodiments of this application, step 103 described above can be implemented by step 103A described below.

[0173] Step 103A: The information processing device displays the second recommendation icon within the preset area corresponding to the second reading information.

[0174] The aforementioned second recommendation identifier is related to the second reading information in the reading interface.

[0175] It is understandable that each second piece of reading information can be a key piece of reading information in the reading interface.

[0176] In some embodiments of this application, the second recommendation identifier can be any of the recommendation identifiers in the Lenovo recommendation identifier list.

[0177] For example, the second recommendation identifier can be a webpage link identifier or a key reading information interpretation identifier.

[0178] In some embodiments of this application, the aforementioned preset area can be the area to the left, right, top, or bottom of the second reading information, so as to intuitively show the user the connection between the "recommendation icon" and the "key content".

[0179] In some embodiments of this application, in order to avoid the second recommendation icon from obscuring the reading information in the reading interface, the second recommendation icon may be appropriately displayed in a blank area near the second reading information.

[0180] In some embodiments of this application, the number of second recommendation identifiers is not limited.

[0181] In some embodiments of this application, the second recommendation identifier may be displayed in the form of a bubble.

[0182] In some embodiments of this application, the second recommendation identifier may also be referred to as a personalized bubble recommendation slot or recommendation identifier.

[0183] In some embodiments of this application, in AI-assisted reading mode and with the selection function enabled, the AI ​​assistant can display a second recommendation icon in real time or randomly for key reading information in the reading interface.

[0184] For example:

[0185] (1) If a website is identified: the recommended icon "Go to the official website" will be displayed. After the user clicks the recommended icon "Go to the official website", the AI ​​assistant will directly open the browser in the system window and jump to the homepage of the identified official website.

[0186] (2) If a video resource entry is identified, a "Play Interview Link" recommendation icon will be displayed. After the user clicks on it, the AI ​​assistant will open the browser and play the video resource through the browser, or the AI ​​assistant will open the video player and play the video resource.

[0187] (3) If a specific keyword is used for the current article / document / post, the recommendation icon corresponding to the key content will be displayed.

[0188] In this way, since the second recommendation identifier can be displayed in the preset area corresponding to the second reading information, users can intuitively understand the correspondence between the second recommendation identifier and the second reading information based on the display position of the second recommendation position during the reading process. This makes it convenient for users to query the answer information corresponding to the second recommendation identifier at any time, thus achieving the effect of "solving problems on the spot when encountering problems" and improving the continuity of reading and the convenience of querying answer information.

[0189] The information processing method provided in the embodiments of this application will be described below with reference to examples.

[0190] Example 1: Taking the book reading interface as an example, after enabling the circle search function, such as Figure 4A As shown, the phone can display a search overlay on the reading interface 40, and display a "Document Highlights" icon 41, a "Key Concepts" icon 42, and a "Background Explanation" icon 43 at the bottom of the screen; and based on the recognition results of key content in the reading interface 40, display two bubble icons, namely bubble icon 44 and bubble icon 45. When the user clicks on bubble icon 45, as shown... Figure 4B As shown, the first reply information 46 corresponding to the bubble icon 45 can be displayed on the reading interface 40.

[0191] Example 2: Taking the file reading interface as an example, after enabling the circle search function, such as... Figure 5AAs shown, the phone can display a search overlay on the reading interface 50, and display a "Document Highlights" icon 51, a "Key Concepts" icon 52, and a "Background Explanation" icon 53 at the bottom of the screen; and based on two key reading pieces of information in the reading interface 50, it displays two bubble icons, bubble icon 54 and bubble icon 55. When the user clicks on bubble icon 54, as shown... Figure 5B As shown, the bubble icon 54 and the corresponding response information 56 for "Tell me about the acquisition of Company X" can be displayed on the reading interface 50.

[0192] Example 3: Taking the post reading interface as an example, after enabling the circle search function, such as Figure 6A As shown, the phone can display a search overlay on the reading interface 60, and show a "Document Highlights" icon 61, a "Key Concepts" icon 62, and a "Background Interpretation" icon 63 at the bottom of the screen; below the key reading information "Type A Graphics Card" in the reading interface 60, a "Advantages and Disadvantages of Type A Graphics Card" recommendation icon 64 is displayed, and below the key reading information "XXX Official Website" in the reading interface 60, a "Go to Official Website" recommendation icon 65 is displayed. When the user clicks on the "Go to Official Website" recommendation icon 65, the following will occur... Figure 6B As shown, the homepage of the XX website (66) can be overlaid on the reading interface (60).

[0193] Example 4: Taking the article reading interface as an example, after enabling the circle search function, such as Figure 7A As shown, the phone can display a search overlay on the reading interface 70, and show the "Document Highlights" icon 71, the "Key Concepts" icon 72, and the "Background Explanation" icon 73 at the bottom of the screen; and display a "What is Transformer?" recommendation icon 75 and a bubble icon 74 in the area adjacent to the reading information "Transformer" on the reading interface 70. When the user clicks on the recommendation icon 75, as shown... Figure 7B As shown, the answer information 76 corresponding to the recommendation icon 75 for "What is Transformer" can be displayed on the reading interface 70.

[0194] Thus, since corresponding recommendation icons can be displayed at corresponding locations on the reading interface based on the reading information, and different recommendation icons are displayed at different locations, users can input the corresponding recommendation icons according to their interaction needs (such as asking questions) during the reading process, so as to trigger the display of response information corresponding to the user's interaction needs.

[0195] In some embodiments of this application, before step 201 above, the information processing method provided in the embodiments of this application may further include the following step 104.

[0196] Step 104: The information processing device receives the user's second input.

[0197] Step 105: The information processing device responds to the second input and displays the AI ​​reading companion icon on the reading interface.

[0198] In some embodiments of this application, the information processing device, in response to a second input, controls the AI ​​assistant to enter AI reading companion mode and displays an AI reading companion icon on the reading interface when it recognizes that the currently displayed interface is a reading interface. Thus, since the AI ​​assistant can be controlled to enter AI reading companion mode and display the AI ​​reading companion icon on the content reading interface when the current interface is a reading scenario, it can provide users with more convenient reading companionship while avoiding the leakage of user privacy information.

[0199] In some embodiments of this application, users can activate the AI ​​reading companion function in the following ways: voice activation, physical button activation, circle search activation, AI reading companion function entry icon activation, etc. In other words, the second input can be the user's voice activation input, physical button input, circle search activation input, or input to the AI ​​reading companion function entry icon.

[0200] For example, taking voice wake-up as an example, a user can wake up the AI ​​assistant by saying a voice wake-up word, such as "Xiao X". If the AI ​​assistant recognizes that the current display scene is a reading scene, such as... Figure 3B As shown, the recognition result card 35 can be displayed in the top area of ​​the screen, such as the status bar area. This result card 35 may include a "Enter AI Reading Mode" recommendation icon 37; Figure 3C As shown, after the user clicks the recommendation icon, the information processing device can activate the AI ​​reading companion function in the AI ​​assistant and display the atomic island in the top area of ​​the screen. Figure 3B As shown, the result card 35 may also include a "Clean Phone" icon 36, which indicates the deletion of historical cache or saved reading context content to free up memory and storage space. This result card can also be called an "Atomic Island Card." After activating the AI ​​reading companion function, the "Atomic Island Card" can be switched to an AI reading companion icon displayed in an atomic island format. It is understood that the display size of the atomic island is smaller than the display size of the atomic island card.

[0201] In some embodiments of this application, when a physical button is used to wake up the device, the physical button may include, but is not limited to, volume buttons and power buttons.

[0202] For example, taking power button wake-up as an example, by touching the power button, such as by long-pressing, the AI ​​assistant is woken up. After the AI ​​assistant recognizes that the current scene is reading, it can display the recognition result card in the top area of ​​the screen, such as the status bar area. The result card may include a "AI Reading Companion Mode" recommendation label. After the user clicks the recommendation label, the information processing device can wake up the AI ​​reading companion function in the AI ​​assistant.

[0203] In some embodiments of this application, circle search wake-up refers to waking up the AI ​​reading companion mode by waking up the "circle search function". The information processing device reuses the input used to wake up the circle search function, and wakes up the AI ​​reading companion function based on waking up the circle search.

[0204] In some embodiments of this application, the wake-up method includes, but is not limited to, any of the following:

[0205] 1) Two-finger long press on the screen: After long pressing the screen with two fingers, a circle search light and shadow animation appears; when the reading scene is recognized, a personalized bubble recommendation "AI Reading Companion Mode" icon appears in the middle of the screen; after clicking, you can enter the AI ​​Reading Companion Mode;

[0206] 2) Long press the bottom navigation bar / button: Same effect as a two-finger long press;

[0207] 3) Other triggering methods: such as displaying a search activation icon in the screenshot image editing page, which the user inputs to activate the search function; another example is the search function entry icon in the quick function entry interface; yet another example is the search function entry icon in the settings interface.

[0208] The examples above illustrate how the user first wakes up the AI ​​assistant, and then the AI ​​assistant actively identifies whether the current scene is a reading scene, and provides the option to wake up the AI ​​reading companion function after recognizing the reading scene. In actual implementation, after waking up the AI ​​assistant, the user can manually enter the command "turn on / enable AI reading companion mode" to trigger the AI ​​assistant to wake up the AI ​​reading companion function.

[0209] In some embodiments of this application, the above example illustrates the process of waking up the AI ​​assistant and then waking up the AI ​​reading companion function. In actual implementation, the AI ​​reading companion function can also be woken up with a single click through a dedicated input.

[0210] For example, when waking up via voice, a wake word can be set specifically for waking up the AI ​​reading companion function, so that users do not need to manually confirm, such as not needing to perform confirmation input for the AI ​​reading companion mode recommendation icon.

[0211] For example, the AI ​​reading companion function can be activated by pressing and holding the volume button and the power button simultaneously, eliminating the need for manual confirmation from the user, such as requiring confirmation input for the AI ​​reading companion mode recommendation.

[0212] In this way, since users can manually trigger the information processing device to display the AI ​​reading companion icon on the reading interface, it can be ensured that the AI ​​reading companion mode is only enabled with the user's authorization or permission. This allows the AI ​​reading companion mode to provide good companionship for the user's reading, while avoiding the need to enable the AI ​​reading companion mode when the user does not need it.

[0213] In some embodiments of this application, after step 102 above, the information processing method provided in the embodiments of this application may further include the following steps 106 and 107.

[0214] Step 106: The information processing device receives a third input, which is used to input second interactive information.

[0215] Step 107: The information processing device displays the second response information on the reading interface based on the context information corresponding to the second interaction information, the first response information, and the third reading information; wherein, the third reading information is information in the reading interface that is related to the second interaction information.

[0216] In some embodiments of this application, if a user needs to ask further questions after reviewing the first response information, the user can perform a third input to determine the second interactive information, which may also be referred to as follow-up question information.

[0217] In some embodiments of this application, the third input may be voice input, text input, or input of reading information selected on the reading interface or the first reply information, etc., and this application does not limit it.

[0218] For example, taking text input as the third input, the first response information can be overlaid on the reading interface in the form of a light and shadow display panel, which includes an input box. The user can type in the text input box to trigger the AI ​​assistant's light and shadow keyboard, allowing the user to input the second interactive information through this keyboard. This is especially useful in places where conversation is inconvenient.

[0219] For further descriptions of steps 106 and 107, please refer to the relevant descriptions of steps 101 and 102 in the above embodiments.

[0220] Thus, after the first response is displayed on the reading interface, the user can continue to ask follow-up questions, and the information processing device can provide further relevant answers to the user's follow-up questions based on the context information corresponding to the first response and the third reading information. Therefore, the coherence and accuracy of the responses between the AI ​​assistant and the user can be further improved, thereby enhancing the performance of human-computer interaction.

[0221] It should be noted that the above-described method embodiments, or the various possible implementations of the method embodiments, can be executed individually, or, provided there are no contradictions, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0222] The information processing method provided in this application can be executed by an information processing device. This application uses an information processing device executing the information processing method as an example to illustrate the information processing device provided in this application.

[0223] This application provides an information processing device, such as... Figure 8 As shown, the information processing device 800 includes: a receiving module 801 and a display module 802;

[0224] The receiving module 801 is used to receive a first input when the reading interface is displayed, the first input being used to determine first interactive information; wherein the reading interface includes first reading information, and the first reading information is associated with the first interactive information;

[0225] The display module 802 is used to display first reply information on the reading interface based on the context information corresponding to the first interaction information and the first reading information received by the receiving module.

[0226] In one possible implementation, the first input includes inputting a first recommendation identifier among at least one recommendation identifier displayed on the reading interface; each recommendation identifier is generated based on different reading information in the reading interface, and the first response information corresponding to each recommendation identifier is different.

[0227] In one possible implementation, the display module 802 is further configured to display a corresponding recommendation icon at a corresponding position in the reading interface based on the reading information in the reading interface before the receiving module receives the first input, wherein the recommendation icon displayed at different positions is different.

[0228] In one possible implementation, the aforementioned recommendation identifier includes at least one of the following: highlight identifier, key concept identifier, background interpretation identifier, and webpage link identifier.

[0229] In one possible implementation, the display module 802 is specifically used to display a second recommendation identifier within a preset area corresponding to the second reading information; wherein the second recommendation identifier is associated with the second reading information.

[0230] In one possible implementation, the receiving module 801 is specifically used to receive the first input when an AI reading companion icon is displayed on the reading interface, wherein the AI ​​reading companion icon is used to indicate that the reading interface is in AI reading companion mode.

[0231] In one possible implementation, the AI ​​companion reading identifier includes at least one of the following: an interactive information input identifier and a voice monitoring status identifier, wherein the interactive information input identifier is used to input interactive information based on user operation, and the voice monitoring status identifier is used to monitor user voice control information.

[0232] In one possible implementation, the receiving module 801 is further configured to receive the second input before receiving the first input;

[0233] The display module 802 is further configured to display an AI reading companion icon on the reading interface in response to the second input received by the receiving module 801.

[0234] In one possible implementation, the receiving module 801 is further configured to receive a third input after the display module 802 displays the first reply information on the reading interface, the third input being used to input second interactive information;

[0235] The display module 802 is further configured to display the second response information on the reading interface based on the context information corresponding to the second interaction information, the first response information, and the third reading information received by the receiving module 801; wherein the third reading information is information in the reading interface that is related to the second interaction information.

[0236] In the information processing device provided in this application embodiment, when a reading interface is displayed, after the user determines the first interactive information through the first input, the first reading information associated with the first interactive information in the reading interface can be automatically locked, and the first reply information can be displayed on the reading interface (such as overlay display or floating display) based on the first interactive information and the context information corresponding to the first reading information. That is, the user does not need to manually lock the first reading information and its corresponding context information, so the convenience of information query, retrieval and extension operations during the reading process can be simplified, and the reading convenience and experience can be improved.

[0237] The information processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0238] The information processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0239] The information processing device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiment shown in Figure 7 achieve the same technical effect, and will not be described again here to avoid repetition.

[0240] Optionally, such as Figure 9 As shown, this application embodiment also provides an electronic device 900, including a processor 901 and a memory 902. The memory 902 stores a program or instructions that can run on the processor 901. When the program or instructions are executed by the processor 901, they implement the various steps of the above-described information processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0241] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0242] Figure 10 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0243] The electronic device 1000 includes, but is not limited to, the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.

[0244] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0245] The user input unit 1007 is configured to receive a first input when the reading interface is displayed, the first input being used to determine first interactive information; wherein the reading interface includes first reading information, and the first reading information is associated with the first interactive information;

[0246] The display unit 1006 is used to display the first reply information on the reading interface based on the context information corresponding to the first interaction information and the first reading information received by the user input unit 1007.

[0247] In one possible implementation, the first input includes inputting a first recommendation identifier among at least one recommendation identifier displayed on the reading interface; each recommendation identifier is generated based on different reading information in the reading interface, and the first response information corresponding to each recommendation identifier is different.

[0248] In one possible implementation, the display unit 1006 is further configured to display a corresponding recommendation identifier at a corresponding position in the reading interface based on the reading information in the reading interface before the user input unit 1007 receives the first input, wherein the recommendation identifier displayed at different positions is different.

[0249] In one possible implementation, the aforementioned recommendation identifier includes at least one of the following: highlight identifier, key concept identifier, background interpretation identifier, and webpage link identifier.

[0250] In one possible implementation, the display unit 1006 is specifically used to display a second recommendation identifier within a preset area corresponding to the second reading information; wherein the second recommendation identifier is associated with the second reading information.

[0251] In one possible implementation, the user input unit 1007 is specifically used to receive the first input when an AI reading companion icon is displayed on the reading interface, wherein the AI ​​reading companion icon is used to indicate that the reading interface is in AI reading companion mode.

[0252] In one possible implementation, the AI ​​companion reading identifier includes at least one of the following: an interactive information input identifier and a voice monitoring status identifier, wherein the interactive information input identifier is used to input interactive information based on user operation, and the voice monitoring status identifier is used to monitor user voice control information.

[0253] In one possible implementation, the user input unit 1007 is further configured to receive a second input before receiving the first input;

[0254] The display unit 1006 is also configured to display an AI reading companion icon on the reading interface in response to the second input received by the user input unit 1007.

[0255] In one possible implementation, the user input unit 1007 is further configured to receive a third input after the display unit 1006 displays the first reply information on the reading interface, the third input being used to input second interactive information;

[0256] The display unit 1006 is further configured to display the second response information on the reading interface based on the context information corresponding to the second interaction information, the first response information, and the third reading information received by the user input unit 1007; wherein the third reading information is information in the reading interface that is related to the second interaction information.

[0257] In the electronic device provided in this application embodiment, when a reading interface is displayed, after the user determines the first interactive information through the first input, the first reading information associated with the first interactive information in the reading interface can be automatically locked, and the first reply information can be displayed on the reading interface (such as overlay display or floating display) based on the first interactive information and the context information corresponding to the first reading information. That is, the user does not need to manually lock the first reading information and its corresponding context information, thus simplifying the convenience of information query, retrieval and extension operations during the reading process, and improving reading convenience and experience.

[0258] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here.

[0259] The memory 1009 can be used to store software programs and various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0260] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.

[0261] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described information processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0262] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0263] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above information processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0264] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0265] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the information processing method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0266] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0267] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0268] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An information processing method, characterized in that, include: When a reading interface is displayed, a first input is received, which is used to determine first interactive information; wherein, the reading interface includes first reading information, and the first reading information is associated with the first interactive information; Based on the context information corresponding to the first interaction information and the first reading information, the first reply information is displayed on the reading interface.

2. The method according to claim 1, characterized in that, The first input includes inputting a first recommendation identifier among at least one recommendation identifier displayed on the reading interface; each recommendation identifier is generated based on different reading information in the reading interface, and the first response information corresponding to each recommendation identifier is different.

3. The method according to claim 2, characterized in that, Before receiving the first input, the method further includes: Based on the reading information in the reading interface, a corresponding recommendation icon is displayed at a corresponding position on the reading interface, and different recommendation icons are displayed at different positions.

4. The method according to claim 2, characterized in that, The at least one recommendation identifier includes at least one of the following: highlight identifier, key concept identifier, background interpretation identifier, and webpage link identifier.

5. The method according to claim 3, characterized in that, The step of displaying a corresponding recommendation icon at a corresponding location on the reading interface based on the reading information in the reading interface includes: A second recommendation identifier is displayed within a preset area corresponding to the second reading information; wherein the second recommendation identifier is associated with the second reading information.

6. The method according to claim 1, characterized in that, The receiving of the first input includes: When the AI ​​reading companion icon is displayed on the reading interface, the first input is received, and the AI ​​reading companion icon is used to indicate that the reading interface is in AI reading companion mode.

7. The method according to claim 6, characterized in that, The AI ​​companion reading identifier includes at least one of the following: an interactive information input identifier and a voice monitoring status identifier. The interactive information input identifier is used to input interactive information based on user operations, and the voice monitoring status identifier is used to monitor user voice control information.

8. The method according to claim 6, characterized in that, Before receiving the first input, the method further includes: Receive the second input; In response to the second input, an AI-assisted reading icon is displayed on the reading interface.

9. The method according to claim 1, characterized in that, After displaying the first reply information on the reading interface, the method further includes: Receive a third input, which is used to input second interactive information; Based on the context information corresponding to the second interaction information, the first reply information, and the third reading information, the second reply information is displayed on the reading interface; wherein, the third reading information is information in the reading interface that is related to the second interaction information.

10. An information processing device, characterized in that, include: Receive module and display module; The receiving module is configured to receive a first input when a reading interface is displayed, the first input being used to determine first interactive information; wherein the reading interface includes first reading information, and the first reading information is associated with the first interactive information; The display module is used to display the first reply information on the reading interface based on the context information corresponding to the first interaction information and the first reading information received by the receiving module.

11. The apparatus according to claim 10, characterized in that, The first input includes inputting a first recommendation identifier among at least one recommendation identifier displayed on the reading interface; each recommendation identifier is generated based on different reading information in the reading interface, and the first response information corresponding to each recommendation identifier is different.

12. The apparatus according to claim 11, characterized in that, The display module is further configured to, before the receiving module receives the first input, display a corresponding recommendation icon at a corresponding position in the reading interface based on the reading information in the reading interface, wherein the recommendation icon displayed at different positions is different.

13. The apparatus according to claim 12, characterized in that, The display module is specifically used to display a second recommendation identifier within a preset area corresponding to the second reading information; wherein the second recommendation identifier is associated with the second reading information.

14. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the information processing apparatus as described in any one of claims 1-9.

15. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the information processing apparatus as described in any one of claims 1-9.

Images