Method, apparatus and device for information interaction and storage medium

Abstract

According to embodiments of the present disclosure, methods, apparatuses, devices and storage media for information interaction are provided. The method comprises: in response to detecting that a digital assistant is triggered for an interaction event based on a first webpage; providing a first link of the first webpage or content in the first webpage to a webpage plug-in, the webpage plug-in being configured to perform a webpage processing task; and in an interaction window of a user and the digital assistant, utilizing the webpage plug-in to perform the interaction of the user and the digital assistant based on the content in the first webpage. In this way, the webpage plug-in can be utilized, so that the digital assistant can obtain content in a third-party webpage and perform interaction based on the content, which not only expands the application scenarios of the digital assistant, but also helps to reduce the difficulty and complexity of the interaction between the user and the digital assistant, and improves the efficiency of the interaction.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatuses, devices, and computer-readable storage media for information interaction. Background Technology

[0002] With the rapid development of internet technology, the internet has become a crucial platform for people to access and share content. Users can access the internet through terminal devices to enjoy various internet services. Terminal devices present relevant content and facilitate interaction with users through application user interfaces, providing services as well. Therefore, rich and varied application interfaces are an important means of enhancing user experience. With the development of information technology, various terminal devices can provide various services to people in work and life. For example, applications that provide services can be deployed on terminal devices. Terminal devices or applications can provide users with digital assistant functions to assist them in using the terminal devices or applications. How to improve the flexibility of user interaction with digital assistants is a technical issue that needs to be explored. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for information interaction is provided. The method includes: in response to detecting that a digital assistant is triggered for an interaction based on a first webpage, providing a first link to the first webpage or content from the first webpage to a webpage plugin, the webpage plugin being configured to perform a webpage processing task; and in an interaction window between the user and the digital assistant, utilizing the webpage plugin to perform an interaction between the user and the digital assistant based on the content from the first webpage.

[0004] In a second aspect of this disclosure, an apparatus for information interaction is provided. The apparatus includes: a webpage providing module configured to provide a first link to the first webpage or content from the first webpage to a webpage plugin in response to detecting that a digital assistant has been triggered to perform an interaction based on a first webpage, the webpage plugin being configured to perform a webpage processing task; and an interaction execution module configured to perform user-digital assistant interaction using the webpage plugin based on the content of the first webpage within an interaction window between the user and the digital assistant.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the description in this section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0010] Figures 2A to 2C A schematic diagram of an example interface of an interactive window according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram illustrating the information interaction flow according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A flowchart illustrating the information flow of information interaction according to some embodiments of this disclosure is shown;

[0013] Figures 5A to 5E A schematic diagram of an example client interface for an interactive window according to some embodiments of the present disclosure is shown;

[0014] Figure 6 A schematic diagram illustrating the flow of information interaction according to other embodiments of the present disclosure is shown;

[0015] Figure 7 A flowchart illustrating the information flow of information interaction according to other embodiments of this disclosure is shown;

[0016] Figure 8 A schematic diagram of an example client interface for an interactive window according to other embodiments of the present disclosure is shown;

[0017] Figure 9 A flowchart illustrating the information flow of webpage redirection according to some embodiments of the present disclosure is shown;

[0018] Figure 10 Examples of information interaction architectures according to some embodiments of this disclosure are shown;

[0019] Figure 11Examples of information interaction according to some embodiments of this disclosure are shown;

[0020] Figure 12 A schematic diagram illustrating a framework for determining recommended instructions according to some embodiments of the present disclosure is shown;

[0021] Figure 13 A flowchart of a process for information interaction according to some embodiments of the present disclosure is shown;

[0022] Figure 14 Block diagrams of apparatus for information interaction according to some embodiments of the present disclosure are shown; and

[0023] Figure 15 A block diagram of an electronic device capable of implementing one or more embodiments of the present disclosure is shown. Detailed Implementation

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0026] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0027] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, relevant users should be informed of the type, scope of use, and usage scenarios of the information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and authorization should be obtained from the relevant users. Among them, relevant users may include any type of rights holder, such as individuals, enterprises, and groups.

[0029] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly inform the user that the requested operation will require obtaining and using the user's information, thereby enabling the relevant user to choose whether to provide information to the software or hardware such as the electronic device, application, server, or storage medium that performs the operation of the technical solution disclosed herein based on the prompt message.

[0030] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide information to the electronic device.

[0031] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0032] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In this example environment 100, a digital assistant 120 and an application 125 are installed on a terminal device 110. A user 140 can interact with the digital assistant 120 and the application 125 via the terminal device 110 and / or an attachment device to the terminal device 110.

[0033] In some embodiments, the digital assistant 120 and application 125 can be downloaded and installed on the terminal device 110. In some embodiments, the digital assistant 120 and application 125 can also be accessed in other ways, such as through a web page. Figure 1 In environment 100, in response to application 125 being launched, terminal device 110 can present the interface 150 of digital assistant 120 and application 125.

[0034] Applications 125 include, but are not limited to, one or more of the following: chat applications (also known as instant messaging applications), document applications, audio and video conferencing applications, email applications, task applications, calendar applications, goal and key results (OKR) applications, etc. Although Figure 1 The image shows a single application, but in reality, multiple applications can be installed on the terminal device 110. In some embodiments, application 125 may include a multi-functional collaboration platform, such as an office collaboration platform (also known as an office suite), which can provide integration of various types of applications or components to facilitate people's office work, communication, and other activities. In a multi-functional collaboration platform, people can launch different applications or components as needed to complete corresponding information processing, sharing, communication, etc.

[0035] Application 125 can provide content entity 126. Content entity 126 can be a content instance created by user 140 or other users on application 125. For example, depending on the type of application 125, content entity 126 can be a document (e.g., a Word document, a PDF document, a presentation, a spreadsheet document, etc.), email, message (e.g., a conversation message on an instant messaging application), calendar, schedule, task, audio, video, image, etc.

[0036] In some embodiments, the digital assistant 120 may be provided by a standalone application or may be integrated into an application 120 capable of providing content entities. The application providing the client interface for the digital assistant may correspond to a single-function application or a multi-functional collaboration platform, such as an office suite or other collaboration platforms capable of integrating multiple components. In some embodiments, the digital assistant 120 supports the use of plugins. Each plugin can provide one or more functions of the application. Such plugins include, but are not limited to, one or more of the following: search plugins, contact plugins, messaging plugins, document plugins, form plugins, email plugins, calendar plugins, schedule plugins, task plugins, etc.

[0037] Digital assistant 120 is a user's intelligent assistant, possessing intelligent dialogue and information processing capabilities. In embodiments of this disclosure, digital assistant 120 is used to interact with user 140 to assist user 140 in using terminal devices or applications. An interaction window with digital assistant 120 can be presented in the client interface. In the interaction window, user 140 can converse with digital assistant 120 by inputting natural language to instruct digital assistant to assist in completing various tasks, including operations on content entity 126.

[0038] In some embodiments, digital assistant 120 may act as a contact for user 140, being included in the current user 140's contact list in the office suite or in the chat component's message stream. In some embodiments, user 140 and digital assistant 120 have a corresponding relationship. For example, a first digital assistant corresponds to a first user, a second digital assistant corresponds to a second user, and so on. In some embodiments, the first digital assistant may uniquely correspond to the first user, the second digital assistant may uniquely correspond to the second user, and so on. That is, the first digital assistant for the first user may be specific to or exclusive to the first user. For example, in the process of the first digital assistant providing assistance or services to the first user, the first digital assistant may utilize its historical interaction information with the first user, the data authorized by the first user that it can access, its current interaction context with the first user, etc. If the first user is an individual, the first digital assistant may be considered a personal digital assistant. It is understood that in the disclosed embodiments, the first digital assistant accesses data with granted permissions based on the first user's authorization. It should be understood that the terms "uniquely corresponding" or similar expressions in this disclosure are not intended to limit the first digital assistant to being updated accordingly based on the interaction process between the first user and the first digital assistant. Of course, depending on the actual application needs, the digital assistant 120 does not necessarily have to be specific to the current user 140, but can be a general digital assistant.

[0039] In some embodiments, multiple interaction modes can be provided for the user 140 and the digital assistant 120, and the user can flexibly switch between these modes. When a certain interaction mode is triggered, a corresponding interaction area is presented to facilitate interaction between the user 140 and the digital assistant 120. The interaction methods between the user 140 and the digital assistant 120 differ under different interaction modes, thus flexibly adapting to the interaction needs of different application scenarios.

[0040] In some embodiments, information processing services specific to user 140 can be provided based on historical interaction information between user 140 and digital assistant 120 and / or data ranges specific to user 140. In some embodiments, historical interaction information of user 140 interacting with digital assistant 120 in multiple interaction modes can all be associated with user 140 and stored. Thus, in one of the multiple interaction modes (any or a specified interaction mode), digital assistant 120 can provide services to user 140 based on the historical interaction information associated with user 140 stored therein.

[0041] The digital assistant 120 can be invoked or activated by an appropriate means (e.g., keyboard shortcuts, buttons, or voice) to present an interaction window with the user 140. The interaction window can be opened by selecting the digital assistant 120. The interaction window may include interface elements for information interaction, such as input boxes, message lists, message bubbles, etc. In other embodiments, the digital assistant 120 can be invoked through entry controls or menus provided on the page, or by inputting preset commands.

[0042] In some embodiments, the interaction window between the digital assistant 120 and the user 140 may include a session window, such as a session window in an instant messaging application or the instant messaging module of a target application. In some embodiments, the interaction window between the digital assistant 120 and the user 140 may include a floating window corresponding to the digital assistant. In some embodiments described below, for ease of discussion, the example of a session window as the interaction window between the user and the digital assistant will be used.

[0043] In some embodiments, the digital assistant 120 may support a conversation window interaction mode, also known as conversation mode. In this interaction mode, a conversation window is presented between the user 140 and the digital assistant 120, in which the user 140 and the digital assistant 120 interact through conversation messages. In conversation mode, the digital assistant 120 can perform tasks based on the conversation messages in the conversation window.

[0044] In some embodiments, the conversation mode between user 140 and digital assistant 120 can be invoked or activated by an appropriate means (e.g., shortcut keys, buttons, or voice) to present a conversation window. A conversation window with digital assistant 120 can be opened by selecting digital assistant 120. The conversation window may include interface elements for information interaction, such as input boxes, message lists, message bubbles, etc.

[0045] In some embodiments, the digital assistant 120 may support a floating window (or floating window) interaction mode, also known as a floating window mode. When the floating window mode is triggered, an operation panel (also known as a floating window) corresponding to the digital assistant 120 is presented, and the user 140 can issue commands to the digital assistant 120 based on the operation panel. In some embodiments, the operation panel may include at least one candidate shortcut command. Alternatively or additionally, the operation panel may include input controls for receiving commands. In floating window mode, the digital assistant 120 can perform tasks according to the commands issued by the user 140 through the operation panel.

[0046] In some embodiments, the floating window mode of user 140 and digital assistant 120 can also be invoked or activated by appropriate means (e.g., shortcut keys, buttons, or voice) to present the corresponding operation panel. In some embodiments, activating digital assistant 120 can be supported in specific applications, such as document applications, to provide floating window mode interaction. In some embodiments, to trigger the floating window mode to present the operation panel corresponding to digital assistant 120, an entry control for digital assistant 120 can be presented in the application interface. In response to detecting a trigger operation on the entry control, it can be determined that the floating window mode has been triggered, and the operation panel corresponding to digital assistant 120 is presented in the target interface area.

[0047] In some embodiments described below, for ease of discussion, the example of a session window serving as the interaction window between the user and the digital assistant will be used.

[0048] In some embodiments, terminal device 110 communicates with server 130 to provide services to digital assistant 120 and application 125. Terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 can also support any type of user-facing interface (such as "wearable" circuitry). Server 130 can be various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0049] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0050] As briefly mentioned earlier, digital assistants can help users use terminal devices or applications. Some applications can provide integration functionality for different plugins. In addition to engaging in free conversation with the digital assistant, users can also use natural language commands to enable the digital assistant to perform more complex business-related operations using different plugins, such as creating documents, scheduling appointments, and creating tasks.

[0051] According to some embodiments of this disclosure, an improved scheme for information interaction is proposed. In embodiments of this disclosure, upon detecting that a digital assistant is triggered to interact with a first webpage, a first link to the first webpage or content from the first webpage is provided to a webpage plugin. Here, the webpage plugin is associated with the digital assistant, for example, selected for interaction with the digital assistant. The webpage plugin is configured to perform webpage processing tasks. Thus, in the user's interaction window with the digital assistant, the webpage plugin performs user-digital assistant interaction based on webpage content.

[0052] In typical solutions, while users can send conversational messages containing links to a digital assistant, allowing the assistant to interact with them based on the data corresponding to the link, such links are usually limited to documents or other applications integrated with the digital assistant. According to embodiments of this disclosure, by providing a web page plugin, the digital assistant can access content from third-party web pages. This not only expands the application scenarios of the digital assistant but also helps reduce the difficulty and complexity of interaction between the user and the digital assistant, thereby improving interaction efficiency.

[0053] Some exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0054] Example interactive window

[0055] As described above, in the embodiments of this disclosure, a digital assistant is used for interaction with a user. An interaction window between the user and the digital assistant can be presented in the client interface. This interaction window may include a session window, in which the interaction between the user and the digital assistant can be presented in the form of session messages. Alternatively or additionally, the interaction window may also include other types of windows, such as a floating window, where the user can trigger the digital assistant to perform corresponding operations by inputting commands, selecting shortcuts, etc. As an intelligent assistant, the digital assistant has intelligent dialogue and information processing capabilities. In the interaction window, the user inputs interactive messages, and the digital assistant responds to the user's input by providing reply messages. The client interface used to provide the digital assistant can correspond to a single-function application or a multi-functional collaboration platform, such as an office suite or other collaboration platforms capable of integrating multiple components.

[0056] Figures 2A to 2C A schematic diagram of an example client interface 200 with an interactive window according to some embodiments of the present disclosure is shown. The client interface 200 can be implemented at a terminal device 110. References are made below. Figure 1 describe Figures 2A to 2C Examples.

[0057] In some embodiments, in response to a user (also known as a first user) activating a digital assistant through a preset operation (e.g., selecting a digital assistant from a contact list), the terminal device 110 may display a session window (also known as a main session window) showing the interaction between the first user and the digital assistant, as well as a plugin selection control. Through this plugin selection control, the first user can select the plugin to use in the main session window.

[0058] In some embodiments, such as Figure 2A As shown, the digital assistant (represented as "XX Assistant" in the figure) and the user's main conversation window are presented in area 210 of interface 200. The digital assistant can be considered one of the user's contacts and is presented in the instant messaging application's feed. The instant messaging application is presented in area 230 of interface 200, and the feed is presented in area 220 of interface 200. The user selects the digital assistant in the instant messaging application, thereby entering the main conversation window. The main conversation window includes a plugin selection control 212. The user can select the plugin to use in the conversation window by clicking the plugin selection control 212.

[0059] In some embodiments, the terminal device 110 responds to a user activating a digital assistant on a page (e.g., a page for browsing a document) through a preset operation (e.g., triggering a preset control, inputting a preset command, or voice wake-up, etc.) and displays a sub-session window for user interaction with the digital assistant. Such a sub-session window includes a plugin selection control. Through this plugin selection control, the user can select and use plugins within the sub-session window.

[0060] In some embodiments, in response to detecting a selection operation (e.g., a click) on the plug-in selection control 212, the terminal device 110 may present a plug-in selection panel 214. The plug-in selection panel 214 displays at least one pre-stored, already created plug-in (e.g., ...). Figure 2A (Examples include search plugins, contact plugins, messaging plugins, etc.). In some embodiments, the plugin selection panel 214 also presents a plugin viewing control 216. For example, in response to detecting a selection operation on the plugin viewing control 216, the terminal device 110 may display all plugins included in the terminal device 110.

[0061] In some embodiments, the terminal device 110 responds to a user activating a digital assistant on a first page (e.g., a page for browsing documents) through a preset operation (e.g., triggering a preset control, inputting a preset command, or voice wake-up, etc.) and displays a sub-session window (also called an interaction window) in which the first user interacts with the digital assistant. Such a sub-session window includes a plugin selection control. Through this plugin selection control, the user can select and use plugins within the sub-session window.

[0062] In some embodiments, the interactive window is presented in association with target content. For example, such as Figure 2B As shown, the user selects a document application in area 230 and views the target content in the document in area 220. During the viewing process, the user can invoke the digital assistant, for example, by triggering preset controls or menus in area 220, or by voice. In response to such an operation, the terminal device 110 displays a separate session window between the user and the digital assistant in area 210 of the interface 200. Note that in... Figure 2B The accompanying drawings illustrate, for example, that region 210, containing the sub-session window, and region 220, containing the target content, are arranged side-by-side. In other embodiments, they can be arranged in other ways as needed. For example, region 210 can be presented on the left and region 220 on the right in the client interface. Alternatively, regions 210 and 220 can be arranged vertically, and so on. The sub-session window includes a plugin selection control 212. Users can select plugins to use in the session window by clicking the plugin selection control 212.

[0063] In some embodiments, the application containing the main session window and the application containing the sub-session windows are different applications. For example, the application containing the main session window could be an instant messaging application used by the user to communicate with other users and digital assistants. For example, the application that can invoke the sub-session window could be another application besides an instant messaging application, such as a document application, a spreadsheet application, a calendar application, a scheduling application, a meeting application, a project management application, a customer relationship management (CRM) application, etc.

[0064] In some embodiments, the terminal device 110 can trigger a digital assistant and present an interactive window in the area where the target content of the first page is displayed. Specifically, as Figure 2C As shown, after selecting a portion of the target document within area 220, a menu (not shown) can be displayed by right-clicking, which includes an entry control for the digital assistant. Triggering the entry control will present... Figure 2C The interactive window shown is presented as a floating window in this example, also known as a floating interactive window or a floating session window. Of course, besides right-clicking, the floating interactive window can be triggered in various other suitable ways, which are not limited to the embodiments disclosed herein. A plugin selection control 222 may also be displayed in the floating interactive window. Users can select plugins to use in the session window by clicking the plugin selection control 222.

[0065] In some embodiments, in response to an operation that invokes a digital assistant on a first page, the terminal device 110 displays an interaction window (also called an interaction window) between the user and the digital assistant. Such an interaction window may include a separate session window and a floating interaction window. The triggering operations for displaying the separate session window and the floating interaction window may be the same or different. For example, if a user invokes the digital assistant by voice while browsing document content using a document application, the terminal device 110 displays a separate session window in one area of ​​the area displaying the document content. As another example, if a user invokes the digital assistant by selecting a section of text while browsing document content using a document application, the terminal device 110 displays a floating interaction window in the area displaying the document content.

[0066] In some embodiments, terminal device 110 can synchronize interactive messages in the interactive window to the main conversation window where the user interacts with the digital assistant. For example, when a user switches to an instant messaging application, the conversation messages in the sub-conversation window of a document application can be synchronized from the sub-conversation window to the main conversation window. In this way, both the user and the digital assistant can better understand the context information.

[0067] It should be understood that the accompanying drawings use a document as an example of target content for illustration. In other embodiments, the target content may also include any other suitable content that the platform can process, including but not limited to audio, video, images, emails, calendars, schedules, tasks, etc. For any of this content, a digital assistant can be invoked and a session window can be presented.

[0068] Figures 2A to 2C Several examples of launching a session window for the user and digital assistant are shown. It should be understood that this session window can also be launched in other ways. Furthermore, Figures 2A to 2C The various specific information, icons, etc. provided are merely examples and are not intended to limit the scope of the embodiments disclosed herein.

[0069] Example of webpage-based interactive implementation

[0070] In some embodiments, in the user's interaction window with the digital assistant (e.g., main session window, sub-session window, floating interaction window, etc.), the terminal device 110 receives a message (which may be referred to as a first message) from the user. The first message may include a first link to a first webpage. The first message may also include other content; for example, the first message may also include a question from the user regarding the first webpage. The first webpage may be any suitable webpage, and this disclosure does not limit the first webpage and / or the first link. In response to receiving the first message containing the webpage link, the terminal device 110 will detect that the digital assistant has been triggered to perform an interaction event based on the first webpage.

[0071] In embodiments of this disclosure, when it is determined that a digital assistant is triggered for interaction based on a first webpage, the terminal device 110 uses a webpage plugin to obtain content from a first link on the first webpage, and at least uses the webpage plugin to perform user-digital assistant interaction based on the content of the first webpage. Here, the webpage plugin is a plugin configured to perform webpage processing tasks. For example, the webpage plugin can perform tasks such as content extraction, content summarization, and content comparison for the webpage; this disclosure does not limit the specific webpage processing tasks performed by the webpage plugin. In some embodiments, the webpage plugin can call a model to complete the webpage processing task. The model can be a machine learning model, a deep learning model, a learning model, a neural network, etc. In some embodiments, the model can be based on a language model (LM). A language model, by learning from a large corpus, can possess question-answering capabilities. The model can also be based on other suitable models. In some embodiments, the webpage plugin can also call open interfaces provided by other applications (e.g., calendar, meeting, etc.) to complete the webpage processing task.

[0072] The following will continue to describe several exemplary embodiments of this disclosure in detail with reference to the accompanying drawings. To better understand the information interaction scheme of this disclosure, we will first combine... Figures 3 to 5E This section describes several examples of performing information interaction within an interactive window. For ease of description, Figures 3 to 5E The following description uses the interactive window as the main session window as an example. It can be understood that information interaction performed in the main session window can be similarly implemented in sub-session windows and floating interactive windows.

[0073] Figure 3 A schematic diagram of an information interaction flow 300 according to some embodiments of the present disclosure is shown. Figure 3As shown in box 301, terminal device 110 can present a main conversation window between the user and the digital assistant. In some embodiments, if there is historical interaction information or historical topics between the user and the digital assistant, terminal device 110 can present a portion of the historical interaction information or historical topics in the main conversation window in response to the digital assistant being invoked. In this document, "topic" corresponds to a specific context of the interaction. During the interaction of each topic, the interaction information between the user and the digital assistant is considered as contextual information to assist the digital assistant in determining subsequent conversation messages. In some embodiments, topics are sometimes also referred to as or presented as the subject. Alternatively or additionally, in some embodiments, terminal device 110 may not present a portion of the historical interaction information or historical topics in the main conversation window in response to the digital assistant being invoked. In this case, terminal device 110 may, for example, present interaction guidance information in the main conversation window in response to the digital assistant being invoked. This interaction guidance information may be used, for example, to guide the user to start a new topic, select a scene, select a plugin, etc. Terminal device 110 may present the interaction guidance information in the main conversation window, for example, by presenting guidance message cards. It is understood that the terminal device 110 may present interactive guidance information in any appropriate manner (such as a pop-up window), and this disclosure does not limit this.

[0074] Terminal device 110 may, for example, present at least one scenario in a bootstrap message card. Here, a scenario refers to a set of tasks of the same type; that is, one scenario corresponds to multiple tasks of the same type. One or more scenarios may be configured with corresponding configuration information to perform tasks of the corresponding type. In some embodiments, terminal device 110 may, in response to receiving a selection of a first scenario from at least one scenario in the bootstrap message card, perform user-digital assistant interaction in a new topic based at least on the configuration information of the first scenario. The scenario configuration information includes at least one of the following: scenario setting information and plugin information. The scenario setting information may influence the digital assistant's response to the user to some extent, or be used to determine the digital assistant's response to the user. The plugin information indicates at least one plugin used to perform tasks in the corresponding scenario. The plugins to be used in the corresponding scenario can be configured through the scenario's plugin information.

[0075] In some embodiments, terminal device 110 may also present operation controls for selecting a scene in the main session window. For example, terminal device 110 may provide a set of scenes in response to detecting a triggered operation on the operation controls for selecting a scene (e.g., a click, long press, double-click, swipe, hover, etc.). This set of scenes may include at least one scene. Terminal device 110 then, in response to receiving a selection of a first scene from the set of scenes, opens a first topic in the session window and performs user-digital assistant interaction within the first topic, at least based on the configuration information of the first scene.

[0076] In box 310, if the user expects the digital assistant to perform a web page processing task, the selected first scenario can be a web page processing-related scenario. This web page processing-related scenario can be any suitable scenario, such as a web page question-and-answer scenario, a web page content summary scenario, etc. For ease of description, all web page processing-related scenarios can be collectively referred to as web page processing scenarios. In this case, in box 311, the terminal device 110 can receive a first message including a first link to a first web page in the main session window. The first link to the first web page can also be simply referred to as the first web page link. That is, the terminal device 110 can receive a first message including the first web page link. Further, the terminal device 110 can use a web page plugin to obtain the content of the first web page from the first link, and at least use the web page plugin to perform user-digital assistant interaction based on the content of the first web page. In some embodiments, the terminal device 110 can perform user-digital assistant interaction based on the configuration information of the web page processing scenario and the content of the first web page. For example, the terminal device 110 can perform the interactions shown in boxes 341, 342, and 343.

[0077] In some embodiments, if the first scenario selected by the user is not a web page processing scenario, or if the user does not select a first scenario, the terminal device 110 can perform user-digital assistant interaction based on the first scenario, or perform user-digital assistant interaction without a scenario. In this case, the terminal device 110 can still receive a first message including a first link of a first web page in the main session window. Taking the first message as only including the first link as an example, in box 320, the terminal device 110 can receive the first link of the first web page. In box 321, the terminal device 110 can automatically switch to the web page processing scenario matching the first message based on the first message. Similarly, the terminal device 110 can use a web page plugin to obtain the content of the first web page from the first link, and at least use the web page plugin to perform user-digital assistant interaction based on the content of the first web page. The terminal device 110 can also perform the interactions shown in boxes 341, 342, and 343, for example.

[0078] In box 341, terminal device 110 supports users asking questions about / summarizing webpage content, and can respond using world knowledge when the webpage content cannot provide a response. Here, world knowledge refers to any appropriate knowledge that the digital assistant can acquire in a rule-compliant manner. Terminal device 110 can, for example, present scenario guidance information in the main session window. This scenario guidance information can be used to prompt the user that the webpage processing scenario supports users asking questions about / summarizing webpage content, etc.

[0079] In box 342, terminal device 110 can also determine a set of recommendation instructions based on the content of the first webpage. Terminal device 110 can, for example, present the determined set of recommendation instructions in a main session window. Terminal device 110 can, in response to detecting a selection operation on a first recommendation instruction within the set of recommendation instructions, present the first recommendation instruction in the main session window. Terminal device 110 can, for example, present the first recommendation instruction in the main session window as a session message from the user. Terminal device 110 can, based on the first recommendation instruction, execute the webpage processing task indicated by the first recommendation instruction.

[0080] In box 343, terminal device 110 can also support paginated webpage opening. The webpage here can be a first webpage or any other suitable webpage. Taking the first webpage as an example, terminal device 110 can display the first webpage in response to detecting a trigger operation on the first link of the first webpage. For example, terminal device 110 can display the first webpage while displaying the main session window. The positional relationship between the first webpage and the main session window can be any suitable positional relationship; for example, the first webpage can be located on the right side of the main session window, the first webpage can be located on the left side of the main session window, etc., and this disclosure does not limit this. This operation of switching from displaying the main session window to displaying both the main session window and the first webpage can also be called a pagination operation. This helps users to view the content of the first webpage while interacting with the digital assistant; such interaction can also be called comparative viewing. That is, terminal device 110 supports comparative viewing services for the first webpage.

[0081] In some embodiments, if the first scenario selected by the user is not a web page processing scenario, or if the user has not selected a first scenario, the first message received by the terminal device 110 in the main session window, which includes a first link to the first web page, may also include a question from the user regarding the first web page. In this case, at box 330, the terminal device 110 may receive the first link to the first web page and the user's question indicated by the question. At box 331, the terminal device 110 may automatically switch to the web page processing scenario matching the first message based on the first message. Similarly, the terminal device 110 may use a web page plugin to obtain the content of the first web page from the first link, and at least use the web page plugin to perform user interaction with the digital assistant based on the content of the first web page. The terminal device 110 may, for example, perform the interactions shown in boxes 344, 345, and 346.

[0082] In box 344, terminal device 110 can answer the user's question indicated in the first message based on the content of the first webpage. In a webpage processing scenario, terminal device 110 can, for example, determine a reply message for the first message based on the content of the first webpage and the user's question. If the reply message cannot be determined based solely on the content of the first webpage, terminal device 110 can, for example, determine the reply to the user based on world knowledge. It is understood that the reply message can be determined by the digital assistant in terminal device 110 and presented by terminal device 110 in the main session window. The interactions shown in boxes 345 and 346 are the same as those shown in boxes 342 and 343, and will not be described again here.

[0083] It should be noted that in some embodiments, before using the web page plugin to extract web page content and perform interactions, the terminal device 110 also needs to determine whether the web page plugin is enabled. It is understood that the terminal device 110 can only use the web page plugin to extract web page content and perform interactions if the web page plugin is enabled. In some embodiments, the terminal device 110 can determine that the web page plugin is enabled based on a first message including a first link. That is, in response to receiving a session message including a link to a web page, the web page plugin can be enabled by default.

[0084] Alternatively or additionally, in some embodiments, the terminal device 110 may further determine the current interaction scenario. The terminal device may determine that the web page plugin is enabled in response to receiving a first message when a scenario related to web page processing is selected for interaction. The terminal device 110 may also determine, based on the first message, that a scenario related to web page processing is selected and the web page plugin is enabled. That is, in a web page processing scenario, the terminal device 110 may default to enabling the web page plugin. In non-web page processing scenarios (e.g., scenarios other than web page processing scenarios, or no scenario), the terminal device 110 may switch to a web page processing scenario and enable the web page plugin in response to receiving a first message including a first link to a first web page.

[0085] In some embodiments, since different web pages may include different types of content, their links may also be of different types. In some embodiments, to ensure the correctness of the invoked plugin, the terminal device 110 may also determine whether the web page plugin is configured to handle the link type corresponding to the first link. The terminal device 110 may then determine that the web page plugin is enabled based on the determination that the web page plugin is configured to handle the link type corresponding to the first link. For example, the terminal device 110 may determine whether the web page plugin is configured to handle the link type corresponding to the first link based on the relevant information of the web page plugin. For example, the terminal device 110 may also provide the first link to the web page plugin, and the web page plugin may determine whether it is a link of a type that it can handle. For example, if the first web page is a document-type web page, the terminal device 110 may determine that the first link of the first web page is not a link of a type that the web page plugin can handle.

[0086] The following will continue to combine Figure 4 This describes the specific way in which terminal device 110 uses web page plugins to perform interactions. Figure 4 A flowchart of an information flow 400 for information interaction according to some embodiments of the present disclosure is shown. Figure 4 The client device 401 can correspond to the terminal device 110, and the instant messaging (IM) component 402, session service component 403, model service component 404, and web page plugin 405 can run on the server 130. For example... Figure 4As shown, client device 401 can receive a first message, including a first link to a first webpage, in the user's interaction window with the digital assistant (e.g., the main session window). Client device 401 can send (406) the received first message (i.e., the message with the first link to the first webpage) to IM component 402. IM component 402 can send (407) a notification, including at least the first message, to session service component 403 based on the first message. This notification may also include, for example, an instruction to invoke a webpage plugin. Session service component 403 can send (408) the first message to model service component 404 based on the received notification. Model service component 404 can, for example, invoke a model to perform webpage processing tasks. Model service component 404 can also invoke webpage plugin 405 to perform webpage processing tasks.

[0087] Different web pages may include different types of content, and their links may also be different types of links. In some embodiments, to ensure the correctness of the called plugin, before calling the web page plugin 405, the model service component 404 may also provide the first link to the web page plugin 405 so that the web page plugin 405 can determine (409) whether the first link belongs to the web page links that the web page plugin 405 can process.

[0088] If web page plugin 405 determines that the first link is not a processable web page link, web page plugin 405 may send a message to model service component 404 indicating that the web page plugin cannot be invoked. If web page plugin 405 determines that the first link is a processable web page link, web page plugin 405 may send a message to model service component 404 indicating that the web page plugin can be invoked. In some embodiments, if the user selects a scenario other than a web page processing scenario or if the user does not select a scenario, model service component 404 may also send (410) a new session identifier (e.g., a new session ID) to session service component 403. Model service component 404 may also, in response to receiving a message indicating that the web page plugin can be invoked, determine that the web page plugin is enabled and send (411) an invocation instruction to web page plugin 405 to invoke web page plugin 405.

[0089] In some embodiments, to ensure data security and compliance, it can also be determined (412) whether the web page plugin 405 can directly crawl the content of the first web page from the first link. This determination can also be called optional looping. For example, if the digital assistant is a digital assistant integrated into application A, and the first web page is a web page provided by application B, it can be determined that the web page plugin 405 cannot directly crawl the content of the first web page from the first link. If it is determined that the web page plugin 405 can directly crawl the content of the first web page from the first link, then the web page plugin located in server 130 (also called the server) can directly crawl (412) the content of the first web page from the first link on the server. If it is determined that the web page plugin 405 cannot directly crawl the content of the first web page from the first link, the web page plugin 405 can notify (415) the client device 401 that initiated the first message to crawl the content of the first web page from the first link. The client device 401 can crawl (414) the content of the first web page based on this notification. The way the content is crawled by the client device 401 can also be called client extraction. The client device 401 can report (416) the content of the crawled first webpage to the webpage plugin 405. Thus, the webpage plugin 405 can obtain the content of the first webpage.

[0090] Client device 401 can also receive user-triggered shortcuts or free conversations from users. Here, free conversations refer to user input received by client device 401 via information acquisition components such as input boxes and microphones. Shortcuts and free conversations can also be collectively referred to as conversation messages from users. Client device 401 can send (417) the received conversation messages to IM component 402. IM component 402 can send (418) a notification to conversation service component 403 that includes at least the conversation message. Conversation service component 403 can determine the corresponding extended information based on the conversation message. Extended information can be, for example, a prompt input determined based on the conversation message. Conversation service component 403 can send (419) the conversation message and extended information to model service component 404. Model service component 404 can send (420) a message to web plugin 705 to invoke web plugin 705. Web plugin 405 can perform the web processing operation indicated by the conversation message based on the message and previously acquired content. Web plugin 405 returns (421) the processing result to model service component 404. Model service component 404 forwards the processing result to session service component 403. Session service component 403 can determine a response message for the session message based on the processing result. Session service component 403 can send a response message (422) to IM component 402. IM component 402 forwards the received response message to client device 401 so that client device 401 can present the response message in the interaction window between the user and the digital assistant.

[0091] Figures 5A to 5E A schematic diagram of an example client interface 500 with an interactive window according to some embodiments of the present disclosure is shown. The client interface 500 can be implemented at a terminal device 110. References are made below. Figure 1 describe Figures 5A to 5E Examples.

[0092] like Figure 5A As shown, interface 500 displays operation controls 501 for selecting scenarios. Terminal device 110 can respond to a trigger operation on operation controls 501 and display window 510. Window 510 can display a set of scenarios (e.g., it may include scenarios such as "Web Q&A," "Content Creation," "Job Consultation," "Writing Interview Reviews," etc.). Window 510 can also display operation controls for managing scenarios (e.g., controls for "Manage and Customize Scenarios") and operation controls for displaying more scenarios (e.g., controls for "Discover More Scenarios"). Terminal device 110 can respond to a trigger operation on a specific scenario within the set of scenarios and determine that a selection of that scenario has been received. It should be noted that the scenario "Web Q&A" can be a scenario related to web page processing. Terminal device 110 can respond to a trigger operation on the scenario "Web Q&A," determine that a selection of the scenario "Web Q&A" has been received, and display as shown... Figure 5B The interface shown is 500.

[0093] Figure 5B The interface 500 shown may include a dividing line 502 to distinguish the "Web Q&A" scenario from previous scenarios. The interface 500 may also include a message card 520 corresponding to the "Web Q&A" scenario. The message card 520 may display guidance information associated with the "Web Q&A" scenario. This guidance information may include scenario guidance information for the "Web Q&A" scenario, at least one recommended question for the digital assistant under the "Web Q&A" scenario, at least one shortcut command for the digital assistant under the "Web Q&A" scenario, instructions for the plugins used in the "Web Q&A" scenario, etc. Figure 5BAs shown, message card 520 displays the scenario guidance information "Hello, in the [Web Q&A] scenario, you can send me a webpage, and I can help you summarize the webpage information or answer your questions about the webpage.", at least one recommended question (e.g., recommended question 521, recommended question 522, and recommended question 523), and plugin instruction 524. Plugin instruction 524 may, for example, indicate that the plugin selected / enabled by default in the "Web Q&A" scenario is a third-party webpage plugin (i.e., a webpage plugin). Interface 500 also displays an input box 530. When the "Web Q&A" scenario is selected, terminal device 110 may, for example, receive a first message input by the user via input box 530, including at least a first link to a first webpage.

[0094] In some embodiments, if the scenario selected by the user is not related to web page processing, or if the user does not select a scenario, the terminal device 110 may switch to the web page processing scenario in response to receiving a first message including a first link of a first web page in the session window. If the first message only includes the first link, the terminal device 110 may present the following: Figure 5C The interface shown is 500. (As shown) Figure 5C As shown, terminal device 110 can, in response to receiving a first message 541 including a first link, determine that the matching scenario is scenario "web page Q&A" and switch to scenario "web page Q&A". If the first message also includes a first question about the first web page, terminal device 110 can present the following: Figure 5D The interface shown is 500. (As shown) Figure 5D As shown, terminal device 110 can, in response to receiving a first message 542 including a first link and a first question about a first webpage, determine that the matching scenario is scenario "webpage Q&A" and switch to scenario "webpage Q&A". Terminal device 120 can also present a response message 550 (also referred to as a reply message) for the first question. Response message 550 is a first answer to the first question determined by the digital assistant based on the content of the first webpage. Response message 550 may contain an indication of the plugin used by the digital assistant to determine the reply message. In some embodiments, terminal device 110 can also present at least one recommendation instruction (e.g., recommendation instruction 551, recommendation instruction 552, and recommendation instruction 553) to guide the user to continue interacting with the first webpage. The recommendation instruction here can also be determined by the digital assistant based on the content of the first webpage.

[0095] It is understood that even if the first message does not include the first question, the terminal device 110 can still receive a second message that includes the first question. In response to the second message, the terminal device 110 can provide a first answer to the first question based on the content of the first webpage. The terminal device 110 can then present the first answer and at least one recommendation instruction on the interface 500.

[0096] In some embodiments, the terminal device 110 may also, in response to detecting a trigger on the first link in the first message, display content from a first webpage in a first area, the first area being independent of the interactive window. For example, the terminal device 110 may determine that a trigger on the first link has been received in response to detecting a click operation on the first link. The terminal device 110 may, in response to receiving a trigger on the first link, display content from a first webpage, such as... Figure 5E The interface shown is 500. (As shown) Figure 5E As shown, interface 500 also includes area 560, which is used to display the content of the first webpage. It should be noted that although... Figure 5E The example given is of region 560 located to the right of other regions; however, in other embodiments, the regions can be arranged in other ways as needed. For example, region 560 can be presented on the far left of the client interface. Alternatively, region 560 can be presented as a floating window. It is understood that in some embodiments, terminal device 110 can also switch from a client interface including an interactive window to a client interface including the first region in response to detecting a triggering of the first link in the first message. In this case, terminal device 110 may not present the interactive window and the first region simultaneously.

[0097] The above describes several examples of performing information interaction in an interactive window. In embodiments of this disclosure, terminal device 110 can also perform information interaction based on specific operations within a webpage. The following continues in conjunction with... Figures 6 to 9 This describes several examples of information interaction performed on a webpage based on specific actions. It can be understood that the webpage here refers to a webpage that the terminal device 110 can open if it complies with the rules.

[0098] Figure 6 A schematic diagram of an information interaction flow 600 according to other embodiments of the present disclosure is shown. For example... Figure 6 As shown, in box 601, terminal device 110 can display a conversation window. This conversation window can include a conversation window between the user and a digital assistant, a private chat window between users, and a group chat window between the user and a group of users. The conversation window can include conversation messages sent by the user, other users, and / or the digital assistant. In some embodiments, at least one conversation message containing a link to a webpage may exist. This conversation message may have been previously sent by the user or by other users and / or the digital assistant.

[0099] In box 602, terminal device 110 may determine that a trigger on a link has been received in response to detecting that a user has clicked a link on a webpage (e.g., a first webpage) in a session message. Terminal device 110 may present the content of the first webpage in a second area independent of the session window. The following description exemplifies switching from a client interface including an interactive window to a client interface including a second area. In this case, terminal device 110 presents the content of the first webpage without presenting the session window. Terminal device 110 may present a wake-up control for the digital assistant in such a client interface. In box 603, terminal device 110 may present a separate session window (i.e., an interactive window) for the user and the digital assistant in response to a trigger on the wake-up control. The presentation area of ​​the interactive window is independent of the presentation area of ​​the first webpage.

[0100] Terminal device 110 can provide content from a first webpage to a webpage plugin, and in an interactive window, utilize the webpage plugin to perform user-digital assistant interactions based on the content of the first webpage. Terminal device 110 can, for example, execute the interactions shown in boxes 604, 605, and 606.

[0101] In box 604, terminal device 110 can automatically load plugins / scenes. For example, terminal device 110 can, in response to a trigger of a wake-up control, default to determining the interaction scene as one related to web page processing. That is, terminal device 110 can, in response to detecting a trigger of a wake-up control while rendering the first web page, determine that a scene related to web page processing is selected for interaction in the interaction window. It is understood that, in the sub-session window, the user's interaction with the digital assistant is also based on the scene configuration information (e.g., configuration information of the scene related to web page processing) and the content of the first web page. Similarly, terminal device 110 can, in response to detecting a trigger of a wake-up control while rendering the first web page, determine that the web page plugin is enabled. That is, terminal device 110 can, in response to detecting a trigger of a wake-up control while rendering the first web page, enable the web page plugin by default. It is understood that terminal device 110 can also enable other plugins to utilize the web page plugin and other plugins together to perform user interaction with the digital assistant. Terminal device 110 can, for example, present guide cards in the sub-session window. The content in the guide card could be determined by the digital assistant based on the content of the first webpage.

[0102] In box 605, terminal device 110 can also perform a question-and-answer operation on the first webpage. For example, terminal device 110 can, in response to receiving user input regarding a question about the webpage content, determine the question indicated by that question. The digital assistant can then determine an answer to that question based on the content of the first webpage. For example, terminal device 110 can, in response to receiving a selection operation on a portion of the content of the first webpage and user input regarding that portion of content, determine the question indicated by that question. The digital assistant can then determine an answer to that question based at least on the selected portion of content.

[0103] In box 606, terminal device 110 can also present at least one recommendation instruction in a sub-session window. The at least one recommendation instruction may, for example, instruct on operations such as summarizing or translating content. Terminal device 110 can, in response to detecting a selection of a second recommendation instruction among the at least one recommendation instructions, present the second recommendation instruction in a sub-main session window. Terminal device 110 may, for example, present the second recommendation instruction in the sub-session window as a session message from the user, and, based on the second recommendation instruction, utilize a web page plugin to perform the web page processing task it indicates.

[0104] The following will continue to combine Figure 7 This describes the specific way in which terminal device 110 uses web page plugins to perform interactions. Figure 7 A flowchart of an information flow 700 for information interaction according to other embodiments of the present disclosure is shown. Figure 7 The client device 701 can correspond to the terminal device 110, and the instant messaging (IM) component 702, the session service component 703, the model service component 704, and the web page plugin 705 can run on the server 130. The client device 701 can present a first web page and a digital assistant wake-up control in the client interface. In response to detecting a trigger on the wake-up control, the terminal device 110 can send (706) a message to the session service component 703 including enabling the web page plugin, the link (i.e., the URL) of the first web page, the content of the first web page, and an indication that the scenario is related to web page processing. The session service component 707 can determine that the session scenario is related to web page processing based on the message, and send (707) a message to the model service component 704 including enabling the web page plugin, the link (i.e., the URL) of the first web page, and the content of the first web page. The model service component 704 can send (708) a new session identifier (e.g., a new session ID) to the session service component 703 based on the received message. The model service component 704 can also send a (709) call instruction to the web page plugin 705 to invoke the web page plugin 705.

[0105] In some embodiments, terminal device 110 may also present guidance information in a sub-session window in response to a triggering of a wake-up control. The guidance information includes at least one recommendation instruction for a digital assistant, the recommendation instruction being determined based on content in a first webpage. The guidance information may be determined by processing the content in the first webpage using webpage plugin 705. Webpage plugin 705 may send (710) the processing result of the content in the first webpage to model service component 704, which forwards the received processing result to session service component 703. Session service component 703 may, for example, determine a guidance card based on the processing result. The guidance card may include the processing result and guidance information. Session service component 703 sends (711) the guidance card to IM component 702, which in turn sends the guidance card to client device 701 so that client device 701 can present the guidance card.

[0106] Client device 701 can also receive user-triggered shortcuts or free conversations from users. Here, free conversations refer to user input received by client device 701 through information acquisition components such as input boxes and microphones. Shortcuts and free conversations can also be collectively referred to as conversation messages from users. Client device 701 can send (712) the received conversation messages to IM component 702. IM component 702 can send (713) a notification to conversation service component 703 that includes at least the conversation message. Conversation service component 703 can determine the corresponding extended information based on the conversation message. Extended information can be, for example, a prompt input determined based on the conversation message. Conversation service component 703 can send (714) the conversation message and extended information to model service component 704. Model service component 704 can send (715) a message to web plugin 705 to invoke web plugin 705. Web plugin 705 can perform the web processing operation indicated by the conversation message based on the message and previously acquired content. Web plugin 705 returns (716) the processing result to model service component 704. Model service component 704 forwards the processing result to session service component 703. Session service component 703 can determine a reply message for the session message based on the processing result. Session service component 703 can send a reply message (717) to IM component 702. IM component 702 forwards the received reply message to client device 701 so that client device 701 can present the reply message in the sub-session window between the user and the digital assistant.

[0107] Figure 8 A schematic diagram of an example client interface 800 with an interactive window according to other embodiments of the present disclosure is shown. The client interface 800 can be implemented at a terminal device 110. References are made below. Figure 1 describe Figure 8 Examples. Figure 8 As shown, interface 800 includes a region 810 for presenting the content of a first webpage. Region 810 includes a wake-up control 801 for the digital assistant. In response to detecting a trigger on wake-up control 801, terminal device 110 detects that the digital assistant has been triggered for an interaction event based on the first webpage, because the digital assistant is woken up in the context of presenting the first webpage. At this time, since terminal device 110 has already obtained the content of the first webpage for presentation, terminal device 110 can also provide the content of the first webpage to the webpage plugin accordingly. In this way, in the interaction window between the user and the digital assistant, the webpage plugin can be used to perform user-digital assistant interaction based on the content of a second webpage.

[0108] In response to the detection of a trigger on the wake-up control 801, the red-end device 110 presents area 820. Area 820 is used to present the user's conversation window with the digital assistant. It should be noted that, although... Figure 8 The example described uses region 820 located to the right of region 810. However, in other embodiments, the arrangement can be varied as needed. For instance, region 820 can be presented to the left of region 810 in the client interface. Alternatively, region 820 can be presented as a floating window.

[0109] In some embodiments, in response to detecting a switch from a first webpage to a second webpage while the interactive window is being presented, the terminal device 110 can also acquire and present the content of the second webpage. The terminal device 110 can provide the content of the second webpage to the webpage plugin, and in the interactive window, utilize the webpage plugin to perform user interaction with the digital assistant based on the content of the second webpage.

[0110] Figure 9 A flowchart of a webpage redirection information flow 900 according to some embodiments of the present disclosure is shown. Figure 9 The client device 901 can correspond to the terminal device 110, and the session service component 902, the model service component 903, and the web page plugin 904 can run on the server 130. The terminal device 110 can obtain (905) the content of the currently displayed first web page. The terminal device 110 can also display a sub-session window between the user and the digital assistant in response to detecting a triggering of a wake-up control while displaying the first web page. The terminal device 110 can also receive a session message with a link to a second web page sent by the user in the sub-session window. The terminal device 110 can send (906) the received session message including the link to the second web page to the session service component 902. The session service component 902 can send (907) a message instructing the model service component 903 to create a scene session based on the received session message.

[0111] Model service component 903 can send (908) a link to a second webpage and a message indicating whether crawling is permitted to the webpage plugin 904 based on the received message. Webpage plugin 904 can determine whether the content of the second webpage can be crawled based on the received message and the link to the second webpage. Webpage plugin 904 then returns (909) the determination result to model service component 903. The determination result may include, for example, a token indicating whether crawling is permitted. Model service component 903 can determine guidance information based on the received determination result. Model service component 903 may, for example, return (910) the guidance information to session service component 902. Session service component 902 may return (911) a thread identifier (e.g., thread ID) to client device 901. Client device 901 may report (912) the content of the webpage (e.g., the content of the first webpage and the content of the second webpage) to webpage plugin 904. Webpage plugin 904 may, for example, transmit (913) the received webpage content to model service component 903. Model service component 903 can perform operations such as content summary based on the content of the webpage, and can update (914) the guidance information.

[0112] Client device 901 may, for example, respond to a detected click on a link to a second webpage and redirect (915) to display the content of the second webpage. Client device 901 may report (912) the content of the webpage (e.g., the content of the second webpage) to webpage plugin 904. Webpage plugin 904 may, for example, cache (917) the received webpage content. Client device 901 may also send (918) the received session message (i.e., a message containing a link to the second webpage) to session service component 902 via IM component. Session service component 902 may send (919) the received session message to model service component 903. Model service component 903 may also invoke (920) webpage plugin 904 to perform webpage processing tasks. Different webpages may include different types of content, and their links may also be different types of links. In some embodiments, to ensure the correctness of the invoked plugin, model service component 903 may also provide a first link to webpage plugin 904 so that webpage plugin 904 can determine whether the link to the second webpage belongs to the webpage links that webpage plugin 904 can process. Web page plugin 904 can send (921) a determination result to model service component 903 indicating whether the link of the second web page belongs to the web page links that web page plugin 904 can process. Similarly, the determination result sent by web page plugin 904 can be a token including an indication of whether the link of the second web page belongs to the web page links that web page plugin 904 can process. Subsequent steps can be referred to steps 409 to 422 in information flow 400, and will not be described in detail here.

[0113] Figure 10 Example 1000 illustrates an architecture for information interaction according to some embodiments of this disclosure. For example... Figure 10 As shown, architecture 1000 may include client device 1010, gateway component 1020, IM component 1030, session service component 1040, model service component 1050, and web page plugin 1060. Here, client device 1010 may correspond to terminal device 110, and gateway component 1020, IM component 1030, session service component 1040, model service component 1050, and web page plugin 1060 may run on server 130.

[0114] like Figure 10 As shown, the client device 1010 can display a main dialog window 1011 corresponding to the main dialog and a sub-dialog window 1012 corresponding to the sub-dialog. The client device 1010 can receive various operations from the user in the main dialog window 1011 and / or the sub-dialog window 1012, such as starting a new dialog (1001), selecting a web page plugin (1002), etc. The client device 1010 can also receive session messages from the user, including web page links (or simply web page links), in the main dialog window 1011 and / or the sub-dialog window 1012.

[0115] Client device 1010 may, for example, send the received session message including a webpage link to gateway component 1020 based on any suitable communication protocol (e.g., HTTP). Gateway component 1020 may also, for example, send the session message including a webpage link to IM component 1030 based on any suitable communication protocol (e.g., RPC). IM component 1030 may also, for example, send the session message including a webpage link to session service component 1040 based on any suitable communication protocol (e.g., RPC). Session service component 1040 may also, for example, send the session message including a webpage link to model service component 1050 based on any suitable communication protocol (e.g., RPC).

[0116] Model service component 1050 can, for example, perform related processing tasks such as handling ordinary messages 1051, web page links 1052, plugin selection 1053, and scene guidance 1054. Model service component 1050 may also include a proxy router 1055. Based on the task to be performed, model service component 1050 can determine at least one directly selected proxy (e.g., directly selected proxy 1056 and directly selected proxy 1057). Each directly selected proxy can include a guidance tool. Model service component 1050 may also include a general module 1058 for session messages. Model service component 1050 can, for example, perform message replies to users based on the directly selected proxies and this general module. Model service component 1050 can also perform web page processing operations by invoking web page plugin 1060.

[0117] The web page plugin 1060 may include an interface 1061. Interface 1061 may include a guidance tool 1062, a query tool 1063, and a suggestion tool 1064. The guidance tool 1062 may be, for example, a tool for determining guidance information based on web page content. The query tool 1063 may be, for example, a tool for querying content within a web page. The suggestion tool 1064 may be, for example, a tool for providing suggestions to the user regarding their next interaction during the interaction process. The web page plugin 1060 may also include a web page link parsing module 1065, a web page data structuring module 1066, a caching module 1067, and a web page crawling module 1068. With these four modules, the web page plugin can possess web page link parsing functionality, web page content structuring functionality, content caching functionality, and web page crawling functionality.

[0118] The interface 1061 of the web plugin 1060 can execute a bootstrapping process (also known as an OnBoarding process) to ensure that other modules in the web plugin 1060 can perform their corresponding functions. Figure 11 Example 1100 of information interaction according to some embodiments of this disclosure is shown. For example... Figure 11As shown, client device 1101 can send (1106) a session message with a webpage link to IM component 1102. Session service component 1103 can obtain the session message with the webpage link forwarded by the IM component and send it (1107) to model service component 1104. Model service component 1104 can provide the webpage link in the session message to webpage plugin 1105 and send (1108) a message indicating whether the webpage link belongs to the webpage link that webpage plugin 1105 can handle. Such a message can, for example, instruct webpage plugin 1105 to perform message recognition on the received webpage link. Webpage plugin 1105 can generate a status code indicating the determination result and send it (1109) to model service component 1104. Model service component 1104 can determine whether the webpage link belongs to the webpage link that webpage plugin 1105 can handle based on the status code, and then determine whether to invoke webpage plugin 1105.

[0119] If it is determined that a web plugin can be invoked, the model service component 1104 may also send (1110) a message to the session service component 1103 including a new session identifier (e.g., a new session ID) and an identifier of the invoked web plugin 1105 (e.g., a web plugin ID). The model service component 1104 may also send (1111) an invocation instruction to the web plugin 105 to invoke the web plugin 1105. In response to receiving the invocation instruction, the web plugin 1105 may execute (1112) a bootstrapping process to determine boot information. During the execution of the bootstrapping process, the web plugin 1105 may determine a unique token that can be used to associate the context of the content obtained by the web plugin 1105. The web plugin 1105 may send this token to the model service component 1104.

[0120] Web page plugin 1105 can return (1113) the guidance information to model service component 1104. Model service component 1104 sends the guidance message back (1114) to session service component 1103. Session service component 1103 can send (1115) the received guidance message to IM component 1102. IM component 1102 can send (1106) the guidance message and the instruction to client device 1101 to perform message rendering to client device 1101. After receiving the instruction, client device 1101 can render the received guidance message to the client interface to present it to the user.

[0121] During a session, client device 1101 can receive user shortcut commands or free-flowing conversations (collectively referred to as session messages) and send the received session messages (1117) to IM component 1102. Session service component 1103 can receive session messages forwarded by IM component 1102 and send them to model service component 1104. In some embodiments, session service component 1103 can also send an identifier of a web page plugin to instruct model service component 1104 to invoke the web page plugin to perform web page processing. Model service component 1104 can provide the processing result to client device 1101 via forwarding from session service component 1103 and IM component 1102.

[0122] Figure 12 A schematic diagram of a framework 1200 for determining recommendation instructions according to some embodiments of the present disclosure is shown. Framework 1200 includes a webpage link 1201, a model service component 1202, a session service component 1205, a client device 1206, and a model 1207. The model service component 1202 includes a model engine 1203 and a language model 1204. In some embodiments, after a webpage plugin performs a bootstrapping process, it can provide a unique token determined by the bootstrapping process to the model service component 1202. The model engine 1203 in the model service component 1202 can obtain context information from the webpage corresponding to the webpage link 1201 based on this token. The model engine 1203 of the model service component 1202 can also obtain at least one recommendation instruction from the model 1207. For example, the model engine 1203 can provide the obtained context information to the model 1207 and obtain at least one recommendation instruction determined by the model 1207 based on the context information. The model engine 1203 can then send the obtained at least one recommendation instruction to the language model 1204. Language model 1204 sends at least one recommendation instruction to session service component 1205. Session service component 1205 then provides at least one recommendation instruction to client device 1206.

[0123] In summary, according to the embodiments of this disclosure, web page plugins can be used to enable digital assistants to obtain content from third-party web pages and perform interactions based on the content. This not only expands the application scenarios of digital assistants, but also helps to reduce the difficulty and complexity of interaction between users and digital assistants, and improves the efficiency of interaction.

[0124] Example process

[0125] Figure 13 A flowchart of a process 1300 for information interaction according to some embodiments of the present disclosure is shown. Process 1300 may be implemented at terminal device 110. For ease of discussion, reference will be made to... Figure 1 The environment 100 describes the process 1300.

[0126] In box 1310, in response to detecting that a digital assistant is triggered for an interaction event based on a first webpage, terminal device 110 provides a first link to the first webpage or content from the first webpage to a webpage plugin, the webpage plugin being associated with the digital assistant and configured to perform webpage processing tasks.

[0127] In box 1320, terminal device 110 uses a web page plugin to perform user-digital assistant interaction based on the content of a first web page in the user-digital assistant interaction window.

[0128] In some embodiments, process 1300 further includes: receiving a first message in the user's interaction window with the digital assistant, the first message including a first link to a first webpage; and detecting, in response to receiving the first message, that the digital assistant has been triggered to perform an interaction event based on the first webpage. In this case, in response to receiving the first message, the first link to the first webpage is provided to the webpage plugin so that the webpage plugin can retrieve content from the first link on the first webpage.

[0129] In some embodiments, obtaining the content of the first webpage from the first link by the webpage plugin includes: the webpage plugin crawling the content of the first webpage from the first link, or the webpage plugin notifying the client device that initiated the first message to crawl the content of the first webpage from the first link, and the client device reporting the content of the first webpage crawled to the webpage plugin.

[0130] In some embodiments, process 1300 further includes: determining, based on the first message including the first link, that the web page plugin is enabled in the user's interaction with the digital assistant.

[0131] In some embodiments, determining that a web page plugin is enabled includes: determining whether the web page plugin is configured to process the link type corresponding to the first link; and determining that the web page plugin is enabled based on the determination that the web page plugin is configured to process the link type corresponding to the first link.

[0132] In some embodiments, determining that a web page plugin is enabled includes: determining that a web page plugin is enabled in response to receiving a first message when a scenario related to web page processing is selected for interaction; or determining that a scenario related to web page processing is selected and a web page plugin is enabled based on the first message in response to receiving the first message.

[0133] In some embodiments, the user's interaction with the digital assistant is based on the scenario's configuration information and the content of a first webpage.

[0134] In some embodiments, performing an interaction between a user and a digital assistant includes at least one of the following: in response to a first message including a first question or receiving a first question for a first webpage, providing a first answer to the first question based on the content of the first webpage; presenting at least one recommendation instruction for the digital assistant in an interaction window, the at least one recommendation instruction being determined based on the content of the first webpage; or in response to detecting a triggering of a first link in a first message, presenting the content of the first webpage in a first area, the first area being independent of the interaction window.

[0135] In some embodiments, a web page plugin includes at least one of the following functions: web page link parsing function, web page content structuring function, content caching function, or web page crawling function.

[0136] In some embodiments, process 1300 further includes: presenting content in a first webpage and a wake-up control for a digital assistant; and detecting, in response to triggering the wake-up control, that the digital assistant has been triggered for an interaction event based on the first webpage. In this case, in response to triggering the wake-up control, a first link to the acquired first webpage is provided to the webpage plugin.

[0137] In some embodiments, process 1300 further includes: in response to detecting a triggering of a wake-up control while rendering content in a first webpage, determining that the webpage plugin is enabled in the user's interaction with the digital assistant.

[0138] In some embodiments, process 1300 further includes: in response to detecting a triggering of a wake-up control while presenting content in a first webpage, determining that a scenario related to webpage processing is selected for interaction in an interactive window, and wherein the interaction between the user and the digital assistant is based on scenario configuration information and content in the first webpage.

[0139] In some embodiments, process 1300 further includes: in response to a triggering of a wake-up control, presenting an interaction window between the user and the digital assistant, presenting guidance information in the interaction window, the guidance information including at least one recommendation instruction for the digital assistant, the at least one recommendation instruction being determined based on content in a first webpage.

[0140] In some embodiments, the presentation area of ​​the user's interaction window with the digital assistant is independent of the presentation area of ​​the first webpage.

[0141] In some embodiments, process 1300 further includes: in response to detecting a switch from a first webpage to a second webpage while the interactive window is being presented, presenting content in the second webpage; providing the content in the second webpage to the webpage plugin; and in the interactive window, utilizing the webpage plugin to perform user-digital assistant interaction based on the content in the second webpage.

[0142] Example devices and equipment

[0143] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes.

[0144] Figure 14 A schematic structural block diagram of an apparatus 1400 for information interaction according to some embodiments of the present disclosure is shown. The apparatus 1400 may be implemented in or included in a terminal device 110, for example. The various modules / components in the apparatus 1400 may be implemented by hardware, software, firmware, or any combination thereof.

[0145] As shown in the figure, device 1400 includes a webpage providing module 1410, configured to provide a first link to the first webpage or content from the first webpage to a webpage plugin in response to detecting that a digital assistant has been triggered for an interaction based on a first webpage. The webpage plugin is configured to perform webpage processing tasks. Device 1400 also includes an interaction execution module 1420, configured to perform user-digital assistant interaction using the webpage plugin based on the content of the first webpage within the user-digital assistant interaction window.

[0146] In some embodiments, the interaction determination module 1410 further includes: a message receiving module configured to receive a first message in an interaction window between the user and the digital assistant, the first message including a first link to a first webpage; and a message-based triggering module configured to detect, in response to receiving the first message, that the digital assistant has been triggered for an interaction event based on the first webpage. In some embodiments, in response to receiving the first message, the first link to the first webpage is provided to the webpage plugin so that the webpage plugin can obtain content from the first webpage from the first link.

[0147] In some embodiments, the webpage providing module 1410 includes: a plugin extraction module configured to allow a webpage plugin to crawl content from a first webpage via a first link, or a crawling notification module configured to allow a webpage plugin to notify a client device that initiated a first message to crawl content from the first link on the first webpage, and the content crawled by the client device on the first webpage to be reported to the webpage plugin.

[0148] In some embodiments, the apparatus 1400 further includes an enable determination module configured to determine, based on a first message including a first link, that the web page plugin is enabled during the user's interaction with the digital assistant.

[0149] In some embodiments, the enable determination module is further configured to: determine whether the web page plugin is configured to process the link type corresponding to the first link; and determine that the web page plugin is enabled based on the determination that the web page plugin is configured to process the link type corresponding to the first link.

[0150] In some embodiments, the enable determination module is further configured to: determine that the web page plugin is enabled in response to receiving a first message when a scenario related to web page processing is selected for interaction; or determine that a scenario related to web page processing is selected and the web page plugin is enabled based on the first message in response to receiving the first message.

[0151] In some embodiments, the user's interaction with the digital assistant is based on the scenario's configuration information and the content of a first webpage.

[0152] In some embodiments, the apparatus 1400 further includes: a response determination module configured to provide a first response to the first question based on content in the first webpage in response to a first message including a first question or receiving a first question for a first webpage; a recommendation instruction module configured to present at least one recommendation instruction for a digital assistant in an interactive window, the at least one recommendation instruction being determined based on content in the first webpage; or a webpage rendering module configured to present content in the first webpage in a first area, independent of the interactive window, in response to detecting a triggering of a first link in the first message.

[0153] In some embodiments, a web page plugin includes at least one of the following functions: web page link parsing function, web page content structuring function, content caching function, or web page crawling function.

[0154] In some embodiments, the apparatus 1400 further includes: a webpage rendering module configured to render content in a first webpage and a wake-up control for a digital assistant; and a wake-up-based interaction module configured to detect, in response to a triggering of the wake-up control, that the digital assistant has been triggered for an interaction event based on the first webpage. In some embodiments, in response to a triggering of the wake-up control, a first link to the acquired first webpage is provided to the webpage plugin.

[0155] In some embodiments, the apparatus 1400 further includes an enable determination module configured to determine that the web page plugin is enabled during user interaction with the digital assistant in response to a triggering of a wake-up control detected while the first web page is being rendered.

[0156] In some embodiments, the apparatus 1400 further includes a scene determination module configured to determine, in response to detecting a triggering of a wake-up control while rendering content in a first webpage, a scene related to webpage processing is selected for interaction in an interactive window. The user's interaction with the digital assistant is based on scene configuration information and the content in the first webpage.

[0157] In some embodiments, the apparatus 1400 further includes: a window presentation module configured to present an interaction window between the user and the digital assistant in response to a triggering control, the interaction window presenting guidance information including at least one recommendation instruction for the digital assistant, the at least one recommendation instruction being determined based on content in a first webpage.

[0158] In some embodiments, the presentation area of ​​the user's interaction window with the digital assistant is independent of the presentation area of ​​the first webpage.

[0159] In some embodiments, the apparatus 1400 further includes: a switching presentation module configured to present content in the second webpage in response to detecting a switch from a first webpage to a second webpage while the interactive window is being presented; and a providing module configured to provide the content in the second webpage to the webpage plugin. The interaction execution module 1420 is configured to perform user-digital assistant interaction in the interactive window using the webpage plugin based on the content in the second webpage.

[0160] The units and / or modules included in device 1400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in device 1400 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0161] It should be understood that one or more steps in the above methods can be performed by suitable electronic devices or combinations of electronic devices. Such electronic devices or combinations of electronic devices may include, for example, […]. Figure 1 The server 130, terminal device 110 and / or a combination of server 130 and terminal device 110.

[0162] Figure 15 A block diagram of an electronic device 1500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 15 The electronic device 1500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 15 The electronic device 1500 shown can be used to achieve Figure 1 Terminal device 110 and / or server 130, or Figure 14 Device 1400.

[0163] like Figure 15 As shown, electronic device 1500 is in the form of a general-purpose electronic device. Components of electronic device 1500 may include, but are not limited to, one or more processors or processing units 1510, memory 1520, storage device 1530, one or more communication units 1540, one or more input devices 1550, and one or more output devices 1560. Processing unit 1510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 1520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 1500.

[0164] Electronic device 1500 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 1500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 1520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 1530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data and accessible within electronic device 1500.

[0165] Electronic device 1500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 15 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 1520 may include computer program product 1525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0166] The communication unit 1540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 1500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 1500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0167] Input device 1550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 1560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 1500 can also communicate with one or more external devices (not shown) via communication unit 1540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 1500, or with any device that enables electronic device 1500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0168] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0169] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0170] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0171] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0172] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0173] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for information exchange, comprising: In response to detecting that a digital assistant is triggered for an interaction event based on a first webpage, a first link to the first webpage is provided to the webpage plugin, the webpage plugin being configured to perform webpage processing tasks; as well as In the user's interaction window with the digital assistant, the web page plugin is used to perform the user's interaction with the digital assistant based on the content of the first web page; The method further includes: receiving a first message in the interaction window between the user and the digital assistant, the first message including a first link to the first webpage; and detecting, in response to receiving the first message, that the digital assistant has been triggered for an interaction event based on the first webpage; and Provide the web page plugin with a first link to the first web page so that the web page plugin can obtain the content of the first web page from the first link; The interaction between the user and the digital assistant includes: in response to the first message including a first question or receiving a first question for the first webpage, providing the digital assistant's first response to the first question in the interaction window based on the content of the first webpage obtained from the webpage plugin.

2. The method according to claim 1, wherein the web page plugin obtains the content of the first web page from the first link, comprising: The web page plugin crawls the content of the first web page from the first link, or The web page plugin notifies the client device that initiated the first message to crawl the content of the first web page from the first link, and the content of the first web page crawled by the client device is reported to the web page plugin.

3. The method according to claim 1, further comprising: Based on the first message including the first link, it is determined that the web page plugin is enabled during the user's interaction with the digital assistant.

4. The method of claim 3, wherein determining that the web page plugin is enabled includes: Determine whether the web page plugin is configured to process the link type corresponding to the first link; as well as Based on the determination that the web page plugin is configured to process the link type corresponding to the first link, it is determined that the web page plugin is enabled.

5. The method of claim 3, wherein determining that the web page plugin is enabled comprises: In response to receiving the first message when an interaction is selected in a scenario related to web page processing, it is determined that the web page plugin is enabled; or In response to receiving the first message, it is determined, based on the first message, that a scenario related to web page processing has been selected and the web page plugin has been enabled.

6. The method of claim 5, wherein the interaction between the user and the digital assistant is based on the scenario configuration information and the content of the first webpage.

7. The method of claim 1, wherein performing the interaction between the user and the digital assistant further comprises at least one of the following: At least one recommendation instruction for the digital assistant is presented in the interactive window, the at least one recommendation instruction being determined based on the content of the first webpage; or In response to detecting a trigger on the first link in the first message, the content of the first webpage is presented in a first area, which is independent of the interactive window.

8. The method according to claim 1, wherein the web page plugin includes at least one of the following functions: Webpage link parsing function Web page content structuring function, Content caching function, or Web crawler functionality.

9. The method according to claim 1, further comprising: Present the content of the first webpage and the wake-up control of the digital assistant; as well as In response to the triggering of the wake-up control, it is detected that the digital assistant has been triggered for an interaction event based on the first webpage. Furthermore, in response to the triggering of the wake-up control, the web page plugin is provided with a first link to the first web page that has been acquired.

10. The method of claim 9, further comprising: In response to detecting a trigger on the wake-up control while displaying content in the first webpage, it is determined that the webpage plugin is enabled during the user's interaction with the digital assistant.

11. The method of claim 9, further comprising: In response to detecting a trigger on the wake-up control while displaying content in the first webpage, it is determined that a scenario related to webpage processing has been selected for interaction in the interactive window, and The user's interaction with the digital assistant is based on the scenario's configuration information and the content of the first webpage.

12. The method according to claim 9, further comprising: In response to the triggering of the wake-up control, the interaction window between the user and the digital assistant is presented, and guidance information is displayed in the interaction window, the guidance information including at least one recommendation instruction for the digital assistant, the at least one recommendation instruction being determined based on the content of the first webpage.

13. The method of claim 9, wherein the presentation area of ​​the interaction window between the user and the digital assistant is independent of the presentation area of ​​the first webpage.

14. The method of claim 9, further comprising: In response to detecting a switch from the first webpage to the second webpage while the interactive window is being presented, the content of the second webpage is presented; Provide the content of the second webpage to the webpage plugin; as well as In the interactive window, the web page plugin is used to perform the interaction between the user and the digital assistant based on the content of the second web page.

15. A device for information interaction, comprising: A webpage providing module is configured to provide a first link to the first webpage to a webpage plugin in response to detecting that a digital assistant is triggered to interact with the first webpage. The webpage plugin is configured to perform webpage processing tasks. The webpage providing module includes a message receiving module and a message-based triggering module. The message receiving module is configured to receive a first message in an interaction window between the user and the digital assistant, the first message including the first link to the first webpage. The message-based triggering module is configured to detect that the digital assistant is triggered to interact with the first webpage in response to receiving the first message, and to provide the first link to the first webpage to the webpage plugin so that the webpage plugin can obtain content from the first webpage from the first link. as well as An interaction execution module is configured to perform the interaction between the user and the digital assistant in an interaction window between the user and the digital assistant using the web page plugin based on the content of the first web page, wherein performing the interaction between the user and the digital assistant includes: in response to the first message including a first question or receiving a first question for the first web page, providing the digital assistant with a first response to the first question in the interaction window based on the content of the first web page obtained from the web page plugin.

16. An electronic device comprising: At least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 14 when executed by the at least one processing unit.

17. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 14.

Images