Information interaction method and apparatus, device, and storage medium

Abstract

According to embodiments of the present disclosure, information interaction and processing methods and apparatus, a device, and a storage medium are provided. The information interaction method comprises: in response to a preset operation, selecting a portable document format (PDF) plug-in in an interaction window between a user and a digital assistant; and during an interaction process between the user and the digital assistant, using the PDF plug-in to execute a task associated with a target PDF file. Therefore, during an interaction process with a digital assistant, a user can be supported to use a PDF plug-in to complete a task related to a PDF file. In this way, efficient assistance experience can be provided to a user.

Description

Information interaction method, device, equipment and storage medium

[0001] This application claims priority to the Chinese invention patent application entitled “Method, device, equipment and storage medium for information interaction” and application number 2023112851776, filed on September 28, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for information interaction. Background Art

[0003] With the development of information technology, various terminal devices can provide various services to people in work and life. For example, terminal devices can be deployed with applications that provide services. Terminal devices or their applications can provide users with assistance functions to help them complete tasks using the terminal device or application. How to make such assistance functions better serve users is a technical problem currently being explored.

[0004] Summary of the Invention

[0005] In a first aspect of the present disclosure, a method for information interaction is provided. The method comprises: selecting a portable document format (PDF) plug-in in a window for interacting with a digital assistant in response to a preset operation; and using the PDF plug-in to execute a task associated with a target PDF file during the interaction between the user and the digital assistant.

[0006] In a second aspect of the present disclosure, an information processing method is provided. The method includes: determining, for a target PDF file in a portable document format (PDF), a storage location of the target PDF file, the target PDF file being associated with an interaction window between a user and a digital assistant; and determining, based at least on the storage location, the availability of a PDF plug-in for the target PDF file, the PDF plug-in being used for PDF-type files, the availability indicating whether the PDF plug-in is used to perform a task on the target PDF file during interaction between the user and the digital assistant.

[0007] In a third aspect of the present disclosure, a device for information interaction is provided. The device includes: a plug-in selection module configured to, in response to a preset operation, select a portable document format (PDF) plug-in in a window in which a user interacts with a digital assistant; and a task execution module configured to use the PDF plug-in to execute a task associated with a target PDF file during the user's interaction with the digital assistant.

[0008] In a fourth aspect of the present disclosure, a device for information processing is provided. The device includes: a storage location determination module configured to determine the storage location of a target PDF file in a portable document format (PDF), the target PDF file being associated with a user's interaction window with a digital assistant; and an availability determination module configured to determine the availability of a PDF plug-in for the target PDF file based at least on the storage location, the PDF plug-in being used for PDF files, the availability indicating whether the PDF plug-in should be used to perform a task on the target PDF file during the user's interaction with the digital assistant.

[0009] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect or the second aspect.

[0010] In a sixth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect or the second aspect.

[0011] It should be understood that the content described in this summary section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0013] FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0014] 2A-2D are schematic diagrams illustrating example interfaces of a main session window when a PDF plug-in is selected according to some embodiments;

[0015] 3A-3B are schematic diagrams illustrating example interfaces of a main session window when a PDF plug-in is not selected according to some embodiments;

[0016] 4A-4B are schematic diagrams illustrating example interfaces of a sub-session window associated with a PDF file according to some embodiments;

[0017] 5A-5C are schematic diagrams illustrating example interfaces for interacting with a floating window associated with a PDF file according to some embodiments;

[0018] FIG6 illustrates an example process for using a PDF plug-in in a main session window according to some embodiments;

[0019] FIG7 illustrates an example process of using a PDF plug-in in a sub-session window and a floating window according to some embodiments;

[0020] FIG8 illustrates an example architecture for performing permission checking on a PDF file according to some embodiments;

[0021] FIG9 shows a schematic flow chart of an information interaction process according to some embodiments;

[0022] FIG10 is a flow chart showing a process of information processing according to some embodiments;

[0023] FIG11 shows a block diagram of an apparatus for information interaction according to some embodiments;

[0024] FIG12 shows a block diagram of an apparatus for information processing according to some embodiments; and

[0025] FIG13 shows a block diagram of a device capable of implementing various embodiments of the present disclosure. DETAILED DESCRIPTION

[0026] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0027] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below.

[0028] Herein, unless explicitly stated otherwise, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0029] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0030] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the type, scope of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and authorization should be obtained from relevant users in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0031] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to the software or hardware such as the electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0032] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message may be sent to the relevant user in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0033] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0034] FIG1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In the example environment 100, an office suite 120 is installed in a terminal device 110. A user 140 can interact with the office suite 120 via the terminal device 110 and / or an attached device of the terminal device 110. The office suite 120 can provide the user 140 with an integration of multiple applications or components. These applications can serve as application modules in the office suite 120. The applications integrated in the office suite 120 are sometimes also referred to as “office applications,” “office components,” “collaborative office platforms,” etc. As an example, the applications integrated in the office suite 120 may include, but are not limited to, one or more of the following: a chat application (also known as an instant messaging (IM) application), a document application, an audio and video conferencing application, an email application, a calendar application, a schedule application, a task application, and an objectives and key results (OKR) application.

[0035] In some embodiments, the office suite 120 can be downloaded as an application and installed on the terminal device 110. In some embodiments, the office suite 120 can also be accessed through other means, such as through a web page.

[0036] In the environment 100 of FIG1 , if the office suite 120 is activated, the terminal device 110 may present an interface 150 of the office suite 120 to the user 140. Interface 150 is sometimes also referred to as a client interface. Interface 150 may include various interfaces provided by the office suite 120, such as a conversation interface for presenting chat content, a video conferencing interface, a file sharing interface, and so on.

[0037] In some embodiments, the terminal device 110 communicates with the server 130 to enable the provision of services to the office suite 120. The terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.). The server 130 can be various types of computing systems / servers that can provide computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0038] It should be understood that the structure and function of each element in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. For example, the embodiments of the present disclosure can be applied to any suitable one or more applications, and are not limited to office suites.

[0039] As briefly mentioned above, a digital assistant can be provided to the user to help the user use the terminal device or application to complete the target task. In an embodiment of the present disclosure, a digital assistant is configured for user 140. The digital assistant can be implemented in any suitable form. In some embodiments, the digital assistant of user 140 can interact with the user 140 as a contact. Such a digital assistant can also correspond uniquely to user 140. In other words, the digital assistant of user 140 can be specific to or exclusive to user 140. If user 140 is an individual or person, such a digital assistant can be regarded as a personal digital assistant. It can be understood that in the embodiment of the present disclosure, the digital assistant is based on the authorization of user 140 to access the data to which permission is granted.

[0040] It should be understood that “uniquely corresponds” or similar expressions in the present disclosure are not intended to limit the digital assistant to be updated accordingly based on the interaction process between the user 140 and the digital assistant.

[0041] In an embodiment of the present disclosure, a digital assistant is used to interact with a user. An interaction window between the user and the digital assistant may be presented in a client interface. As an intelligent assistant, the digital assistant has intelligent dialogue and information processing capabilities. In the interaction window, the user enters an interactive message, and the digital assistant provides a reply message in response to the user input. The client interface for providing the digital assistant may correspond to a single-function application or a multi-function collaboration platform, such as an office suite or other collaboration platform capable of integrating multiple components.

[0042] The interaction window between the user and the digital assistant may include a conversation window, in which the interaction between the user and the digital assistant may be presented in the form of conversation messages. Alternatively or additionally, the interaction window between the user and the digital assistant may also include a non-conversation window. For example, a window in floating mode, in which the user can trigger the digital assistant to perform corresponding operations by inputting instructions, selecting shortcut commands, etc.

[0043] In some embodiments, after waking up the digital assistant through a preset application, user 140 interacts with the digital assistant through a primary session. The interaction window corresponding to the primary session may be referred to as a primary session window. As an example, the preset application may be an IM application in an office suite. In other examples, the preset application may also be another application or component.

[0044] In some embodiments, the digital assistant can be woken up and interacted with through applications other than the default applications. After waking up the digital assistant through these non-default applications, the user and the digital assistant can interact through a sub-session. The interaction window corresponding to a sub-session can be called a sub-session window. For example, the user can wake up the digital assistant through a page in the document application and then interact with the digital assistant through a sub-session.

[0045] As briefly mentioned above, digital assistants can assist users in using terminal devices or applications. PDF files are a commonly used type of file. Therefore, if digital assistants can assist users in processing PDF files during the interaction between users and digital assistants, it can help users improve their work efficiency.

[0046] According to some embodiments of the present disclosure, an improved scheme for information interaction is proposed. In this embodiment of the present disclosure, in response to a preset operation, a PDF plug-in is selected in the interaction window between the user and the digital assistant. Then, during the interaction between the user and the digital assistant, the PDF plug-in is used to perform tasks associated with the target PDF file. This allows users to use the PDF plug-in to complete tasks related to PDF files during the interaction with the digital assistant. In this way, an efficient assistance experience is provided to users.

[0047] According to some embodiments of the present disclosure, an improved scheme for information processing is proposed. In an embodiment of the present disclosure, for a target PDF file, the storage location of the target PDF file is determined, and the target PDF file is associated with an interaction window between a user and a digital assistant. Based at least on the storage location, the availability of a PDF plug-in for the target PDF file is determined, and the PDF plug-in is used for files of the PDF type. Availability indicates whether a PDF plug-in is used to perform a task on the target PDF file during the interaction between the user and the digital assistant. In this embodiment, whether a PDF plug-in is enabled for the PDF file is determined based on the storage location of the PDF file. Thus, permission management in the use of the PDF plug-in is implemented to avoid inappropriate copying or editing of the content of the PDF file.

[0048] In some embodiments, in response to a preset operation, terminal device 110 may select a PDF plug-in in the interaction window between user 140 and the digital assistant. During the interaction between user 140 and the digital assistant, terminal device 110 may use the PDF plug-in to perform tasks associated with the target PDF file. For example, the plug-in may summarize the content of the PDF file, translate at least part of the content of the PDF file, answer questions about the PDF file, and so on.

[0049] The preset operations described herein can trigger the automatic selection of the PDF plug-in and can include one or more operations. The preset operations can be related to the type of interactive window in which the user 140 interacts with the digital assistant. Some example operations are described below with reference to the accompanying drawings. In the examples below, the digital assistant is shown as having the name "XX Assistant", but it should be understood that this is merely exemplary and is not intended to limit the scope of the present disclosure. In an embodiment of the present disclosure, the digital assistant can have any suitable name, and the user 140 can customize the name of the digital assistant.

[0050] PDF plugin in the main session window

[0051] In some embodiments, user 140 can interact with the digital assistant in the main session window. In some embodiments, terminal device 110 can display a plug-in selection control in the main session window for selecting a plug-in to use in the main session window. If user 140 selects a PDF plug-in in the main session window, terminal device 110 can select the PDF plug-in for the target PDF file. The target PDF file can be the PDF file indicated by user 140 in the main session window. For example, after selecting the PDF plug-in, the user can upload the PDF file or send a link to the PDF file in the main session window.

[0052] In some embodiments, the display of the plug-in selection control may be triggered by the opening of a new topic. For example, if an instruction to open a new topic is received from user 140 in the main conversation window, terminal device 110 may display the plug-in selection control in the main conversation window.

[0053] An example interface for selecting a PDF plug-in in the main conversation window is described with reference to Figures 2A to 2D. As shown in Figure 2A, a main conversation window 210 between a user and a digital assistant is presented in the client interface. User 140 triggers a new topic control 211 for starting a new topic, for example, by clicking on the new topic control 211. In response, a topic separation mark 212 may be displayed in main conversation window 210 to indicate the start of a new topic and to separate different topics, as shown in Figure 2B. Accordingly, a plug-in selection control 213 may be displayed in main conversation window 210. If user 140 selects the PDF plug-in through the plug-in selection control 213, terminal device 110 may enable the PDF plug-in, as shown in Figure 2C.

[0054] In addition, in some embodiments, in response to the selection or activation of the PDF plug-in, prompt information about the PDF plug-in can be displayed in the main session window. For example, as shown in Figure 2C, prompt information 214 informs user 140 that the PDF plug-in can be used to interact and prompts the user to indicate the PDF file desired to work with.

[0055] As shown in FIG2C , an indication regarding a PDF file is received in the main conversation window 210. In this example, the user has sent a PDF file, also referred to as a target PDF file, in the main conversation window 210. Subsequently, during the user 140's interaction with the digital assistant, the PDF plug-in can be used to perform tasks related to the target PDF file.

[0056] In some embodiments, to further improve user efficiency, one or more instructions related to PDF files may be provided to the user for selection. Each of these instructions corresponds to a preset task, such as summarizing a PDF file, translating all or part of a PDF file's content, etc. Such instructions are also referred to as shortcut instructions. For example, such instructions may be displayed in the main session window. If user 140 selects one of these instructions, the preset task corresponding to that instruction may be executed.

[0057] 2C , after the user uploads the PDF file, a message 215 from the digital assistant is displayed in the main conversation window 210. For example, the message 215 may include instructions 221, 222, and 223. The user 140 can conveniently indicate the task they want the digital assistant to perform by selecting the displayed instruction.

[0058] It should be understood that the display of the instructions is only for the convenience of the user to quickly make instructions. The user can also send the task related to the target PDF file that the digital assistant is expected to perform in the input box 230.

[0059] In some embodiments, the simultaneous display of a conversation window and a PDF file can be supported. That is, the conversation window and the PDF file can be displayed in different areas of the client interface. Referring to Figure 2D, for example, if the user clicks on the PDF file sent in the main conversation window 210, the page of the PDF file can be displayed in area 250 of the interface. In other words, the conversation window can be displayed in separate pages from the PDF file. In this way, the user can interact with the digital assistant while viewing the PDF file without switching pages. This further helps to improve the user's interactive experience with the digital assistant. It should be understood that Figure 2D is merely exemplary, and the simultaneous display of a separate conversation window and a PDF file can also be supported.

[0060] The above describes an example embodiment in which a user selects a PDF plug-in in the main session window. In some embodiments, user input indicating a target PDF file is received in the main session window, for example, the user uploads a PDF file or sends a link to a PDF file in the main session window. In response to this user input, if the user does not select a PDF plug-in, one or more instructions related to PDF files may be displayed in the main session window. Each of these instructions corresponds to a pre-set task, such as summarizing a PDF file, translating all or part of the content of a PDF file, etc. If user 140 selects one of the displayed instructions, the PDF plug-in may be automatically selected. In other words, the PDF plug-in is automatically enabled for the target PDF file.

[0061] In some embodiments, in response to the user selecting a certain instruction in the displayed instructions, the PDF plug-in can be used to perform a preset task corresponding to the selected instruction on the target PDF file. In some embodiments, in response to the user selecting a certain instruction in the displayed instructions, a new topic can be automatically opened.

[0062] An example is described below with reference to Figures 3A and 3B. As shown in Figure 3A, user 140 can send a PDF file or a link to a PDF file to main session window 310. Specifically, in this example, user 140 sends a PDF file titled "Introduction Outline." Because user 140 has not previously selected the PDF plug-in, a prompt message 320 from the digital assistant may be displayed in main session window 310, as shown in Figure 3B. Prompt message 320 may include several instructions 321, 322, and 323 related to PDF files.

[0063] In this example, user 140 selects instruction 323. In response to instruction 323 being selected, terminal device 110 may automatically select a PDF plug-in for user 140. Accordingly, terminal device 110 may perform the task corresponding to instruction 323. In this example, the digital assistant sends a reply message 330 to user 140.

[0064] In some embodiments, if the user does not select any displayed instructions but directly enters a message in the input box, the message entered by the user may not be replied according to the PDF file indicated by the user 140 .

[0065] Alternatively, in some embodiments, in response to receiving user input indicating a target PDF file in the main session window, the PDF plug-in can be automatically selected. For example, once user 140 indicates a PDF file in the main session window, the PDF plug-in can be directly selected for user 140 without further input or indication from the user.

[0066] The above describes how to select and use the PDF plug-in in the main session window. You can also select and use the PDF plug-in in other types of interactive windows.

[0067] In some embodiments, a PDF plug-in can be automatically selected in response to invoking the digital assistant via a target PDF file. For example, user 140 can open a PDF file locally on a user device (e.g., terminal device 110) or a PDF file stored in the cloud. User 140 can then invoke the digital assistant within the opened PDF file. In other words, if user 140 invokes the digital assistant via a PDF file, a PDF plug-in can be automatically selected for user 140 without requiring additional input from the user.

[0068] In some embodiments, a sub-session window between the user and the digital assistant can be triggered via the target PDF file. In response to this, a PDF plug-in can be automatically selected.

[0069] An example is described with reference to FIG4A . User 140 opens a PDF file titled "Introduction Outline." User 140 clicks a control 401 displayed on the file page for invoking a digital assistant. In response to control 401 being triggered, terminal device 110 may present a separate conversation window 410 between the user and the digital assistant. Furthermore, terminal device 110 may automatically select a PDF plug-in.

[0070] Additionally, in some embodiments, the terminal device 110 can also use a PDF plug-in to execute a preset task on a target PDF file and display the results of the task in the main session window. An example is described with reference to FIG4B . FIG4B illustratively shows an interface displayed on a mobile device, but this is merely an example; similar displays can be performed on other types of devices. In this example, the preset task is to summarize the contents of a PDF file. The results of executing the preset task 430 , i.e., a summary of the contents of the PDF file "Introduction Outline," can be displayed in the sub-session window 420 .

[0071] Additionally, in some embodiments, terminal device 110 may also display one or more recommendation controls related to the target PDF file in the sub-session window. Each recommendation control indicates the next action of the digital assistant. In other words, questions or instructions to be issued to the digital assistant may be recommended to user 140. Continuing with the example of FIG4B , recommendation controls 431, 432, and 433 are displayed in sub-session window 420.

[0072] In some embodiments, a non-conversational window, such as a floating window, can be triggered via a target PDF file to display between the user and the digital assistant. In response, a PDF plug-in can be automatically selected. One or more instructions related to the PDF plug-in can be displayed in the floating window. Each instruction corresponds to a pre-set task.

[0073] An example is described with reference to FIG5A . An open PDF file titled "Introduction Outline" is displayed in the interface. In response to a user selecting a portion of the PDF file, a floating window 510 is displayed. In floating window 510, commands such as "Explain," "Translate," and "Summarize" are displayed. These commands may be shortcut commands registered with server 130.

[0074] In some embodiments, the command displayed in the floating window may have multiple sub-commands for the user to select. Referring to FIG5B , if the user selects the "Translate" command, a menu 515 may be displayed. Menu 515 displays multiple sub-commands translated into different languages. The user can make a selection based on their needs.

[0075] As another example, Figure 5C shows an example interface on another type of user device (e.g., a mobile device). The interface displays an open PDF file titled "XXX Project Plan." In response to the user selecting a portion of the PDF file, a floating window 530 is displayed. In floating window 530, commands such as "Explain," "Translate," and "Summarize" are displayed.

[0076] In the above description, the number, shape, etc. of each element shown in the drawings are exemplary and not intended to be limiting. In particular, the number of instructions shown is also exemplary and not intended to be limiting. In the embodiments of the present disclosure, any suitable number of instructions may be shown.

[0077] Example usage process and permission check

[0078] Referring now to FIG6 , an example process 600 for using a PDF plug-in in the main conversation window between a user and a digital assistant will be described. At block 601, user 140 uploads a PDF file in the main conversation window. At block 602, terminal device 101 determines whether the PDF plug-in is selected. If the PDF plug-in is selected, for example, via the plug-in selection control described above, process 600 proceeds to block 605. At block 605, one or more instructions may be recommended based on the content of the PDF file. For example, as described above with reference to FIG2C , instructions 221, 222, and 223 may be displayed in the main conversation window.

[0079] If the PDF plug-in is not selected, process 600 proceeds to block 603. At block 603, terminal device 110 may provide multiple instructions to guide the user in selecting a PDF plug-in. For example, as described above with reference to FIG3B , instructions 321, 322, and 323 may be displayed in main conversation window 310. If the user clicks on the displayed instructions, process 600 proceeds to block 604. At block 604, terminal device 110 may open a new topic, automatically select the PDF plug-in, and execute the selected instructions, such as described above with reference to FIG3B . Process 600 then proceeds to block 605.

[0080] 6 , if the user clicks on the uploaded PDF file, process 600 proceeds to block 606. At block 606, the PDF file can be viewed in a paginated manner. That is, the PDF file can be displayed simultaneously with the main session window, as described above with reference to FIG. 2D .

[0081] FIG7 illustrates an example process 700 for using a PDF plug-in in a sub-session window and a floating window according to some embodiments. In block 701, user 140 uploads a PDF file, for example, via an IM application. If user 140 clicks on the PDF file, process 700 proceeds to block 702. In block 702, terminal device 110 may open the PDF file in a tab for the user to view. In block 703, terminal device 110 may send guidance information in the sub-session window, for example, as a digital assistant. Alternatively or additionally, terminal device 110 may also automatically summarize the PDF file and / or recommend instructions to user 140, such as described above with reference to FIG4B .

[0082] Continuing with block 702, if a section of content is selected in the open PDF file, process 700 proceeds to block 704. In block 704, terminal device 110 may invoke and display a pop-up window. In block 705, a free command Q&A session may be performed via the pop-up window. In other words, the user may enter interactive messages to the digital assistant. Alternatively, the user may select a displayed shortcut command, such as interpreting the content in the file, translating all or part of the content in the file, or summarizing the file.

[0083] Considering that the PDF plug-in can perform operations such as copying PDF files, in order to ensure data security, in some embodiments, the PDF files involved in the interaction between the user and the digital assistant can be checked for permissions. Figure 8 shows an example architecture for checking permissions on PDF files according to some embodiments. In some embodiments, the PDF file that the user 140 may indicate may include two sources, one is a PDF file 801 local to the user's device, and the other is a PDF file 803 stored in the cloud. In view of this, in the interaction between the user and the digital assistant such as described above, the storage location of the PDF file indicated by the user can be determined. Then, based on the storage location, the availability of the PDF plug-in for the target PDF file is determined. That is, it is determined whether the PDF plug-in can be used to perform tasks on the target PDF file.

[0084] As shown in Figure 8, for PDF files stored locally on the user's device, the usage policy shown in box 802 can be implemented. That is, the digital assistant can answer the user's messages based on the PDF content and can recommend guiding questions to the user. For PDF files stored in the cloud, it can be determined whether user 140 has copy permission for the PDF file. If user 140 has copy permission, the usage policy in box 8025 can be implemented. If user 140 does not have copy permission, the usage policy shown in box 806 can be implemented. That is, the digital assistant cannot answer the user's messages based on the PDF content and does not recommend guiding questions to the user.

[0085] In this way, it can be ensured that the PDF plug-in is used properly for PDF files.

[0086] Example Operation of PDF Plug-in

[0087] In embodiments of the present disclosure, a PDF plug-in can implement any suitable operations to perform tasks associated with a target PDF file. Example operations of the PDF plug-in are described below. In some embodiments, to perform a task, the content of the target PDF file can be extracted. For example, blocks within the target PDF file can be parsed, such as titles, author information, text, images, formulas, etc. The parsed blocks can then be segmented, concatenated, and combined to obtain complete text blocks, table blocks, image blocks, etc., and the reading order of these blocks can be restored.

[0088] In some embodiments, a PDF file understanding module can be used to extract the content of a target PDF file. The PDF file understanding module may include a basic information parsing submodule, a table extraction submodule, an image extraction submodule, a formula extraction submodule, a semantic tag classification submodule, a layout segmentation submodule, and a paragraph concatenation submodule. The basic information parsing submodule can be used to obtain basic text information and bounding box information from the target PDF file. The table extraction submodule can be used to identify and extract table content from each page. The image extraction submodule can be used to identify and extract image content from each page. The formula extraction submodule can be used to identify and extract formula content from each page. The layout segmentation submodule can be used to segment each page of the PDF file into multiple visually isolated blocks and restore the reading order of these blocks according to a preset order. The preset order can be, for example, from top to bottom or from left to right. The semantic tag classification submodule can be used to identify the labels of the blocks segmented by the layout segmentation submodule, such as, but not limited to, titles, text, tables, images, headers, and footers. The Paragraph Join submodule can be used to join the text in each block into complete paragraphs based on the tags of each block. After processing the blocks on each page, the paragraphs that span the pages can be joined.

[0089] It should be understood that the PDF file understanding described above is only exemplary and is not intended to limit the scope of the present disclosure. In the embodiments of the present disclosure, the PDF plug-in can use any suitable technology to understand the PDF file.

[0090] After the content of the target PDF file has been extracted, further processing can be performed according to the extracted content to perform the task. In some embodiments, the task can be performed according to the paragraphs obtained by splicing.

[0091] As an example, a message from user 140 may instruct to rewrite the content of a target PDF file. Hereinafter, the content that the user desires to rewrite is also referred to as the target content. One or more paragraphs extracted from the target content may be determined and processed separately. For example, prompt words may be generated for each of these paragraphs, and based on the prompt words, a machine learning model may be used to generate corresponding rewriting results for these paragraphs. The corresponding rewriting results for these paragraphs may be merged and fed back to user 140 as the rewriting results for the target content.

[0092] As another example, the message from user 140 may instruct a summary of the target PDF file. A map-reduce process may be employed to generate a summary of the target PDF file. Specifically, the content of each paragraph may be summarized. For example, a prompt word may be generated for each paragraph, and a summary of each paragraph may be generated using a machine learning model. Subsequently, a summary of the entire target PDF file may be generated based on the summaries of each paragraph in the target PDF file. For example, a prompt word for the target PDF file may be generated based on the summaries of each paragraph, and a summary of the entire target PDF file may be generated using a machine learning model.

[0093] The above describes the exemplary operations of the PDF plug-in using content rewriting and summarization as examples. It is understandable that, based on understanding the PDF file to extract its content, the PDF plug-in can use any suitable method to perform tasks associated with the PDF file.

[0094] Example procedures, devices, and equipment

[0095] FIG9 shows a flow chart of a process 900 of information interaction according to some embodiments. The process 900 may be implemented at the terminal device 110. The process 900 is described below with reference to FIG1.

[0096] In box 910, the terminal device 110 selects a portable document format PDF plug-in in the interaction window between the user and the digital assistant in response to a preset operation.

[0097] In block 920 , during the user's interaction with the digital assistant, the terminal device 110 uses the PDF plug-in to perform tasks associated with the target PDF file.

[0098] In some embodiments, the target PDF file is stored locally on the user's user device, or the target PDF file is stored in the cloud and the user has copy permission for the target PDF file.

[0099] In some embodiments, the interaction window includes a main session window between the user and the digital assistant, and the preset operations include: displaying a plug-in selection control in the main session window for selecting a plug-in to be used in the main session window; and receiving a user selection of a PDF plug-in in the main session window.

[0100] In some embodiments, presenting the plug-in selection control in the main conversation window includes: receiving an instruction to start a new topic from the user in the main conversation window; and presenting the plug-in selection control in the main conversation window in response to the instruction to start a new topic.

[0101] In some embodiments, using a PDF plug-in to perform a task associated with a target PDF file includes: displaying one or more instructions related to a PDF type file in a main session window, each of the one or more instructions corresponding to a preset task; and in response to a first instruction of the one or more instructions being selected, using the PDF plug-in to perform the preset task corresponding to the first instruction on the target PDF file.

[0102] In some embodiments, the session window includes a main session window between the user and the digital assistant, and the preset operations include: receiving user input indicating a target PDF file in the main session window; in response to the user input, displaying one or more instructions related to PDF type files in the main session window, each of the one or more instructions corresponding to a preset task; and receiving a selection of a second instruction among the one or more instructions.

[0103] In some embodiments, using the PDF plug-in to execute the task associated with the target PDF file includes: using the PDF plug-in to execute a preset task corresponding to the second instruction on the target PDF file.

[0104] In some embodiments, the conversation window includes a main conversation window between the user and the digital assistant, and the preset operation includes: receiving user input indicating a target PDF file in the main conversation window.

[0105] In some embodiments, the preset operation includes: invoking the digital assistant via the target PDF file.

[0106] In some embodiments, the interaction window includes a sub-session window between the user and the digital assistant, and process 900 further includes: displaying the result of performing a preset task on the target PDF file using the PDF plug-in in the sub-session window.

[0107] In some embodiments, process 900 further includes: displaying one or more recommendation controls related to the target PDF file in the sub-session window, each of the one or more recommendation controls indicating a next operation of the digital assistant.

[0108] In some embodiments, the interaction window includes a non-session window between the user and the digital assistant, and process 900 also includes: displaying one or more instructions related to the PDF plug-in in the non-session window, each of the one or more instructions corresponding to a preset task.

[0109] FIG10 is a flow chart of an information processing process 1000 according to some embodiments. The process 1000 may be implemented in the terminal device 110 or the server 130. The process 1000 will be described below with reference to FIG1 , taking the implementation in the terminal device 110 as an example.

[0110] In box 1010, for a target PDF file of a portable document format PDF type, the terminal device 110 determines a storage location of the target PDF file, and the target PDF file is associated with an interaction window between the user and the digital assistant.

[0111] In box 1020, the terminal device 110 determines the availability of the PDF plug-in for the target PDF file based at least on the storage location, where the PDF plug-in is used for files of PDF type, and the availability indicates whether the PDF plug-in is used to perform tasks on the target PDF file during the user's interaction with the digital assistant.

[0112] In some embodiments, determining the availability of the PDF plug-in for the target PDF file includes determining that the PDF plug-in is available for the target PDF file in response to the target PDF file being stored on a user device of the user.

[0113] In some embodiments, determining the availability of a PDF plug-in for a target PDF file includes: in response to the target PDF file being stored in a cloud, determining whether a user has copy permission for the target PDF file; in response to the user having copy permission for the target PDF file, determining that the PDF plug-in is available for the target PDF file; and in response to the user not having copy permission for the target PDF file, determining that the PDF plug-in is unavailable for the target PDF file.

[0114] 11 shows a schematic structural block diagram of an apparatus 1100 for information interaction according to certain embodiments of the present disclosure. Apparatus 1100 may be implemented as or included in a terminal device 110. Each module / component in apparatus 1100 may be implemented by hardware, software, firmware, or any combination thereof.

[0115] The apparatus 1100 includes a plug-in selection module 1110 configured to select a portable document format (PDF) plug-in in response to a preset operation in a window in which a user interacts with a digital assistant. The apparatus 1100 also includes a task execution module 1120 configured to use the PDF plug-in to execute a task associated with a target PDF file during the user's interaction with the digital assistant.

[0116] In some embodiments, the target PDF file is stored locally on the user's user device, or the target PDF file is stored in the cloud and the user has copy permission for the target PDF file.

[0117] In some embodiments, the interaction window includes a main session window between the user and the digital assistant, and the preset operations include: displaying a plug-in selection control in the main session window for selecting a plug-in to be used in the main session window; and receiving a user selection of a PDF plug-in in the main session window.

[0118] In some embodiments, presenting the plug-in selection control in the main conversation window includes: receiving an instruction to start a new topic from the user in the main conversation window; and presenting the plug-in selection control in the main conversation window in response to the instruction to start a new topic.

[0119] In some embodiments, the task execution module 1120 is further configured to: display one or more instructions related to PDF type files in the main session window, each of the one or more instructions corresponds to a preset task; and in response to a first instruction among the one or more instructions being selected, use the PDF plug-in to execute the preset task corresponding to the first instruction on the target PDF file.

[0120] In some embodiments, the session window includes a main session window between the user and the digital assistant, and the preset operations include: receiving user input indicating a target PDF file in the main session window; in response to the user input, displaying one or more instructions related to PDF type files in the main session window, each of the one or more instructions corresponding to a preset task; and receiving a selection of a second instruction among the one or more instructions.

[0121] In some embodiments, the task execution module 1120 is further configured to use a PDF plug-in to execute a preset task corresponding to the second instruction on the target PDF file.

[0122] In some embodiments, the conversation window includes a main conversation window between the user and the digital assistant, and the preset operation includes: receiving user input indicating a target PDF file in the main conversation window.

[0123] In some embodiments, the preset operation includes: invoking the digital assistant via the target PDF file.

[0124] In some embodiments, the interaction window includes a sub-session window between the user and the digital assistant, and the device 1100 also includes: a result display module configured to display the results of performing a preset task on the target PDF file using the PDF plug-in in the sub-session window.

[0125] In some embodiments, the device 1100 also includes: a recommendation control display module, configured to display one or more recommendation controls related to the target PDF file in the sub-session window, each of the one or more recommendation controls indicating the next operation of the digital assistant.

[0126] In some embodiments, the interactive window includes a non-session window between the user and the digital assistant, and the device 1100 also includes: an instruction display module, configured to display one or more instructions related to the PDF plug-in in the non-session window, each of the one or more instructions corresponding to a preset task.

[0127] Figure 12 shows a schematic block diagram of an apparatus 1200 for information processing according to certain embodiments of the present disclosure. Apparatus 1200 may be implemented as or included in terminal device 110 or server 130. Each module / component in apparatus 1200 may be implemented by hardware, software, firmware, or any combination thereof.

[0128] Apparatus 1200 includes a storage location determination module 1210 configured to determine, for a target PDF file of the portable document format (PDF) type, a storage location of the target PDF file, the target PDF file being associated with a window in which a user interacts with a digital assistant. Apparatus 1200 also includes an availability determination module 1220 configured to determine, based at least on the storage location, the availability of a PDF plug-in for the target PDF file, the PDF plug-in being used for PDF-type files, the availability indicating whether the PDF plug-in should be used to perform a task on the target PDF file during interaction between the user and the digital assistant.

[0129] In some embodiments, the availability determination module 1220 is further configured to determine that a PDF plug-in is available for the target PDF file in response to the target PDF file being stored on the user device of the user.

[0130] In some embodiments, the availability determination module 1220 is further configured to: in response to the target PDF file being stored in the cloud, determine whether the user has copy permission for the target PDF file; in response to the user having copy permission for the target PDF file, determine that the PDF plug-in is available for the target PDF file; and in response to the user not having copy permission for the target PDF file, determine that the PDF plug-in is not available for the target PDF file.

[0131] FIG13 shows a block diagram of an electronic device 1300 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 1300 shown in FIG13 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 1300 shown in FIG13 may be used to implement the terminal device 110 or the server 130 of FIG1 .

[0132] As shown in FIG13 , electronic device 1300 is a general-purpose electronic device. Components of electronic device 1300 may include, but are not limited to, one or more processors or processing units 1310, memory 1320, storage device 1330, one or more communication units 1340, one or more input devices 1350, and one or more output devices 1360. Processing unit 1310 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 1320. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of electronic device 1300.

[0133] The electronic device 1300 typically includes a plurality of computer storage media. Such media can be any accessible media that is accessible to the electronic device 1300, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 1320 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 1330 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data and can be accessed within the electronic device 1300.

[0134] The electronic device 1300 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 13 , a disk drive for reading from or writing to a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 1320 may include a computer program product 1325 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0135] The communication unit 1340 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 1300 can be implemented as a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 1300 can operate in a networked environment using a logical connection to one or more other servers, a network personal computer (PC), or another network node.

[0136] Input device 1350 may be one or more input devices, such as a mouse, keyboard, or trackball. Output device 1360 may be one or more output devices, such as a display, a speaker, or a printer. Electronic device 1300 may also communicate with one or more external devices (not shown) via communication unit 1340, as needed. External devices such as storage devices, display devices, and the like may also be used to communicate with one or more devices that allow a user to interact with electronic device 1300, or with any device that allows electronic device 1300 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0137] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0138] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0139] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0140] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0141] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0142] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. An information interaction method, comprising: In response to a preset operation, a portable document format PDF plug-in is selected in an interaction window between the user and the digital assistant; as well as During the interaction between the user and the digital assistant, the PDF plug-in is used to perform tasks associated with the target PDF file.

2. The method according to claim 1, wherein The target PDF file is stored locally on the user's device, or The target PDF file is stored in the cloud and the user has copy permission for the target PDF file.

3. The method according to claim 1, wherein the interaction window comprises a main conversation window between the user and the digital assistant, and the preset operation comprises: In the main session window, display a plug-in selection control for selecting a plug-in to be used in the main session window; as well as In the main session window, a selection of the PDF plug-in by the user is received.

4. The method according to claim 3, wherein presenting the plug-in selection control in the main session window comprises: receiving an instruction to start a new topic from the user in the main conversation window; as well as In response to the instruction to start a new topic, the plug-in selection control is displayed in the main conversation window.

5. The method of claim 3, wherein using the PDF plug-in to perform a task associated with a target PDF file comprises: Displaying one or more instructions related to a PDF file in the main session window, each of the one or more instructions corresponding to a preset task; as well as In response to a first instruction among the one or more instructions being selected, the PDF plug-in is used to execute a preset task corresponding to the first instruction on the target PDF file.

6. The method according to claim 1, wherein the conversation window comprises a main conversation window between the user and the digital assistant, and the preset operation comprises: receiving, in the main session window, a user input indicating the target PDF file; In response to the user input, displaying one or more instructions related to a PDF-type file in the main session window, each of the one or more instructions corresponding to a preset task; as well as A selection of a second instruction of the one or more instructions is received.

7. The method of claim 6, wherein using the PDF plug-in to perform a task associated with a target PDF file comprises: The PDF plug-in is used to execute a preset task corresponding to the second instruction on the target PDF file.

8. The method according to claim 1, wherein the conversation window comprises a main conversation window between the user and the digital assistant, and the preset operation comprises: A user input indicating the target PDF file is received in the main session window.

9. The method according to claim 1, wherein the preset operation comprises: The operation of the digital assistant is invoked via the target PDF file.

10. The method according to claim 9, wherein the interaction window comprises a sub-conversation window between the user and the digital assistant, and the method further comprises: In the sub-session window, the result of using the PDF plug-in to execute the preset task on the target PDF file is displayed.

11. The method according to claim 10, further comprising: In the sub-session window, one or more recommendation controls related to the target PDF file are displayed, and each of the one or more recommendation controls indicates the next operation of the digital assistant.

12. The method according to claim 9, wherein the interaction window comprises a non-separated conversation window between the user and the digital assistant, and the method further comprises: The non-session window displays one or more instructions related to the PDF plug-in, and each of the one or more instructions corresponds to a preset task.

13. An information interaction method, comprising: For a target PDF file of a portable document format PDF type, determining a storage location of the target PDF file, wherein the target PDF file is associated with an interaction window between a user and a digital assistant; as well as Based at least on the storage location, the availability of a PDF plug-in for the target PDF file is determined, the PDF plug-in being used for files of PDF type, the availability indicating whether to use the PDF plug-in to perform a task on the target PDF file during the interaction between the user and the digital assistant.

14. The method of claim 13, wherein determining the availability of a PDF plug-in for the target PDF file comprises: In response to the target PDF file being stored on a user device of the user, it is determined that the PDF plug-in is available for the target PDF file.

15. The method of claim 13, wherein determining the availability of a PDF plug-in for the target PDF file comprises: In response to the target PDF file being stored in the cloud, determining whether the user has copy permission for the target PDF file; In response to the user having copy permission for the target PDF file, determining that the PDF plug-in can be used for the target PDF file; as well as In response to the user not having copy authority for the target PDF file, determining that the PDF plug-in is unavailable to the target PDF file.

16. A device for information interaction, comprising: A plug-in selection module, configured to select a portable document format PDF plug-in in the interaction window between the user and the digital assistant in response to a preset operation; as well as The task execution module is configured to use the PDF plug-in to execute the task associated with the target PDF file during the interaction between the user and the digital assistant.

17. An apparatus for information processing, comprising: A storage location determination module is configured to determine the storage location of a target PDF file of a portable document format PDF type, wherein the target PDF file is associated with an interaction window between a user and a digital assistant; as well as An availability determination module is configured to determine the availability of a PDF plug-in for the target PDF file based at least on the storage location, the PDF plug-in being used for files of PDF type, the availability indicating whether to use the PDF plug-in to perform a task on the target PDF file during the interaction between the user and the digital assistant.

18. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to execute the method according to any one of claims 1 to 12 or the method according to any one of claims 13 to 15.

19. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 12 or any one of claims 13 to 15.

Images