Method, apparatus, device and storage medium for session interaction

Abstract

A method, apparatus, device, medium, and program product for conversational interaction are provided. The proposed method includes: presenting first information on a conversational interface for a conversation between a first user and at least one second user, the first information including a first message; and presenting second information, the second information including a first instruction issued to an intelligent system, the first instruction being at least related to the first message.

Description

Technical Field

[0001] The various example implementations generally relate to the field of computers, and in particular to methods, apparatuses, devices, computer-readable storage media, and computer program products for conversational interaction. Background Technology

[0002] With the development of information technology, various terminal devices can provide people with a variety of services in work and life. Applications providing these services can be deployed on these terminal devices. Terminal devices or applications can provide users with intelligent system functions to assist them in using the devices or applications. How to improve the flexibility of user interaction with intelligent systems is a technical problem that needs to be explored. Summary of the Invention

[0003] In a first aspect, a method for conversational interaction is provided. The method includes: presenting first information on a conversational interface for a conversation between a first user and at least one second user, the first information including a first message; and presenting second information, the second information including a first instruction issued to an intelligent system, the first instruction being at least related to the first message.

[0004] In a second aspect, a device for conversational interaction is provided. The device includes: a first information presentation module configured to present first information on a conversational interface for a conversation between a first user and at least one second user, the first information including a first message; and a second information presentation module configured to present second information, the second information including a first instruction issued to an intelligent system, the first instruction being at least related to the first message.

[0005] In a third aspect, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect, a computer program product is provided, including computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method of the first aspect.

[0008] In this way, intelligent system command recommendations related to conversation messages can be provided during user conversations, thereby improving interaction efficiency.

[0009] It should be understood that the content described in this section is not intended to limit the key or important features to be protected, nor is it intended to restrict the scope of protection. Other features will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the various implementations will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram of an example environment in which one or more scenarios can be implemented is shown; Figures 2A to 2E A schematic diagram of an example interface for conversational interaction in an automatically triggered instruction recommendation scenario is shown, based on certain situations. Figures 3A to 3F A schematic diagram of an example interface for conversational interaction in an automatically triggered instruction recommendation scenario is shown, depending on other circumstances. Figures 4A to 4D A schematic diagram of an example interface for conversational interaction in a user-triggered instruction recommendation scenario is shown, based on certain situations. Figure 5 A flowchart illustrating an example process of conversational interaction under certain circumstances is shown; Figure 6 A schematic block diagram of an example device for conversational interaction under certain circumstances is shown; and Figure 7 A block diagram of an electronic device capable of implementing multiple illustrative scenarios is shown. Detailed Implementation

[0011] The examples in this document will now be described in more detail with reference to the accompanying drawings. While some examples are shown in the drawings, it should be understood that solutions can be implemented in various forms and should not be construed as limited to the examples presented herein. Rather, these examples are provided to provide a more thorough and complete understanding of the solutions. It should be understood that the drawings and examples in this document are for illustrative purposes only and are not intended to limit the scope of protection of the solutions.

[0012] It should be noted that the headings of any section / subsection provided herein are not restrictive. Various examples are described throughout this document, and examples of any type may be included under any section / subsection. Furthermore, examples described in any section / subsection may be combined in any way with any other examples described in the same section / subsection and / or different sections / subsections.

[0013] In the description of the examples in this document, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an example" or "the example" should be understood as "at least one example". The term "some examples" should be understood as "at least some examples". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0014] The examples in this document may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and provisions. In the examples presented herein, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0015] In this manual and the sample solutions, any processing of personal information will be conducted only under legal grounds (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0016] In this paper, an intelligent system refers to a system capable of autonomous control based on machine learning models. An intelligent system, for example, can make decisions and autonomously execute actions based on machine learning models to achieve preset goals or complete preset tasks. An intelligent system can be an automated program that understands user intent and can utilize models or invoke tools to complete various types of tasks. In some contexts, examples of intelligent systems may include, but are not limited to: agents, bots, chatbots, digital avatars, intelligent customer service, digital assistants, etc. Alternatively, an intelligent system can also be an intelligent role implemented based on machine learning models. An "intelligent system" can process user requests based on generative models (e.g., language models, multimodal models) to perform specified types of tasks.

[0017] In some scenarios, intelligent systems can be represented as virtual avatars or physical entities for interaction with users. The term "virtual object" can refer to a digital entity capable of interacting with a user. Intelligent systems can therefore also be called virtual objects, digital assistants, intelligent assistants, AI assistants, chatbots, virtual agents, etc. Virtual objects can possess intelligent dialogue and information processing capabilities, responding to user queries and providing appropriate answers. In some examples, virtual objects can be presented in a graphical form within the interactive interface, such as as avatars, animated characters, or other visual representations.

[0018] Figure 1 A schematic diagram of example environment 100 is shown. (e.g.) Figure 1 As shown, example environment 100 may include terminal device 110. Application 120 is installed on terminal device 110. User 140 can interact with application 120 via terminal device 110 and / or an attached device of terminal device 110.

[0019] In some cases, application 120 can be any appropriate application that can provide query services. Figure 1 In environment 100, if application 120 is active, terminal device 110 can display page 150 of application 120. Page 150 may include various types of pages that application 120 can provide, such as information interaction pages, query pages, search pages, search result display pages, etc.

[0020] In some cases, terminal device 110 communicates with server 130 to provide services to application 120. Terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, virtual reality (VR) devices, augmented reality (AR) devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some cases, terminal device 110 may also support any type of user-facing interface (such as "wearable" circuitry).

[0021] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide background services for applications 120 that support session interaction in terminal device 110.

[0022] A communication connection can be established between server 130 and terminal device 110. This communication connection can be established via wired or wireless means. The communication connection can include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections. In some cases, server 130 and terminal device 110 can exchange signaling information through their communication connection.

[0023] It should be understood that the structure and function of the various elements in environment 100A are described for illustrative purposes only and do not imply any limitation on the scope of the scheme.

[0024] In some cases, application 120 can provide interaction capabilities with the intelligent system. Application 120 can be dedicated to providing services from the intelligent system, or it can be an application integrated with the intelligent system (meaning it can provide functions or services other than those of the intelligent system). Although Figure 1 The image shows a single application, but in reality, multiple applications can be installed on the terminal device 110.

[0025] In this paper, the intelligent system can be deployed locally on terminal device 110 or remotely. In the case of remote deployment, terminal device 110 can directly call the intelligent system, or it can call the intelligent system via server 130.

[0026] In some scenarios, during interaction with user 140, the intelligent system can respond to user 140's requests and handle tasks instructed by the user. In some scenarios, the intelligent system may have intelligent dialogue and task processing capabilities. Terminal device 110 can provide an interface 150 for interacting with the intelligent system. In interface 150, user 140 can initiate task requests to the intelligent system by inputting natural language (e.g., text input or voice input). Alternatively or additionally, user 140 can upload local files or specify online files to instruct the intelligent system to assist in completing various tasks.

[0027] As briefly described above, terminal devices or applications can provide users with intelligent system-like functions to assist them in using the devices or applications. Typically, this allows users to interact with intelligent systems within a session with one or more intelligent systems. However, this approach only allows interaction with the intelligent system within a session to respond to user needs.

[0028] In view of this, an improved scheme for conversational interaction is proposed. In this scheme, first information is presented on the conversational interface. The conversational interface is used for a conversation between a first user and at least one second user, and the first information includes a first message. Correspondingly, second information is presented, which includes a first instruction issued to the intelligent system, and the first instruction is at least related to the first message.

[0029] In this way, during multiple user interactions, the intelligent system can proactively participate in the conversations between users and provide secondary information to respond to certain messages in the conversation. Based on this, by providing intelligent system command recommendations related to conversation messages during user conversations, interaction efficiency can be improved. Furthermore, through the proactive participation of the intelligent system, user interaction can be assisted, thereby improving interaction efficiency.

[0030] The following description of the examples will continue with reference to the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and various page designs are possible in practice. The various graphic elements on the page may have different arrangements and different visual representations, one or more elements may be omitted or replaced, and one or more other elements may also be present. This document is not limited in this respect. Furthermore, the examples will be described primarily with respect to terminal device 110 in the following text. It should be understood that the actions described with respect to terminal device 110 can also be performed by application 120 on terminal device 110, or by application in conjunction with its server (e.g., server 130).

[0031] For ease of understanding, the following text will refer to Figures 2A to 2E , Figures 3A to 3F ,as well as Figures 4A to 4D To describe an example used for session interaction. Figures 2A to 2E Schematic diagrams of example interfaces 200A to 200E for conversational interactions in scenarios where presentation instructions are automatically triggered, based on certain situations. Figures 3A to 3F Schematic diagrams of example interfaces 300A to 300F for conversational interactions in scenarios where presentation instructions are automatically triggered, based on other situations. Figures 4A to 4DThe diagrams show example interfaces 400A to 400D for conversational interaction in manually triggered presentation instruction recommendation scenarios, based on certain situations. Interfaces 200A to 200E, 300A to 300F, and 400A to 400D are all example interfaces for conversational interfaces.

[0032] In some scenarios, terminal device 110 presents first information on a conversational interface. The conversational interface is used for a conversation between a first user and at least one second user. In some examples, the conversational interface can be used for a one-on-one chat between user 140 (i.e., the first user) and a single second user. The conversational interface can also be used for a group chat between user 140 and multiple second users. For ease of understanding, the following description will use a one-on-one chat between user 140 and a second user as an example. This is merely exemplary, and it should be understood that the group chat between user 140 and multiple second users is also applicable to the conversational interaction scheme proposed herein.

[0033] The first information may include at least one message from a first user and at least one second user. The first information may also include other content, such as, but not limited to, user avatars, the time corresponding to the message, the name of the session (e.g., the name of a group chat), etc. For ease of understanding, the following description uses the first message in the first information as an example to illustrate the method of session interaction. See also Figure 2A Terminal device 110 can present messages such as message 211 and message 214 (which is an example of the first message) in the session interface 200A. Message 211 may be sent by user 140, and message 214 may be sent by a second user.

[0034] In some scenarios, terminal device 110 presents second information, which includes a first instruction issued to the intelligent system, the first instruction being at least related to the first message. In some examples, the second information may also be referred to as recommendation information, capable of recommending potential needs to the user (e.g., questions to ask) based on the content of the conversation. In some scenarios, the content of the first instruction at least matches the content of the first message. For example, if the first message is "Go to City A," the first instruction could be "What's the weather like in City A recently?"

[0035] See Figure 2A Terminal device 110 may present second information 213. Second information 213 may include instruction 215 (instruction 215 being an example of the first instruction), which is related to message 214. Instruction 215 is control information used to instruct the intelligent system to perform a specific task. Alternatively / additionally, second information 213 may include reference information 216 for the intelligent system (e.g., an assistant).

[0036] In some cases, refer to Figure 2ATerminal device 110 can present a recommended reply 212, which includes content for replying to message 214. For example, see [reference needed]. Figure 2A and Figure 2C If the terminal device 110 determines that the user 140 has chosen the recommended reply 212 sent via the intelligent system, it can present message 231 in the conversation interface 200C as the user 140's reply to message 214.

[0037] In some scenarios, terminal device 110 may generate second information, recommended responses, and instructions within the second information based on first information in the session (e.g., message 211, message 214, or any appropriate message in the session). As an example, terminal device 110 may understand and analyze these messages in the session to generate the second information, recommended responses, and instructions within the second information. For instance, terminal device 110 may generate the second information, recommended responses, and instructions within the second information based on the context of the first message. Alternatively / additionally, terminal device 110 may generate the second information, recommended responses, and instructions within the second information based on content items (e.g., documents) relevant to the current session. In some examples, terminal device 110 may utilize machine learning models to generate the second information, recommended responses, and instructions within the second information. Machine learning models may include, for example, any appropriate model such as a language model or a visual language model.

[0038] In some cases, terminal device 110 may present the second message 213 and the recommended reply 212 in a first visual style in the session interface 200A. Terminal device 110 may also present the first message in a second visual style. The first visual style differs from the second visual style. The first visual style may, for example, indicate that the borders of the second message 213 and the recommended reply 212 are presented in a dotted line style. It should be understood that the first visual style includes, but is not limited to, highlighting / enlarging the style of presenting the second message 213 and the recommended reply 212. For example, the borders of the second message 213 and the recommended reply 212 may be highlighted, or a more prominent border style may be used, etc., which are not limited herein.

[0039] In some cases, refer to Figure 2B Terminal device 110 can display the second information 213 and the recommended reply 212 in window 221. Window 221 can be overlaid on the conversation interface 200B. For example, in a scenario where user 140 interacts with multiple second users, displaying the second information 213 and the recommended reply 212 in window 221 can improve the readability of the information for user 140.

[0040] The preceding text described how the second information is presented. In some cases, terminal device 110 can proactively present the second information based on messages in the session. This will continue to be discussed below. Figures 2A to 2E and Figures 3A to 3F Describe the process of conversational interaction in a scenario where second information is automatically presented.

[0041] In some scenarios, if terminal device 110 receives a first message and determines that the first message indicates a matter to be responded to, it may present second information. In this case, the first instruction in the second information instructs the intelligent system to process the matter to be responded to. The matter to be responded to may include, for example, a question to be answered or an operation to be performed. A question to be answered may include, for example, a query about a user's daily needs (e.g., office collaboration, travel, shopping, or any other appropriate need). An operation to be performed may include, for example, content generation (e.g., Artificial Intelligence Generated Content, AIGC content generation), editing, sharing, etc. In some examples, terminal device 110 may determine whether the first message indicates a matter to be responded to based on keyword matching. For example, if terminal device 110 determines that the first message includes preset keywords (e.g., "excuse me," "is it," "create," "modify," etc., or any other appropriate keywords), it may determine that the first message indicates a matter to be responded to.

[0042] In some cases, the matter to be responded to may include questions to be answered. For example, see Figure 2A If terminal device 110 determines that message 214 indicates a question to be answered by user 140 (e.g., "Are you going to [location]?", "What's the weather like in [location]?", "What's the weather like in [location] for the next few days?"), it can trigger the presentation of second information 213. The instruction 215 indicated by the second information 213 indicates that the intelligent system needs to answer the question. Instruction 215 can be used to instruct the intelligent system to perform certain tasks (e.g., query the weather in [location]) to obtain auxiliary information, which can be used to help user 140 decide how to answer the question to be answered.

[0043] In some cases, terminal device 110 can receive a second input. The second input can indicate feedback on second information; for example, the second input can provide feedback from a first user on the second information. For instance, the second input can indicate feedback from user 140 on the second information. Further, terminal device 110 can present a second message in the session interface. The second message includes instructions for the intelligent system, and the instructions included in the second message match the second input. As an example, the second input can instruct user 140 to confirm the second information; for example, user 140 can select the second information. As an example, the second input can also instruct user 140 to edit the second information; for example, user 140 can edit the instructions included in the second information, or change the mention of the intelligent system corresponding to the second information to a mention of another intelligent system. For ease of understanding, the session interaction process in the case where the second input indicates a confirmation operation on the second information will be described first with reference to the accompanying drawings. The session interaction process in the case where the second input indicates editing of the second information will be described in detail below.

[0044] In some cases, terminal device 110 may receive a confirmation operation instructing user 140 to select second information. Terminal device 110 may present the second message in the session interface. See also Figure 2A and Figures 2D to 2E If terminal device 110 receives a selection of second information 213 from user 140, it can present message 241 (an example of the second message) in session interface 200D. Message 241 can be a message sent by user 140, and message 241 includes instruction 215. Further, terminal device 110 can present a response 251 (also referred to as "auxiliary information") to message 241 sent via the intelligent system in session interface 200E. For example, user 140 can decide on a response to message 214 based on response 251. In some examples, response 251 may include the processing result of instruction 215 by the intelligent system.

[0045] In some scenarios, after receiving a selection of the second information 213 from the user 140, the terminal device 110 may invoke the intelligent system to generate a response 251. The intelligent system may generate the response 251 by executing the task indicated by instruction 215 (e.g., generating a weather forecast for a specific location) based on messages in the session. See also [examples omitted for brevity]. Figure 2D During the process of terminal device 110 calling the intelligent system to generate a response, a prompt message 242 can be displayed in the conversation interface. This allows the user to know that the intelligent system is processing a matter awaiting a response.

[0046] In some cases, terminal device 110 can invoke at least one functional component included in the intelligent system to execute the task indicated by the first instruction, wherein the type of the task matches the at least one functional component. Each functional component can be understood as a tool that the intelligent system can invoke when processing a user request, and each functional component is capable of performing a corresponding function or service. Continuing with the example above, after determining that the type of task indicated by instruction 215 is a certain type of task (e.g., a query task), terminal device 110 can invoke a functional component (e.g., a query component) that matches that type of task to execute the task, thereby generating a response 251.

[0047] In some cases, the event to be responded to may include an action to be performed. An action to be performed may, for example, include a content generation operation. See also [example example]. Figure 3A If terminal device 110 determines that message 312 (which is an example of the first message) indicates the generation of content (e.g., “make a painting”), it can trigger the presentation of second information 313. The instruction 315 indicated by the second information 313 indicates that the intelligent system needs to generate content that meets the needs of user 140.

[0048] In some cases, see Figures 3A to 3B If terminal device 110 receives a selection of second information 313 from user 140, it can present message 321 (an example of a second message) in the session interface 300B. Message 321 can be a message sent by user 140, and message 321 includes instruction 315. Further, see... Figure 3C Terminal device 110 can present a response 331 to message 321 sent by the intelligent system in the conversation interface 300C, such as generating a picture of coconut trees by the sea.

[0049] In some scenarios, after receiving a selection of the second information 313 from the user 140, the terminal device 110 may invoke the intelligent system to generate a response 331. The intelligent system may execute the task indicated by instruction 315 (e.g., generate a picture) based on messages in the session, thereby generating the response 331. After determining that the task indicated by instruction 315 is of a certain type (e.g., a content generation task), the terminal device 110 may invoke a functional component (e.g., a content generation component) matching that type of task to execute the task, thereby generating the response 331.

[0050] In some cases, the operation to be performed may include editing operations, such as modifying or deleting. See, for example... Figure 3DIf terminal device 110 determines that message 340 (which is an example of the first message) indicates an editing operation on some content (e.g., "modify the background of content 341"), it can trigger the presentation of second information 342 and second information 343. The instruction 345 indicated by second information 342 indicates that the intelligent system needs to modify the background of content 341.

[0051] In some cases, see Figures 3D to 3E If terminal device 110 receives a selection of second information 342 from user 140, it can present message 351 (an example of a second message) in the session interface 300E. Message 351 can be a message sent by user 140, and message 351 includes instruction 345. Further, see... Figure 3F Terminal device 110 can display a response 361 to message 340 sent via the intelligent system in the conversation interface 300F. For example, it can display content 362 with a modified background in the conversation interface 300F.

[0052] In some scenarios, after receiving a selection of the second information 342 from the user 140, the terminal device 110 can invoke the intelligent system to generate a response 361. The intelligent system can execute the task indicated by the instruction 345 (e.g., modifying the background of content 341) based on messages in the session, thereby generating the response 361. After determining that the task indicated by the instruction 345 is of a certain type (e.g., an editing task), the terminal device 110 can invoke a functional component (e.g., an editing component) matching that type of task to execute the task, thereby generating the response 361.

[0053] Therefore, through the proactive participation of intelligent systems, user interaction behavior can be analyzed and decision-making can be assisted, thereby improving the interaction efficiency between the first user and the second user.

[0054] The preceding text describes how terminal device 110 can proactively present second information based on messages within a session. The following text will continue to refer to... Figures 4A to 4D This describes the process of conversational interaction in scenarios where second information is manually presented.

[0055] In some scenarios, during interaction between user 140 and at least one second user, user 140 may manually mention the intelligent system (e.g., @intelligent system or any other appropriate mention) to instruct the intelligent system to perform a task. In some scenarios, terminal device 110 may receive first input via an input component, the first input mentioning the intelligent system. If terminal device 110 determines that the first input mentions the intelligent system, it may present second information via the input component. See also Figure 4AIf the terminal device 110 receives the mention information 410 of the intelligent system through the input component 411, it can present the second information through the input component 411.

[0056] In some cases, refer to Figure 4A Terminal device 110 can present instruction 215, along with instructions 413 and 414 included in other second information, in area 415 of input component 411 (which is an example of a first area). These instructions are presented in a triggerable manner, meaning that user 140 can trigger them. (Return to Reference) Figure 2D and Figure 2E If the terminal device 110 receives a trigger for the instruction 215 via the session interface 400A, it can display the message 215 in the session interface 200D, followed by the response 251 issued by the intelligent system.

[0057] In some cases, refer to Figure 4B Terminal device 110 can present instructions 215 for the second information in natural language within area 425 of input component 411 (which is an example of a second area). Natural language can include, but is not limited to, text and speech. (Continue to the reference) Figures 2D to 2E If the terminal device 110 receives the instruction 215 via the session interface 400B, it can display the message 215 in the session interface 200D, followed by the response 251 issued by the intelligent system.

[0058] In some cases, user 140 may select at least a portion of message 214 and invoke the intelligent system to instruct the intelligent system to perform a task associated with message 214. See also Figure 4C and Figure 4A Terminal device 110 can receive a first trigger operation (e.g., a selection operation) on at least a portion of a first message (e.g., message 214). Accordingly, after receiving the second trigger operation, terminal device 110 can present an entry control 431 instructing the intelligent system to be accessed. Further, if terminal device 110 receives a second trigger operation on entry control 431, it can present an input component 411. User 140 can send a first instruction to the intelligent system via input component 411. In some cases, see [reference needed]. Figure 4D The terminal device 110 can receive the first input via the input box 441 and display the message 241 of the instruction 215 in the interface 400D. Thus, by providing intelligent system instruction recommendations related to the conversation message during the user session, interaction efficiency can be improved.

[0059] The preceding text described the conversational interaction process in a scenario where the second information is presented manually. The following text will describe in detail the conversational interaction process in a scenario where the second input indicates an editing operation on the second information.

[0060] In some scenarios, terminal device 110 can receive a second input for editing second information, which can be used to update the first instruction to a second instruction. If terminal device 110 receives the second input (e.g., an editing operation), it can present a second message in the session interface. The second message is issued by user 140 and includes the second instruction. In some examples, user 140 can modify the first instruction presented by terminal device 110. After user 140 modifies the first instruction, terminal device 110 can present the modified instruction in the session interface. Accordingly, the intelligent system can perform a task based on the modified instruction. In some examples, user 140 can modify the first instruction in a second area of ​​the input component. User 140 can input content to modify the first instruction via the input component.

[0061] As an example, terminal device 110 can receive a second input for modifying a first task into a second task. For example, suppose a first instruction issued to the intelligent system instructs that the background of the content be changed to white (which is an example of the first task). In this scenario, user 140 can modify "change the background of the content to white" to "change the background of the content to blue (which is an example of the second task)".

[0062] As an example, terminal device 110 can receive a second input for modifying the currently mentioned smart system to refer to another smart system. For example, suppose the current second information includes a first instruction issued to smart system A. In this scenario, user 140 can modify "the first instruction issued to smart system A" to "the first instruction issued to smart system B". For example, user 140 can modify the second information "@A Assistant, check the weather" to "@B Assistant, check the weather".

[0063] In summary, this approach allows the intelligent system to proactively participate in user conversations during multiple interactions, providing secondary information to respond to certain messages within the conversation. Based on this, providing intelligent system command recommendations related to conversation messages during user sessions can improve interaction efficiency. Furthermore, the proactive participation of the intelligent system can assist user interactions, thereby enhancing interaction efficiency.

[0064] Figure 5 A flowchart of a process 500 based on session interactions in some situations is shown. Process 500 can be implemented at terminal device 110. See below for reference. Figure 1 Describe the process 500.

[0065] In box 510, terminal device 110 presents first information in a session interface, which is used for a session between a first user and at least one second user. The first information includes a first message.

[0066] In frame 520, terminal device 110 presents second information, which includes a first instruction issued to the intelligent system, the first instruction being at least related to the first message.

[0067] In some cases, the first message indicates the matter to be responded to, and the first instruction instructs the intelligent system to process the matter to be responded to.

[0068] In some cases, the matters to be responded to include questions to be answered, and the first instruction directs the intelligent system to answer the questions.

[0069] In some cases, the matters to be responded to include operations to be performed, and the first instruction directs the intelligent system to perform the operations.

[0070] In some cases, the session involves content items, and the operations include editing or generating content items.

[0071] In some cases, presenting the second information includes: receiving a first input via an input component, the first input mentioning the intelligent system; and presenting the second information via the input component in response to the first input mentioning the intelligent system.

[0072] In some cases, presenting second information through an input component includes at least one of the following: presenting the first instruction in natural language in a first area of ​​the input component, or presenting the first instruction in a triggerable instruction style in a second area of ​​the input component.

[0073] In some cases, process 500 further includes: receiving a first trigger operation, the first trigger operation indicating selection of at least a portion of a first message; presenting an entry control, the entry control indicating querying the intelligent system; receiving a second trigger operation, the second trigger operation indicating triggering the entry control; and presenting an input component.

[0074] In some implementations, process 500 further includes: receiving a second input indicating feedback on the second information; and presenting a second message on a session interface, the second message including instructions to the intelligent system, the instructions included in the second message matching the second input.

[0075] In some cases, the second input indicates the selection of a second message, and the second message includes the first instruction.

[0076] In some cases, the second input instructs editing of the second information to update the first instruction to the second instruction, and the second message includes the second instruction.

[0077] In some cases, a first instruction directs a first task, a second input is used to modify the first task into a second task, and a second instruction directs the second task.

[0078] In some cases, the second input is used to modify the reference to an intelligent system to a reference to another intelligent system, and the second instruction is issued to the other intelligent system.

[0079] In some cases, presenting the second information includes at least one of the following: presenting the second information in a first visual style on a conversation interface, the first visual style being different from the second visual style; presenting the first message in a second visual style; or presenting the second information in a window overlaid on the conversation interface.

[0080] In some cases, the first instruction directs at least one functional component of the intelligent system to perform a task, the type of which matches the at least one functional component.

[0081] In some cases, process 500 may also include presenting a recommended response, which includes content used to reply to the first message.

[0082] In some cases, corresponding apparatus for implementing the above methods or processes is also provided. Figure 6 A schematic block diagram of an example device 600 based on session interactions in some scenarios is shown. Device 600 can be implemented as or included in terminal device 110. The various modules / components in device 600 can be implemented by hardware, software, firmware, or any combination thereof.

[0083] As shown in the figure, device 600 includes a first information presentation module 610, configured to present first information on a conversation interface used for a conversation between a first user and at least one second user, the first information including a first message. Device 600 also includes a second information presentation module 620, configured to present second information, the second information including a first instruction issued to the intelligent system, the first instruction being at least related to the first message.

[0084] In some cases, the first message indicates the matter to be responded to, and the first instruction instructs the intelligent system to process the matter to be responded to.

[0085] In some cases, the matters to be responded to include questions to be answered, and the first instruction directs the intelligent system to answer the questions.

[0086] In some cases, the matters to be responded to include operations to be performed, and the first instruction directs the intelligent system to perform the operations.

[0087] In some cases, the session involves content items, and the operations include editing or generating content items.

[0088] In some cases, the second information presentation module 620 is also configured to receive a first input via an input component, the first input mentioning the intelligent system; and to present second information via the input component in response to the first input mentioning the intelligent system.

[0089] In some cases, presenting second information through an input component includes at least one of the following: presenting the first instruction in natural language in a first area of ​​the input component, or presenting the first instruction in a triggerable instruction style in a second area of ​​the input component.

[0090] In some cases, device 600 further includes an input component presentation module configured to receive a first trigger operation, the first trigger operation indicating the selection of at least a portion of a first message; present an entry control, the entry control indicating an inquiry into the intelligent system; receive a second trigger operation, the second trigger operation indicating the triggering of the entry control; and present an input component.

[0091] In some cases, the message presentation module 610 is also configured to receive a second input indicating feedback on a second message; and to present a second message on a conversational interface, the second message including instructions to the intelligent system, the instructions included in the second message matching the second input.

[0092] In some cases, the second input indicates the selection of a second message, and the second message includes the first instruction.

[0093] In some cases, the second input instructs editing of the second information to update the first instruction to the second instruction, and the second message includes the second instruction.

[0094] In some cases, a first instruction directs a first task, a second input is used to modify the first task into a second task, and a second instruction directs the second task.

[0095] In some cases, the second input is used to modify the reference to an intelligent system to a reference to another intelligent system, and the second instruction is issued to the other intelligent system.

[0096] In some cases, the second information presentation module 620 is also configured to present the second information in a first visual style on the conversation interface, the first visual style being different from the second visual style, the first message being presented in the second visual style, or to present the second information in a window overlaid on the conversation interface.

[0097] In some cases, the first instruction directs at least one functional component of the intelligent system to perform a task, the type of which matches the at least one functional component.

[0098] In some cases, device 600 also includes a recommended response presentation module, configured to present recommended responses, which include content for replying to the first message.

[0099] The modules included in device 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some cases, one or more modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 600 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0100] Figure 7 A block diagram of an electronic device 700 in which one or more examples may be implemented is shown. It should be understood that... Figure 7 The electronic device 700 shown is merely exemplary and should not be construed as limiting the functionality and scope of the examples described herein. Figure 7 The illustrated electronic device 700 can be used to implement the electronic device 110 discussed above.

[0101] like Figure 7 As shown, electronic device 700 is in the form of a general-purpose electronic device. Components of electronic device 700 may include, but are not limited to, one or more processing units or processors 710, memory 720, storage devices 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processor 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 700.

[0102] Electronic device 700 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 700.

[0103] Electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 7 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions of various examples.

[0104] The communication unit 740 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 700 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 700 can operate in a networked environment using logical connections to one or more other servers, networked personal computers, or another network node.

[0105] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 700 can also communicate with one or more external devices (not shown) via communication unit 740 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 700, or with any device that enables electronic device 700 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0106] A computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. A computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0107] The flowcharts and / or block diagrams of the methods, apparatus, devices, and computer program products referred to herein describe various aspects. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0108] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0109] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0110] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products under various scenarios. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0111] Various examples have been described above. The foregoing descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for conversational interaction, comprising: The first information is presented in the conversation interface, which is used for a conversation between a first user and at least one second user, and the first information includes a first message. as well as The system presents a second message, which includes a first instruction issued to the intelligent system, the first instruction being at least related to the first message.

2. The method of claim 1, wherein the first message indicates a matter to be responded to, and the first instruction instructs the intelligent system to process the matter to be responded to.

3. The method of claim 2, wherein the matter to be responded to includes a question to be answered, and the first instruction instructs the intelligent system to answer the question.

4. The method of claim 2, wherein the event to be responded to includes an operation to be performed, and the first instruction instructs the intelligent system to perform the operation.

5. The method of claim 4, wherein the session relates to content items, and the operation includes editing the content items or generating the content items.

6. The method of claim 1, wherein presenting the second information comprises: The system receives a first input via an input component, the first input referring to the intelligent system. as well as In response to the first input mentioning the intelligent system, the second information is presented through the input component.

7. The method of claim 6, wherein presenting the second information via the input component comprises at least one of the following: In the first area of ​​the input component, the first instruction is presented in natural language, or In the second area of ​​the input component, the first instruction is presented in a triggerable instruction style.

8. The method according to claim 6, further comprising: Receive a first trigger operation, wherein the first trigger operation indicates the selection of at least a portion of the first message; An entry control is presented, which instructs the intelligent system to be queried; Receive a second trigger operation, the second trigger operation indicating that the entry control is triggered; as well as Present the input component.

9. The method according to claim 1, further comprising: Receive a second input, which indicates feedback on the second information; as well as A second message is presented in the session interface. The second message includes instructions to the intelligent system, and the instructions included in the second message match the second input.

10. The method of claim 9, wherein the second input indicates selection of the second information, and the second message includes the first instruction.

11. The method of claim 9, wherein the second input instructs editing the second information to update the first instruction to a second instruction, and the second message includes the second instruction.

12. The method of claim 11, wherein the first instruction indicates a first task, the second input is used to modify the first task into a second task, and the second instruction indicates the second task.

13. The method of claim 11, wherein the second input is used to modify the reference to the intelligent system to a reference to another intelligent system, and the second instruction is issued to the other intelligent system.

14. The method of claim 1, wherein presenting the second information comprises at least one of the following: The second information is presented in the session interface using a first visual style, which is different from the second visual style. The first message is presented using the second visual style, or The second information is presented in a window that is overlaid on the session interface.

15. The method of claim 1, wherein the first instruction instructs at least one functional component of the intelligent system to perform a task, the type of the task being matched with the at least one functional component.

16. The method according to claim 1, further comprising: A recommended response is presented, which includes content used to reply to the first message.

17. A device for conversational interaction, comprising: A first information presentation module is configured to present first information in a conversation interface, the conversation interface being used for a conversation between a first user and at least one second user, the first information including a first message; as well as The second information presentation module is configured to present second information, which includes a first instruction issued to the intelligent system, the first instruction being at least related to the first message.

18. An electronic device comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 16 when executed by the at least one processor.

19. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 16.

20. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 16.

Images