Method and device for human-computer interaction

Abstract

A method for human-computer interaction, comprising: in response to receiving a first input content, outputting a resulting response corresponding to the first input content in a session window of a first application, wherein a presentation method or operability of the resulting response varies with the first input content.

Description

RELATED REGISTRATION(S)

[0001] This application claims priority over Chinese patent application No. 202410466132.7, which was filed with the National Intellectual Property Administration, PRC, on April 17, 2024, and is fully incorporated herein by reference. AREA OF TECHNOLOGY

[0002] Embodiments of the present disclosure relate to the field of artificial intelligence and relate to, but are not limited to, a method and a device for human-computer interaction. BACKGROUND

[0003] The human-computer interactions in certain existing technical solutions are simple question-and-answer interactions that may not meet users' needs for different questions and answers. SUMMARY

[0004] Embodiments of the present disclosure provide a method, a device, equipment and a storage medium for human-computer interaction.

[0005] The technical solution of the embodiments of the present disclosure is implemented as follows:

[0006] In one aspect, the present disclosure provides a human-computer interaction method. The method comprises, in response to receiving an initial input, the output of a resulting response corresponding to that initial input in a session window of a first application performing an interactive task, wherein a presentation mode or operability of the resulting response varies with the initial input.

[0007] Another aspect of the present disclosure is a device for human-computer interaction. The device comprises a first execution module for outputting, in response to receiving a first input, a resulting response corresponding to the first input in a session window of a first application performing an interactive task, wherein a presentation method or operability of the resulting response varies with the first input.

[0008] In another aspect, the present disclosure provides an electronic device. The device comprises a memory that stores computer program instructions and a processor that is coupled to the memory and configured to execute the computer program instructions in order to: in response to receiving an initial input, output a resulting response corresponding to the initial input, in a session window of an initial application that performs an interactive task, wherein a presentation mode or operability of the resulting response varies with the initial input.

[0009] In another aspect, the present disclosure provides a storage medium that stores executable instructions for implementing the above method when executed by a processor.

[0010] In response to receiving the initial input, a resulting response corresponding to that input is displayed in the session window of the first application performing the current interactive task. The presentation mode and / or usability of the resulting response, corresponding to different initial inputs, will vary. Accordingly, it is possible to present a variety of presentation modes to suit the different questions posed by users, and the presentation can further differentiate usability to meet the various response needs of users. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1A is a schematic diagram of an implementation process of a human-computer interaction method according to certain embodiments of the present disclosure; Fig. Figure 1B is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. Figure 1C is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. 1D is a schematic representation of two display layouts according to certain embodiments of the present disclosure; Fig. 2A is a schematic diagram of an implementation procedure for outputting a resulting response according to certain embodiments of the present disclosure; Fig. 2B is a schematic representation of two session windows according to certain embodiments of the present disclosure; Fig. 2C is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. 2D is a schematic representation of two session windows according to certain embodiments of the present disclosure; Fig. 2E is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. 2F is a schematic representation of three session windows according to certain embodiments of the present disclosure; Fig. 2G is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. 2H is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. 2I is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. 2J is a schematic representation of a session window according to certain embodiments of the present disclosure; Fig. 3A is a schematic diagram of an implementation process of a method for deploying a third-party APP (application) and an intent command set in the cloud according to certain embodiments of the present disclosure; Fig. Figure 3B is a schematic diagram of an implementation process of a method for a third-party app to access an intelligent agent according to certain embodiments of the present disclosure; Fig. Figure 4 is a schematic diagram of an implementation process for outputting a resulting response according to certain embodiments of the present disclosure; Fig. Figure 5 is a schematic diagram of a composite structure of a human-computer interaction device according to certain embodiments of the present disclosure; and Fig. Figure 6 is a schematic representation of a hardware unit of an electronic device according to certain embodiments of the present disclosure. DETAILED DESCRIPTION

[0011] To clarify the purpose, technical solution, and advantages of the embodiments of this disclosure, certain technical solutions of the embodiments of this disclosure are described in detail below in conjunction with the accompanying drawings. The following embodiments are used to illustrate this disclosure but do not limit its scope.

[0012] Reference is made to "some embodiments" or "certain embodiments" that describe a subset of all possible embodiments. "Some embodiments" or "certain embodiments" can be the same subset or different subsets of possible embodiments and can be combined without conflict.

[0013] The terms "first" and "second" are used to distinguish similar objects and do not necessarily represent a specific order of the objects. The terms "first," "second," and "third" may be interchanged with a specific order or sequence where permissible, so that the embodiments of the present disclosure described herein may be implemented in a different order than that shown or described here.

[0014] Unless otherwise stated, the technical and scientific terms used herein have the same meaning as those generally understood in the technical field. The terms used here serve to describe the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0015] The present disclosure provides, in certain embodiments, a human-computer interaction method as described in Fig. 1A shows the procedure, which includes the following steps.

[0016] Step S110, in response to receiving an initial input, outputs a resulting response corresponding to the initial input in a session window of an initial application performing an actual interactive task; wherein the presentation method and / or operability of the resulting response corresponding to different initial inputs differ.

[0017] In certain embodiments, the first input content can be a string entered into the conversation input field, can be speech content entered via speech, can contain an input operation, or can be a recognized gesture operation.

[0018] The first application can be an AI agent application capable of human-computer interaction, or an interactive application that can interact with users, such as a social networking application, a conferencing application, an email application, or similar applications that can display a session window. In certain implementations, the AI ​​agent is an intelligent entity capable of perceiving the environment, making decisions, and performing actions. Unlike traditional artificial intelligence, the AI ​​agent has the ability to achieve a specific goal step by step by thinking independently and invoking tools. In certain implementations, the first application can also be an application such as a chat assistant, a device assistant, or something similar.

[0019] The session window of the first application is a window that can interact with the AI ​​agent. In certain implementations, the session window can be the AI ​​agent's main window or its extended window.

[0020] Fig. Figure 1B is a schematic representation of a session window provided according to certain embodiments of the present disclosure. As in Fig. As shown in Figure 1B, the schematic diagram includes a main window 11 and an extended window 12.

[0021] In Fig. In example 1B, the AI ​​agent is given the name "Xiaotian". The first input displayed in the main window 11 is: "Help me write a PPT about the development of the AIPC industry". Xiaotian responds: "OK, I have found relevant services for you, please select one" and offers a "Use" button in the main window 11. The user clicks the "Use" button to display the expanded window 12 and create a PPT that answers the user's requests in the expanded window.

[0022] Fig. Figure 1C is a schematic diagram of a session window provided in certain embodiments of the present disclosure. As in Fig. As shown in Figure 1C, the schematic diagram includes a main window 11 and an extended window 12.

[0023] In the example of Fig. 1C is the first input displayed in the main window 11: "What else can I do with my new computer?" Xiaotian replies: "Xiaotian recommends a new machine wizard with several functions to help you get started with the new computer," and provides a card layout in the main window 11 with the options "Check Configuration," "Install Software," "New Machine Tips," and "Show More." The user clicks on "Check Configuration" on the card to view various configuration information of the electronic device in the expanded window 12.

[0024] In certain implementations, the presentation mode can be a display layout. Different usability can consist of whether the resulting responses of different types or scenarios have operable controls, or whether further operations can be performed.

[0025] In certain embodiments Fig. Figure 1D shows a schematic diagram of two display layouts provided in certain embodiments of the present disclosure. As shown in Fig. Figure 1D shows a schematic diagram including subfigure (a) and subfigure (b). In subfigure (a), the user sends "Help me change the display resolution" to Xiaotian, and Xiaotian replies "OK, I helped you set the optimal resolution for the screen" and displays "Display Resolution". The user can select the appropriate resolution from the drop-down menu shown in subfigure (b).

[0026] In certain embodiments of the present disclosure, upon receipt of the first input content, a resulting response corresponding to that first input content is displayed in the session window of the first application performing the current interactive task. The presentation mode and / or usability of the resulting response corresponding to different first input contents may vary. In certain embodiments, it is possible to present a variety of presentation modes according to the different questions posed by the users, and the presentation may further achieve different usability to satisfy the users' diverse response needs.

[0027] In certain embodiments, such as in Fig. As shown in Figure 2A, in step S110 the “output of a resulting response corresponding to the first input content in a session window of the first application performing the current interactive task” can be implemented by at least one of the following steps:

[0028] Step S210: Identifying the user intent represented by the initial input content, obtaining a feedback result provided by a target application or target knowledge base for the user intent, processing the feedback result into a resulting response in a target form, and outputting the resulting response in the session window.

[0029] In certain embodiments, user intent may be a simple semantic understanding or the recognition of user input by the intent-understanding model within the AI ​​agent.

[0030] The target application can be an application that is registered in the first application (e.g., Xiaotian agent) via a registration interface such as an application programming interface (API) and can interact with Xiaotian via commands and data.

[0031] In certain implementations, the target application can be an application bound to the Xiaotian agent via an API, such as a Vantage application (or other device manager applications) that implements device control, an album application, a file manager, an application store that enables software (application, app) downloads, a device switching assistant, a wallpaper application, or similar. Vantage is an application that manages computer performance and enables device security. In an implementation process, the selection and determination of the target application may depend on the nature, semantic content, or intent of the initial input.

[0032] The target knowledge base comprises a local knowledge base and a cloud knowledge base. The local knowledge base might include a local file library, a local atlas, a local video library, or similar; the cloud knowledge base might include a cloud hard drive, a peripheral device's file system, or similar. In an implementation process, the local or cloud knowledge base might also include a peripheral knowledge base located on a peripheral device, determined based on various usage scenarios, such as its physical location relative to the electronic device on which the initial application runs. If the device is a peripheral, the peripheral knowledge base can be selected, and if it is a cloud server, the cloud knowledge base can be selected.

[0033] The feedback result can be data of any form, such as at least one of the following: an image, a link, a control icon, a parameter value, a configuration parameter, a configuration option, a processing option, and similar items.

[0034] The resulting response from a target form can be any or a combination of maps, atlases, operating windows, links, or the link itself, which are converted into appropriate formats depending on the type of feedback result.

[0035] In certain embodiments Fig. 2B A schematic diagram of two session windows provided in certain embodiments of the present disclosure. As in Fig. As shown in Figure 2B, the schematic diagram includes subfigure (c) and subfigure (d). As shown in subfigure (c), the system recognizes that the user's initial input is: "What is commonly used software for new computers?". After intent detection, Xiaotian recognizes that the user wants to recommend some commonly used software for installation on new computers and then outputs the response "OK, this commonly used software for new computers has been recommended to you." It then displays the recommendation card and the installation button for the commonly used software, and the user can install the recommended software by clicking the installation button. In certain embodiments, the target application can be an application store.In an implementation process, the installation software link from the application store can be retrieved, and based on this link, an installation button is provided to the user in the session window. As shown in subfigure (d), the user's initial input is identified as "Help me recommend several AI applications that improve efficiency." Based on the user's intent to improve efficiency, the intent of the corresponding application can be retrieved from the application store. The relevant information and download link of the recommended AI application, returned by the application store, can then be obtained. The AI ​​application recommended by the application store to improve efficiency is displayed in the session window, and a button for installing the recommended AI application is shown.In certain implementations, the target application can be an application store. During implementation, a link to the application store's installation software can be obtained, and an installation button can be provided to the user in the session window based on this link.

[0036] Step S220: Receiving a feedback result that corresponds to the initial input content from a target application or target knowledge base via a target interface, processing the feedback result into a resulting response in a target form, and outputting the resulting response to the session window.

[0037] In certain embodiments, the target application is an application that establishes a target association relationship with the first application, and the target knowledge base includes a local knowledge base and / or a cloud knowledge base of the first application.

[0038] In certain embodiments, the target interface can be the interface used by the target application, such as a photo album, device management application, or similar, for registration with the first application (smart Xiaotian agent), such as the corresponding API interface.

[0039] In an implementation process, the feedback result corresponding to the initial input is retrieved from the target application or knowledge base via the target interface. In certain embodiments, the initial input can be retrieved without identifying the user's intent. The system analyzes which target application or knowledge base the initial input corresponds to and submits the corresponding call or search instruction to the relevant target application or knowledge base. In certain embodiments, Fig. 2C is a schematic representation of the session window provided in certain embodiments of the present disclosure. As in Fig. As shown in Figure 2C, if the system detects that the user is typing "Download Tencent Video," it can directly pass the "Download Tencent Video" command to the app store. The app store recognizes and provides the Tencent Video download and installation link by matching it with "Tencent Video" and returns the installation link to Xiaotian. Xiaotian processes the feedback result into a card-like response and displays it to the user. As shown in the figure, Xiaotian responds: "Okay, Xiaotian has determined through the app store that Tencent Video is not yet installed on your device. You can download Tencent Video first." The Tencent Video installation button appears, and the user can install Tencent Video by clicking it.

[0040] Fig. 2D is a schematic diagram of two session windows provided in certain embodiments of the present disclosure. As in Fig. Shown in 2D, the schematic diagram includes subfigure (e) and subfigure (f), where, as shown in subfigure (e), the first input content is "Help me find AIPC-related files," which can be matched with "AIPC" to allow the user to query various types of corresponding files in the electronic device. Therefore, the various found and associated files are displayed in the form of a file list, and a link to each corresponding file is shown so that the user can see the corresponding file in the session window and view it by clicking the link.As shown in subfigure (f), the first input content is “Help me find AIPC-related files”, which can be matched with “AIPC” to allow the user to query various types of related files and images in the electronic device and to display the related files and images in the form of a file library.

[0041] The target format can be in various map, atlas, link, or other layout forms, and each map can be considered a loaded H5 component. In certain implementations, H5 is a standard computer language for creating World Wide Web pages, which is a simplified vocabulary of HTML5. H5 components refer to components written in the HTML5 language.

[0042] In certain embodiments of the present disclosure, the feedback result can be obtained by identifying the user intent represented by the initial input content; the feedback result corresponding to the initial input content can also be obtained from the target application or target knowledge base via the target interface. In certain embodiments, the feedback result can be obtained by identifying the user intent or by matching it to the initial input content, and the feedback result can be processed into a resulting response and displayed in the session window, thereby improving the accuracy and variety of the response to the initial input content, expanding the content of the response, and satisfying the various response requests of users.The feedback results can be processed in the target form and displayed in the session window, allowing users to perform operations within a session window without having to jump to other pages, thus enabling users to see the feedback results directly in the form of a dialog, which improves usability.

[0043] In certain embodiments, in step S110, the “output of a resulting response corresponding to the first input content in the session window of the first application performing the current interactive task” can also be implemented by the following steps:

[0044] Step S230: Determine a target application or target knowledge base that corresponds to the initial input content, based on the initial input content, and load a target response component of the target application or target knowledge base into the initial application to process the feedback result into a resulting response in a target form and display the resulting response in the session window via the target response component.

[0045] During implementation, one or more keyword matching strategies, user habits, and device configurations can be considered to determine a corresponding target application or knowledge base that responds to the initial input content.

[0046] In certain embodiments, the target response component can be a component used by every application that is to be displayed in the session window, such as a map, a pop-up window, and the like.

[0047] In step S230, “determining a target application or target knowledge base that corresponds to the first input content, based on the first input content” can be implemented by at least one of the following steps:

[0048] Step 231: Identify a target keyword in the initial input content and determine a target application or target knowledge base that corresponds to the target keyword, based on the target keyword;

[0049] In certain embodiments, the target keyword may be a keyword that carries an application name or knowledge base, a keyword that represents a functional requirement, or something similar.

[0050] In certain embodiments, such as in Fig. As shown in 2C, the target keyword “Tencent Video” can be identified based on the first input content “Download Tencent Video”, and based on this “Tencent Video”, the appropriate target application is determined as an application store application that provides a link to install Tencent Video, such as Lenovo App Store and App Store.

[0051] Step 232: Obtain user portrait data of a target user entering the initial input content and determine a target application or target knowledge base that corresponds to the user portrait data, based on the user portrait data.

[0052] In certain embodiments, the user portrait data includes data on user habits or data on the historical behavior of users, or similar information.

[0053] In certain embodiments, the first content entered by the user consists of searching for an image, and then, based on the user's habits or historical behavior, it is determined that the user frequently views images from an album application, and the album application can be determined as the target application;

[0054] If, in certain implementations, it is determined based on user habits or historical behavior that the user frequently views images from a cloud gallery, the cloud gallery can be identified as the target knowledge base that corresponds to the user.

[0055] Step 233: Determine a target application or target knowledge base that corresponds to the user intent represented by the initial input content.

[0056] In certain embodiments, as in subfigure (d) of Fig. As shown in Figure 2B, the first input is "Recommend several AI applications that improve efficiency." Based on the recognition that the user intent is to install AI applications that improve efficiency, the application of the application store that provides the AI ​​applications can be determined as the target application that corresponds to the user intent.

[0057] Step 234: Determining the target application or target knowledge base based on the initial input content and the configuration information of the electronic device on which the initial application is running.

[0058] In certain implementations, the user types "Please help me edit this video" into Xiaotian's session window, and Xiaotian accesses three applications that can edit or create the video. However, considering the electronic device's configuration information, the first application might place excessive demands on the device's resources. Therefore, the second or third application among the three can be selected as the target application.

[0059] In certain embodiments of the present disclosure, by identifying the target keywords of the first input content, or by determining the user's intent based on user portrait data, or by the first input content, or based on the first input content or configuration information of the electronic device, a suitable target application or target knowledge base can be determined and made available to the user, which can improve the variety and accuracy of the recommendations.

[0060] In certain embodiments, in steps S210 and S220, the "processing of the feedback result into a resulting response in a target form and output of the resulting response in the session window" can be implemented by at least one of the following steps:

[0061] Step 211: Using the target response component of the target application or the target knowledge base to process the feedback result into a resulting response in the form of a card and outputting the resulting response in the session window.

[0062] In certain embodiments, the target response component is a front-end component for displaying the feedback result of the target application or target knowledge base in the first application, such as an H5 component or another HTML component.

[0063] The map format can be a map-shaped display of several different file types in the session window, as in subfigure (e) of Fig. Shown in 2D. The different file types can originate from the target knowledge base, and the corresponding file links in the target knowledge base are output as map components. As shown in Fig. As shown in 2C, the Tencent video can originate from a target application such as an application store, and the link to install the Tencent video can be displayed as a card component in the session window.

[0064] Fig. Figure 2G is a schematic representation of the session window provided in certain embodiments of the present disclosure. As in Fig. As shown in 2G, the feedback results queried from the configuration information can be processed into a map form based on the first input content "Do you see what my computer configuration looks like?" and displayed in the session window as a map component.

[0065] During the process, the feedback result is processed into a resulting answer in the form of a card and displayed in the session window, including the main session window and / or the expanded window. As in Fig. As shown in 1B, the response component of the resulting answer “PAPPT” can be displayed in the form of a card in the main window 11, and the response components “Topic description” and “Supplementary materials” can be displayed in the form of cards in the extended window 12.

[0066] Step 212: Processing the feedback result into a resulting answer in the form of a control and output of the resulting answer in the session window.

[0067] In certain embodiments, the control method may correspond to the device control class or to the configuration settings, to the download and installation, or similar.

[0068] Fig. 2F is a schematic diagram of three session windows provided in certain embodiments of the present disclosure. As in Fig. Figure 2F shows a schematic diagram including subfigure (g), subfigure (h), and subfigure (i). As shown in subfigure (g), a control form for a power switch can be provided in the session window, and the user selects the power switch to enable or disable power saving mode. As shown in subfigure (h), a control form for a power switch and a slider can be provided in the session window, and the user can select the switch to adjust the brightness and further adjust the brightness value by moving the slider. As shown in subfigure (i), a control form for a power switch can be provided in the session window, and the user can click the confirmation switch to confirm the shutdown operation.

[0069] Step 213: Processing the feedback result into a resulting answer in the form of a snapshot and outputting the resulting answer in the session window.

[0070] In certain embodiments, the snapshot format can be a representation of the corresponding atlas or document result.

[0071] Fig. Figure 2H is a schematic representation of the session window provided in certain embodiments of the present disclosure. As in Fig. As shown in 2H, the feedback results based on the first input content "Help me find the landscape photos taken in the last year" can form an album and be displayed as thumbnails.

[0072] As in subfigure (e) of Fig. Shown in 2D, the feedback result based on the first input content "Help me find AIPC-related files" can be displayed as a document result in the session window.

[0073] Step 214: Processing the feedback result into a resulting answer in the form of a hyperlink and outputting the resulting answer in the session window.

[0074] In certain embodiments, the form of the hyperlink can correspond to the presentation of results such as a website, an access address, or similar.

[0075] As shown in subfigure (d) of Fig. As shown in Figure 2B, the feedback result based on the first input content "Recommend several AI applications that improve efficiency" can be processed into a hyperlink form of a download address and displayed in the session window.

[0076] Step 215: Processing the feedback result into a resulting answer in the form of a file library and outputting the resulting answer in the session window.

[0077] In certain embodiments, the form of the file library can correspond to the representation of the results of combining images and texts and different types of file collections.

[0078] As in subfigure (f) of Fig. Shown in 2D, the feedback results based on the first input content "Help me find files related to AIPC" can be processed into images and file collections of various types and output in the session window.

[0079] In certain embodiments of the present disclosure, the feedback results can be displayed in the session window in the form of cards, controls, snapshots, hyperlinks and / or file libraries according to the different feedback result types, thereby enriching the variety of display of the resulting responses and providing a response format that better matches the feedback results.

[0080] In certain embodiments, in steps S210 and S220, the "processing of the feedback result into a resulting response in a target form and output of the resulting response in the session window" can be implemented by at least one of the following steps:

[0081] Step 216: Based on the attribute information of the feedback result, processing the feedback result into a resulting answer in a target form corresponding to the attribute information and outputting the resulting answer to the main window and / or the extended window of the session window.

[0082] In certain embodiments, the attribute information of the feedback result can be the type of the feedback result. In certain embodiments, the type includes at least documents, images, device configuration information, applications to be called, or device control and configuration options to be presented. During implementation, the display form (target form) of the resulting feedback can be determined based on the types mentioned here. In certain embodiments, a decision is made as to whether the feedback result waits for further feedback from the user to determine whether the resulting response is transformed into a form with operable controls.In certain embodiments, if the feedback result is a display image or device configuration information, the resulting response is a map or small window without operable controls; if the feedback result is a device control option or configuration option, the resulting response is processed into a map or small window with operable controls.

[0083] The main window can be the agent's (Xiaotian's) initial session window, and the extended window can be a window that expands from the initial session window upwards, downwards, leftwards, and rightwards. The extended window can open automatically or based on user actions.

[0084] Step 217: Based on the user portrait information of the target user, processing the feedback result into a resulting answer in the target form and outputting the resulting answer in the main window and / or extended window of the session window.

[0085] In certain embodiments, the user portrait information may contain or reflect the user's habits or historical operating data.

[0086] During an implementation process, a form frequently used by the user can be generated based on user profile information. In certain implementations, if the user is accustomed to using a slider, a tab or window containing a slider can be generated.

[0087] Step 218: Processing the feedback result into a resulting answer in a target form based on the data set of the feedback result and outputting the resulting answer in the main window and / or in the extended window of the session window;

[0088] In certain embodiments, the resulting response is displayed in the main window and / or the expanded window based on the amount or size of the data. In certain embodiments, if the amount of data is small, the resulting response may be displayed in full in the main window and / or the expanded window; if the amount of data is large, the resulting response may be displayed in a folded or thumbnail format, and the "show more" control icon or logo may be used to view the full response, flip the page, or open the expanded window for viewing.

[0089] Step 219: Based on the operating status of the electronic device and / or the first application, the feedback result is processed into a resulting response in a target format and the resulting response is displayed in the main window and / or the extended window of the session window. The electronic device is the device on which the first application is running.

[0090] In certain embodiments, the resulting response can be displayed in the main window and / or the expanded window of the session window, depending on whether multiple tasks are running concurrently in the agent and whether other applications are running concurrently on the electronic device. In certain embodiments, if both the electronic device and the agent are in an idle state, the resulting response can be displayed in a large format and in an expanded window; however, if the electronic device and / or the agent are in a busy state, a small format can be selected and displayed in the main window.

[0091] In certain embodiments of the present disclosure, it can be determined, based on at least one of the following information: attribute information of the feedback result, user portrait information of the target user, the data volume of the feedback result, the operating status of the electronic device and / or the initial application, to process the feedback result into a resulting response in the target format and to output the resulting response in the main window and / or extended window of the session window. The target format of the resulting response in the main window and / or extended window can be determined more effectively so that the display of the resulting response meets different display requirements and provides users with a variety of display main windows and / or extended windows.

[0092] As in subfigure (d) of Fig. As shown in Figure 2B, the user can ask the question "Recommend some AI applications that improve efficiency" in Xiaotian's session window; Xiaotian's intent-understanding model recognizes the instructions from the Lenovo App Store and loads the components that match the instructions into Xiaotian's dialog box; Xiaotian's framework provides an interface to support two-way communication between the components and the Lenovo App Store app and handles the installation, progress, and opening of the corresponding software.

[0093] The present disclosure provides, in certain embodiments, a method for deploying a third-party application and a command set in the cloud. As described in Fig. As shown in Figure 3A, a third-party app can be deployed in the cloud by implementing the steps of implementing intent instructions, background registration of the app, and background review and approval; the intent understanding model can be obtained by training the app intent corpus and deployed in the cloud.

[0094] During an implementation process, third-party apps can develop Xiaotian's command components and plug-ins based on access specifications from Xiaotian's third-party apps, register and publish the CoreApp application in the background, and train the intent understanding model based on the corpus provided by the app.

[0095] The present disclosure provides, in certain embodiments, a method for a third-party app to access an intelligent agent. As described in Fig. As shown in Figure 3B, when the intelligent agent Xiaotian is launched, information about the third-party app and the set of intent instructions for the third-party app can be retrieved from the cloud and loaded into the intelligent agent Xiaotian. When a user sends a message to Xiaotian, the intent understanding model can be used to identify the app intent instruction, and the app component can be loaded based on the recognition result. The app component can connect to the app plug-in via Xiaotian's app communication framework to provide the user with a software installation function, which corresponds to the one described in Figure 3B. Fig. The installation button shown on 2C corresponds to this.

[0096] During an implementation process, after Xiaotian starts, the information and instruction set information of the third-party app are loaded. After the user sends a message in the Xiaotian session window, if the intent-understanding model recognizes the instruction of the third-party app, the app's component is loaded, and a set of framework interfaces are provided for the component to perform two-way communication between the component and the app.

[0097] A natural language dialogue based on the intelligent agent Xiaotian can be implemented, relying on the model's ability to understand intentions, so that third-party apps can be quickly connected to the intelligent agent Xiaotian.

[0098] In certain embodiments, such as in Fig. As shown in Figure 4, in step S110 “Output of a resulting response corresponding to the first input content in a session window of the first application performing the current interactive task” can be implemented by at least one of the following steps:

[0099] Step S410: If the first input content is used to configure a target component of the electronic device, processing of the feedback result provided by the second application into a map or interface window with controls and display of the map or interface window in the main window and / or extended window of the session window.

[0100] In certain embodiments, the target components include hardware and software, such as configuration elements of the electronic device, in particular the resolution and brightness of the screen, the volume of the speaker and microphone, the read and write speed of the hard drive, the size of the application window, startup items, the authorization configuration, or similar.

[0101] The second application can be an application that is capable of managing the configuration of the target component, such as Lenovo's Vantage Software, Lenovo Manager, Tencent Manager, 360 Manager, or similar.

[0102] The form of the control, such as a toggle switch, a slider, a drop-down menu, or similar, is not restricted. The interface window includes an input field or an on-screen display (OSD) pop-up window.

[0103] In certain embodiments, subfigure (b) shows in Fig. 1D, that the display resolution is set via a drop-down field in the session window. Subfigure (g) in Fig. Figure 2F shows that a toggle switch in the session window is used to determine whether to enable power-saving mode. Subfigure (h) in Fig. Figure 2F shows that the screen brightness can be adjusted using a slider in the session window. Subfigure (i) in Fig. 2F shows that a confirmation button can be clicked in the session window to confirm the shutdown process.

[0104] Step S420: When the initial input content is used to search for a target file, the feedback result provided by a third-party application and / or an initial knowledge base is processed into a thumbnail and / or file list and displayed and output in the main window and / or the extended window of the session window.

[0105] In certain implementations, the third application can be an application such as a file manager, a superfile, a photo album, or similar. The first knowledge base can consist of images or files stored in the cloud or on a cloud hard drive.

[0106] During implementation, the feedback result can be processed into a collection of thumbnail image sets and / or file lists.

[0107] In certain embodiments, this can be achieved in Fig. The session window shown in Figure 2H displays a collection of thumbnail images, where the displayed images may originate from applications such as a local photo album, a cloud photo album, or similar. The one shown in subfigure (e) of Fig. The 2D session window can display a collection of file lists, with the displayed files potentially originating from applications such as a file manager, a superdocument, or similar. The one shown in subfigure (f) in Fig. The 2D session window can display a collection of thumbnail sets and file lists, where the displayed images can come from applications such as local photo albums or cloud photo albums, and the displayed files can come from applications such as file managers or superdocuments.

[0108] Step S430: If the first input content is used to search for a target application, the feedback result provided by a fourth application is processed into an application list with controls and displayed in the main window and / or the extended window of the session window.

[0109] In certain embodiments, the fourth application can be similar to an application store. The control can be a control that provides a download link, and after the user clicks the control, the user can be directly connected to the application store or the official download address via an associated plug-in that provides a download and installation service.

[0110] In certain embodiments, the in Fig. In the session window shown in 2C, a download link control (installation control) is provided. After the user clicks on the installation control, the user can be directly connected to the fourth application (application store or official download address) via a corresponding plug-in that offers download and installation services.

[0111] Step S440: If the first input content is used to display configuration information of the electronic device, process the feedback result provided by the second application and / or the second knowledge base into a map or configuration table and display the map or configuration table in the main window and / or the extended window of the session window;

[0112] In certain embodiments, the configuration information of the electronic device includes the configuration information of the local device or the configuration information of other devices such as peripherals or similar. The map or configuration table is not limited to displaying the configuration information of the electronic device but can also be used to display results in other scenarios.

[0113] In certain embodiments, the text reads: Fig. 2G, the first input, is "Can you see how my computer is configured?". The configuration information of the electronic device, provided by the second knowledge base (computer configuration information) or the second application (such as Computer Manager), can be processed in a configuration table and displayed in the session window. To view further configuration information, the "Show More" button can be clicked.

[0114] Step S450: If the first input content is used for data migration and / or restarting the electronic device, processing the feedback result provided by the fifth application into a video animation and displaying and outputting the video animation in the main window and / or extended window of the session window.

[0115] In certain implementations, data migration can be performed using application software, such as a device switching wizard. The fifth application can be a device switching wizard or a data migration application.

[0116] In certain implementations, if a user wishes to switch devices, a device switching assistance application or a data migration application can be processed into a video animation and displayed in the main window and / or extended window of the session window. Fig. 2I is a schematic representation of a session window provided in certain embodiments of the present disclosure. As in Fig. As shown in Figure 2I, the first input of the session window in the schematic diagram is “Help me synchronize the data from the old computer”, and the fifth application can be a device switching assistant that processes the feedback supplied by the device switching assistant into a card that shows the use of a button, and outputs the card in the main window 21, and displays and outputs the processed video animation in the extended window 22.

[0117] In certain embodiments of the present disclosure, the output form of the feedback result can be determined based on different first input contents, so that the output form of the feedback result better matches the corresponding first input content and the display form is more varied and appropriate.

[0118] In certain embodiments, the human-computer interaction method also includes one or more of the following steps:

[0119] Step S120, in response to receiving the second input content, updates the corresponding resulting response in the session window and / or controls the electronic device to perform a target operation in response to the second input content.

[0120] In certain embodiments, the second input content may or may not be related to the resulting response. In certain embodiments, the second input content may be an input operation for a control in the resulting response, or new content entered into the input field via a microphone.

[0121] In certain embodiments, as in subfigure (c) of Fig. As shown in 2B, the second input can be a prompt that the agent gives the user based on the first input: “You can continue to ask me: What is the commonly used software for a new computer?”, and then the corresponding answer can be updated in the session window. As shown in Fig. As shown in 2C, the second input content can also be a further request from the agent to the user: “You can continue to ask me: WeChat, what software can convert PDF to World?”, and then the corresponding resulting answer of the second input content can be updated in the session window, so that a continuous dialogue effect can be created in an implementation procedure.

[0122] Fig. 2J is a schematic representation of the session window provided in certain embodiments of the present disclosure. As in Fig. As shown in 2J, if the second input content in the schematic diagram is "No need for other files, just find images", then the resulting response corresponding to the second input content can be updated in the session window to "OK, use Superfile to find 6 files for you", and the corresponding 6 thumbnails will be displayed in the session window.

[0123] Updating the resulting response in the session window and / or controlling the electronic device to perform the target operation in response to the second input content can be based on the association between the second input content and the first input content to update the data content or control state of the resulting response or to regenerate a new resulting response; or, based on the second input content or the input operation, the electronic device is controlled to perform a target effect operation, such as controlling the electronic device to shut down, restart, change brightness and volume, disconnect from the network, enter safe mode, or similar actions.

[0124] In certain embodiments, as in subfigure (g) of Fig. As shown in Figure 2F, the electronic device can be controlled to activate the energy-saving mode based on the user-selected target operation of turning on the energy-saving mode. As shown in subfigure (h) of Fig. As shown in Figure 2F, the screen brightness of the electronic device can be adjusted based on the selected target operation of adjusting the screen brightness. As shown in subfigure (i) of Fig. As shown in Figure 2F, the electronic device can be controlled to shut down for confirmation based on the target process of executing the shutdown.

[0125] In certain embodiments, such as in the schematic representation of the session window in Fig. As shown in Figure 2E, the most frequently used screen-off time dates can first be determined, and these frequently used dates can then be sorted based on usage time to obtain a target time value with the longest usage time. In a non-restrictive scenario where the target time value is set to 3 minutes, the user can be offered a 3-minute screen-off time when the screen is not charging and a 3-minute screen-off time when it is charging.

[0126] In certain embodiments, the screen-off time to which the user is accustomed can also be determined based on the user portrait data, and then the default screen-off time of the device can be changed based on the screen-off time to which the user is accustomed.

[0127] In certain embodiments of the present disclosure, upon receipt of the second input content, the resulting response in the session window is updated and / or the electronic device is controlled to perform a target operation in response to the second input content. The resulting response can be updated and / or the target operation can be performed based on the second input content, further enriching the functional diversity of the agent.

[0128] In certain embodiments, step S120 can be performed by at least one of the following steps:

[0129] Step 121: if the second input content contains an input operation on a control in the resulting response that corresponds to the first input content, update the display state of the control in the session window and control the electronic device to perform a target operation that corresponds to the input operation.

[0130] In certain embodiments, the input operation to the control element may be an operation of clicking a download control, an operation of clicking a switch, a slider or a drop-down field, or an operation of clicking "more" and the like.

[0131] The display status includes "On or activate", "Off or deactivate", downloaded, expanded, or similar; the corresponding target operations may be changing the brightness, volume, activating, changing the background image, or similar.

[0132] In certain embodiments, the second input content of the opening process for the control of whether the "eye protection mode" should be turned on in the map control displayed in the main window of Xiaotian can be "My eyes are a little tired", whereby the state of the switch control of the "eye protection mode" in the main window is updated to the state "On or Activate", and the electronic device is controlled to perform the operation of turning on the eye protection mode.

[0133] Step 122: if the second input content has a first association relationship with the first input content, update the display state and / or the display content of the resulting response in the session window.

[0134] In certain embodiments, the first association relationship can represent that the second input content is related to the first input content, and the display state can be that the map size increases or decreases, the main window and the expanded window are switched, the transparency is adjusted, the position changes, or similar; the results of further expansion or further summarization can be taken into account with respect to the change in the display content.

[0135] In certain embodiments, such as in Fig. As shown in 2H, the second input option is "Show more," which relates to the first input option, "Help me find the landscape photos taken last year." If the main window no longer displays images, the expanded window can be displayed to view more pictures.

[0136] In certain embodiments, such as in Fig. As shown in Figure 2H, the second input can be an input operation of a "Show More" control in a response card that appears on the first input, "Help me find the landscape photos I took last year." The updated display status of the "Show More" control can be shown in the session window, and the electronic device can be controlled to perform a target operation to display additional images corresponding to the control, such as displaying more landscape photos and changing the position of the "Show More" control or removing it from view.

[0137] Step 123: If the second input content has a second association relationship with the first input content, generate a resulting response again in the session window or control the electronic device to perform a corresponding target operation.

[0138] In certain embodiments, the first association relationship can represent that the second input content is independent of the first input content, and the second input content can be responded to in a similar way to the first input content.

[0139] In certain embodiments, such as in Fig. 2C shows that if the user determines that the second input content, "What software can convert PDF to World," is unrelated to the first input content, a PDF-to-World application status installation link can be provided for the second input content in a similar way to how the Tencent video installation software is provided.

[0140] In certain embodiments of the present disclosure, it is possible to update the display state of the control and to perform a suitable target operation based on the control's input operation in the resulting response; if the second input content is associated with the first input content, the display state and / or the display content of the resulting response in the session window is updated; if the second input content is not associated with the first input content, the resulting response in the session window is regenerated or the target operation is performed by generating and displaying a new resulting response.

[0141] In certain embodiments, the human-computer interaction process includes one or more of the following steps:

[0142] Step S130: In response to establishing a connection with a target terminal, display and output the target terminal in the session window to control the target terminal to perform an appropriate operation based on a third-party input content acting in the session window, or in response to a target execution from the terminal, instruct a response operation.

[0143] In certain embodiments, the target terminal device can be another device connected to the electronic device. In certain embodiments, after the devices are connected, the other connected terminal devices can be displayed in the session window of the electronic device's agent, so that the other terminal devices can be controlled by the electronic device's agent or controlled by the other terminal devices via the electronic device's agent.

[0144] In certain embodiments, the target device can be a Bluetooth speaker, a projector, or an extended display. After connecting to the electronic device, the connected Bluetooth speaker, projector, or extended display can be displayed in the session window of the electronic device's smart body, allowing the Bluetooth speaker, projector, or extended display to be controlled via the electronic device's smart body.

[0145] If the target device is a server or other electronic device on the network side, after connecting to the electronic device, the connected server or other electronic device can be displayed in the session window of the intelligent body of the electronic device, and the server or other electronic device can be controlled by the intelligent body of the electronic device.

[0146] In certain embodiments of the present disclosure, it is possible to display the target terminal in a session window when the electronic device establishes a connection with the target terminal, and to control the target terminal so that it performs an operation based on a third input content acting in the session window, or performs a response operation in reaction to a target from the target terminal.

[0147] The present disclosure provides, in certain embodiments, a human-computer interaction device comprising the included modules, wherein each module comprises each submodule, each submodule comprising a unit that can be implemented by a processor in an electronic device; the module or unit can also be implemented by a specific logic circuit; in an implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or the like.

[0148] Fig. Figure 5 is a schematic diagram of the assembly structure of a human-computer interaction device provided in certain embodiments of the present disclosure. As in Fig. As shown in section 5, the device includes 500:

[0149] A first execution module 510 to output a resulting response corresponding to the first input content in a session window of a first application that performs an actual interactive task in response to receiving the first input content; wherein the presentation method and / or operability of the resulting response corresponding to different first input contents are different.

[0150] In certain embodiments, the first execution module 510 includes at least one of the following submodules: a first execution submodule and a second execution submodule, wherein the first execution submodule serves to identify the user intent represented by the first input content, to obtain the feedback result provided by the target application or target knowledge base for the user intent, and to process the feedback result into a resulting response in the target form and output the resulting response in the session window; the second execution submodule is used to obtain a feedback result corresponding to the first input content from a target application or a target knowledge base via a target interface, and to process the feedback result into a resulting response in a target form and output the resulting response in the session window;where the target application is an application that establishes a target association relationship with the first application, and the target knowledge base includes a local knowledge base and / or a cloud knowledge base of the first application.

[0151] In certain embodiments, the first execution module 510 further includes a third execution submodule which is used to determine a target application or target knowledge base corresponding to the first input content, based on the first input content, and to load a target response component of the target application or target knowledge base into the first application in order to process the feedback result to a resulting response in a target form and to display the resulting response in the session window via the target response component;The third execution submodule contains at least one of the following units: a first target unit, a second target unit, a third target unit, and a fourth target unit, wherein the first target unit is used to identify a target keyword in the first input content and to determine a target application or target knowledge base corresponding to the target keyword based on the target keyword; the second target unit is used to obtain user portrait data of a target user entering the first input content and to determine a corresponding target application or target knowledge base based on the user portrait data; the third target unit is used to determine a corresponding target application or target knowledge base based on the user intent represented by the first input content;and the fourth determination unit is used to determine the target application or target knowledge base based on the first input content and configuration information of the electronic device on which the first application is running.

[0152] In certain embodiments, the "processing of the feedback result into a resulting response in a target form and output of the resulting response in the session window" in the first execution submodule and the second execution submodule includes at least one of the following units: a first output unit, a second output unit, a third output unit, a fourth output unit, and a fifth output unit, wherein the first output unit is used to process the feedback result into a resulting response in a card form and output the resulting response in the session window using the target response component of the target application or the target knowledge base; the second output unit is used to process the feedback result into a resulting response in a control form and output the resulting response in the session window;The third output unit is used to process the feedback result into a resulting response in a snapshot format and to display the resulting response in the session window; the fourth output unit is used to process the feedback result into a resulting response in a hyperlink format and to display the resulting response in the session window; the fifth output unit is used to process the feedback result into a resulting response in a file library format and to display the resulting response in the session window.

[0153] In certain embodiments, the "processing of the feedback result into a resulting response in a target form and output of the resulting response in the session window" in the first execution submodule and the second execution submodule includes at least one of the following units: a sixth output unit, a seventh output unit, an eighth output unit, and a ninth output unit, wherein the sixth output unit is used to process the feedback result into a resulting response in a target form corresponding to the attribute information based on the attribute information of the feedback result and to output the resulting response in the main window and / or the extended window of the session window;The seventh output unit is used to process the feedback result into a resulting response in a target form based on the target user's user portrait information and to output the resulting response in the main window and / or the expanded window of the session window; the eighth output unit is used to process the feedback result into a resulting response in a target form based on the data volume of the feedback result and to output the resulting response in the main window and / or the expanded window of the session window;The ninth output unit is used to process the feedback result into a resulting response in a target form based on the operating status of the electronic device and / or the first application, and to output the resulting response in the main window and / or the extended window of the session window, and the electronic device is a device on which the first application is running.

[0154] In certain embodiments, the first execution module 510 includes at least one of the following submodules: a fourth execution submodule, a fifth execution submodule, a sixth execution submodule, a seventh execution submodule, and an eighth execution submodule, wherein the fourth execution submodule serves to process the feedback result provided by the second application into a map or interface window with controls and to display the map or interface window in the main window and / or in the extended window of the session window when the first input content is used to configure the target component of the electronic device;The fifth execution submodule is used to process the feedback result provided by the third application and / or the first knowledge base into a preview image and / or a file list, and to display the preview image and / or the file list in the main window and / or the expanded window of the session window when the first input content is used to search for the target file; the sixth execution submodule is used to process the feedback result provided by the fourth application into an application list with controls, and to display the application list in the main window and / or expanded window of the session window when the first input content is used to find the target application;The seventh execution submodule is used to process the feedback result provided by the second application and / or the second knowledge base into a map or configuration table and to display the map or configuration table in the main window and / or expanded window of the session window when the first input content is used to view the configuration information of the electronic device; and the eighth execution submodule is used to process the feedback result provided by the fifth application into a video animation and to display the video animation in the main window and / or expanded window of the session window when the first input content is used to migrate data and / or start the electronic device.

[0155] In certain embodiments, the device / apparatus includes an update module to update the resulting response in the session window in response to the receipt of the second input content and / or to control the electronic device to perform a target operation in response to the second input content.

[0156] In certain embodiments, the update module includes at least one of the following submodules: a first update submodule, a second update submodule, and a third update submodule, wherein the first update submodule is used to update the display state of the control in the session window and to control the electronic device to perform a target operation corresponding to the input operation when the second input content contains an input operation on a control in a resulting response corresponding to the first input content; the second update submodule is used to update the display state and / or the display content of the resulting response in the session window when the second input content has a first association relationship with the first input content;The third update submodule is used to regenerate a resulting response in the session window or to control the electronic device to perform a corresponding target operation if the second input content has a second association relationship with the first input content.

[0157] In certain embodiments, the device includes a second execution module that is used to display the target terminal in the session window in response to establishing a connection with the target terminal, to control the target terminal to perform a corresponding operation based on the third input content acting in the session window, or to perform a corresponding response operation in response to a target execution from the target terminal.

[0158] The description of the devices described herein is similar to the description of the methods described herein and has similar advantageous effects. For technical details not disclosed in the device descriptions of this disclosure, reference may be made to the description of the methods in this disclosure for additional or further understanding.

[0159] If the method described herein is implemented in the form of a software function module and is sold or used as an independent product, the method may also be stored on a computer-readable storage medium. The technical solution may be represented in the form of a software product that contributes to the technology in question. The computer software product is stored on a storage medium containing various instructions to enable an electronic device (which may be a mobile phone, tablet computer, laptop computer, desktop computer, or the like) to execute all or part of the method described in certain embodiments of this disclosure. The aforementioned storage medium includes: various media capable of storing program code, such as…A USB drive, a portable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk. The embodiments of the present disclosure are not limited to a specific combination of hardware and software.

[0160] The present disclosure provides, in certain embodiments, a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the human-machine interaction method provided in the present embodiments are implemented.

[0161] The present disclosure provides, in certain embodiments, an electronic device, and Fig. Figure 6 is a schematic diagram of a hardware unit of the electronic device. As shown in Fig.As shown in Figure 6, the hardware unit of the device 600 comprises: a memory 601 and a processor 602, wherein the memory 601 stores a computer program that can be executed on the processor 602, and the processor 602 implements the steps of the human-computer interaction procedure when it executes the program.

[0162] The memory 601 is configured to store instructions and applications that can be executed by the processor 602, and it can also temporarily store data that is processed by the processor 602 and various modules in the electronic device 600 (in certain embodiments image data, audio data, voice communication data and video communication data), which can be implemented by a flash memory (FLASH) or a random access memory (RAM).

[0163] The description of the embodiments of storage media and devices described herein is similar to the description of the embodiments of methods described herein and has similar positive effects. For technical details not disclosed in the embodiments of the storage medium and the device, reference can be made to the description of the embodiments of the method to obtain additional or further information.

[0164] The terms “a single embodiment,” “an embodiment,” “certain embodiments,” or “executions” mentioned herein refer to the fact that certain features, structures, or properties relating to the embodiment are included in at least one embodiment of the present disclosure. Therefore, the expression “in a single embodiment,” “in an embodiment,” “in certain embodiments,” or “in embodiments” does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or properties may be combined in one or more embodiments in any suitable manner.In various embodiments of the present disclosure, the size of the sequence numbers of the processes mentioned herein is not necessarily a condition for the execution order, and the execution order of each process can be determined by its function and internal logic and should not represent a restriction on the implementation method of the embodiments. The sequence numbers of the embodiments of the present disclosure mentioned herein serve only for descriptive purposes and do not necessarily represent the advantages and disadvantages of the embodiments.

[0165] Terms such as "include" and "comprise," or other variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus containing a set of elements includes not only those elements but also other elements not expressly listed, or elements inherent to the process, method, article, or apparatus. In the absence of further limitations, an element defined by the expressions "contains" and "comprises" does not necessarily exclude the presence of other identical elements in the process, method, article, or apparatus that contain the element.

[0166] The described devices and methods can be implemented in other suitable ways. The device designs described here are only schematic. In certain embodiments, the division of units is merely a logical division of function. In actual implementation, other division methods may be applied; for example, several units or components may be combined or integrated into another system, or some functions may be ignored or not performed. Furthermore, the coupling, direct coupling, or communication link between the components shown or discussed may be established via various interfaces, and the indirect coupling or communication link between the devices or units may be electrical, mechanical, or otherwise.

[0167] The units described here as separate components may or may not be physically separate, and the components represented as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to the actual needs to achieve the purpose of the present embodiment.

[0168] Functional units in the embodiments of the present disclosure can be integrated into a processing unit, or each unit can be configured separately as a unit, or two or more units can be integrated into one unit; the integrated units mentioned herein can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0169] All or part of the steps for implementing the procedures described herein may be performed by hardware in conjunction with program instructions, and the program mentioned herein may be stored on a computer-readable storage medium. When the program is executed, the program performs the steps of the procedure executions contained herein; and the storage medium mentioned herein includes: a portable storage device, read-only memory (ROM), a disk or optical disk, and other media capable of storing program code.

[0170] If the integrated unit of the present disclosure is implemented in the form of a software function module and is sold or used as an independent product, the unit may also be stored on a computer-readable storage medium. The technical solution of the embodiments of the present disclosure may be represented in the form of a software product that contributes to the technology in question. The computer software product is stored on a storage medium and contains several instructions for an electronic device (which may be a mobile phone, tablet computer, laptop computer, desktop computer, or the like) to execute all or part of the methods described herein according to certain embodiments of the present disclosure. The storage medium may include: various media capable of storing program code, such as…mobile storage devices, ROMs, magnetic disks or optical hard drives.

[0171] The method disclosed herein, according to certain embodiments of the present disclosure, can be combined arbitrarily and without conflict to obtain new method embodiments.

[0172] The features disclosed in the various product versions of this disclosure can be combined arbitrarily and without conflict to obtain new product versions.

[0173] The features disclosed in several method or apparatus embodiments of the present disclosure can be combined arbitrarily without conflict to obtain new method or apparatus embodiments.

[0174] This document describes an implementation method of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Those experienced in the technical field can easily consider modifications or substitutions within the technical domain disclosed in the present disclosure that should be included in the scope of protection of the present disclosure. The scope of protection of the present disclosure should be based on the scope of protection of the accompanying claims. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] CN 202410466132.7

[0001]

Claims

[1] Human-computer interaction methods, including: in response to receiving an initial input, output a resulting response corresponding to the initial input in a session window of an initial application performing an interactive task; where the presentation mode or usability of the resulting response varies with the initial input content. [2] The method of claim 1, wherein outputting the resulting response comprises one or both: Identifying a user intent represented by the initial input content, receiving a feedback result provided by a target application or knowledge base for the user intent, processing the feedback result into a resulting response in a target form, and outputting the resulting response in the session window; or Receiving the feedback result corresponding to the initial input content from the target application or target knowledge base via a target interface, processing the feedback result into the resulting response in the target form, and outputting the resulting response in the session window. where the target application is an application that establishes a target association relationship with the first application, and the target knowledge base includes a local knowledge base and / or a cloud knowledge base of the first application. [3] The method of claim 2, further comprising: Determining the target application or target knowledge base that corresponds to the initial input content, loading a target response component of the target application or target knowledge base into the initial application so that the feedback result is processed into the resulting response in the target form, and the resulting response is displayed in the session window by the target response component, wherein determining the target application or target knowledge base includes one or more of the following: Identifying a target keyword in the initial input content, and determining the target application or target knowledge base based on the target keyword; Obtaining user portrait data from a target user entering the initial input content, and determining the target application or target knowledge base based on the user portrait data; Determining the target application or knowledge base based on a user intent represented by the initial input content; or Determining the target application or target knowledge base based on the initial input content and configuration information of an electronic device on which the initial application is running. [4] Method according to claim 2, wherein the processing of the feedback result into the resulting response includes one or more of the following: Using a target response component of the target application or target knowledge base to process the feedback result into the resulting answer in a card format and output the resulting answer in the session window; Processing the feedback result into a control format and outputting the resulting response in the session window; Processing the feedback result into the resulting answer in a snapshot form and outputting the resulting answer in the session window; Processing the feedback result into a resulting answer in a hyperlink form and displaying the resulting answer in the session window; or Processing the feedback result into the resulting answer in a file library format and outputting the resulting answer in the session window. [5] Method according to claim 2, wherein the processing of the feedback result into the resulting response comprises one or more of the following: based on attribute information of the feedback result, processing the feedback result into the resulting answer in a target form corresponding to the attribute information and outputting the resulting answer in a main window and / or an extended window of the session window; based on user portrait information of a target user, processing the feedback result into the resulting response in a target form that corresponds to the user portrait information, and outputting the resulting response in the main window and / or the extended window of the session window; based on a data volume of the feedback result, processing the feedback result into the resulting answer in a target form that corresponds to the data volume, and outputting the resulting answer in the main window and / or the extended window of the session window; or based on a working state of an electronic device and / or the first application, processing the feedback result into the resulting response in a target form that corresponds to the working state, and outputting the resulting response in the main window and / or the extended window of the session window, wherein the electronic device is a device on which the first application is running. [6] Method according to claim 1, wherein the output of the resulting response comprises one or more of the following: in response to the first input being used to configure a target component of the electronic device, processing the feedback result provided by a second application into a map or interface window with controls and displaying the map or interface window in a main window and / or an extended window of the session window; in response to the initial input being used to search for a target file, processing the feedback result provided by a third-party application and / or an initial knowledge base into a thumbnail and / or file list, and displaying the thumbnail and / or file list in the main window and / or the extended window of the session window; In response to the use of the initial input content to locate the target application, the feedback result provided by a fourth application is processed into an application list with controls and the application list is displayed in the main window and / or the extended window of the session window; in response to the first input being used to display configuration information of an electronic device, processing the feedback result provided by a second application and / or a second knowledge base into a map or configuration table and displaying the map or configuration table in the main window and / or extended window of the session window; or In response to the use of the initial input content to migrate data and / or start the electronic device, the feedback result supplied by a fifth application is processed into a video animation and the video animation is displayed in the main window and / or the extended window of the session window. [7] The method according to claim 1 further comprises: in response to receiving the second input content, updating the resulting response in the session window and / or controlling an electronic device to perform a target operation in response to the second input content. [8] Method according to claim 7, wherein updating the resulting response in the session window and / or controlling the electronic device comprises one or more of the following: In response to the fact that the second input content contains an input operation that acts on a control in the resulting response corresponding to the first input content, updating a display state of the control in the session window and controlling the electronic device to perform a target operation corresponding to the input operation; In response to the second input having a first association relationship with the first input, the display state and / or the display content of the resulting response in the session window is updated; or In response to the fact that the second input content has a second association relationship with the first input content, the resulting response is regenerated in the session window or the electronic device is controlled to perform the target operation. [9] Method according to claim 1, further comprising: in response to establishing a connection with a target terminal, displaying and outputting the target terminal in the session window to control the target terminal to perform a corresponding operation based on a third input content acting in the session window; or In response to a target from the targeting device, performing an appropriate response operation. [10] Human-computer interaction device comprising a first execution module to: output a resultant response corresponding to a first input content in a session window of a first application that performs an interactive task in response to receiving the first input content, wherein a presentation method or operability of the resultant response varies with the first input content. [11] Electronic device with a memory that stores computer program instructions; and one or more processors coupled to the memory and configured to execute the computer program instructions and to perform the following in response to receiving an initial input, output a resulting response corresponding to the initial input, in a session window of an initial application performing an interactive task; where the presentation mode or usability of the resulting response varies with the initial input content. [12] Electronic device according to claim 11, wherein the output of the resulting response comprises one or both of the following: Identifying a user intent represented by the initial input content, receiving a feedback result provided by a target application or knowledge base for the user intent, processing the feedback result into a resulting response in a target form, and outputting the resulting response to the session window; or Receiving the feedback result corresponding to the initial input content from the target application or target knowledge base via a target interface, processing the feedback result into the resulting response in the target form, and outputting the resulting response in the session window. where the target application is an application that establishes a target association relationship with the first application, and the target knowledge base includes a local knowledge base and / or a cloud knowledge base of the first application. [13] Electronic device according to claim 12, wherein the method further comprises: Determine the target application or target knowledge base that corresponds to the initial input content, load a target response component of the target application or target knowledge base into the initial application so that the feedback result is processed into the resulting response in the target form, and the resulting response is displayed in the session window by the target response component. [14] Electronic device according to claim 13, wherein determining the target application or target knowledge base comprises: Identifying a target keyword in the initial input content and determining the target application or knowledge base based on the target keyword; Obtaining user portrait data from a target user entering the initial input content, and determining the target application or target knowledge base based on the user portrait data; Determining the target application or knowledge base based on a user intent represented by the initial input content; or Determining the target application or target knowledge base based on the initial input content and configuration information of an electronic device on which the initial application is running. [15] Electronic device according to claim 13, wherein the determination of the target application or target knowledge base comprises: Identifying a target keyword in the initial input content and determining the target application or knowledge base based on the target keyword. [16] Electronic device according to claim 13, wherein the determination of the target application or target knowledge base comprises: Obtaining user portrait data from a target user entering the initial input content, and determining the target application or target knowledge base based on the user portrait data. [17] Electronic device according to claim 13, wherein the determination of the target application or target knowledge base comprises: Determining the target application or knowledge base based on a user intent represented by the initial input content. [18] Electronic device according to claim 13, wherein the determination of the target application or target knowledge base comprises: Determining the target application or target knowledge base based on the initial input content and the configuration information of an electronic device on which the initial application is running. [19] Electronic device according to claim 12, wherein the processing of the feedback result into the resulting response includes one or more of the following: Using a target response component of the target application or target knowledge base to process the feedback result into the resulting answer in a card format and output the resulting answer in the session window; Processing the feedback result into the resulting answer in a control format and outputting the resulting answer in the session window; Processing the feedback result into the resulting answer in a snapshot form and outputting the resulting answer in the session window; Processing the feedback result into a resulting answer in a hyperlink form and displaying the resulting answer in the session window; or Processing the feedback result into the resulting answer in a file library format and outputting the resulting answer in the session window. [20] Electronic device according to claim 12, wherein the processing of the feedback result into the resulting response comprises one or more of the following: based on attribute information of the feedback result, processing the feedback result into the resulting answer in a target form corresponding to the attribute information and outputting the resulting answer in a main window and / or an extended window of the session window; based on user portrait information of a target user, processing the feedback result into the resulting response in a target form that corresponds to the user portrait information, and outputting the resulting response in the main window and / or the extended window of the session window; based on a data volume of the feedback result, processing the feedback result into the resulting answer in a target form that corresponds to the data volume, and outputting the resulting answer in the main window and / or the extended window of the session window; or based on a working state of an electronic device and / or the first application, processing the feedback result into the resulting response in a target form that corresponds to the working state, and outputting the resulting response in the main window and / or the extended window of the session window, wherein the electronic device is a device on which the first application is running.

Images