Computer-implemented method and electronic device

Abstract

Embodiments of the present disclosure provide a computer-implemented method and an electronic device. The method comprises: when a desktop or an interface of a running application is displayed, in response to a wake-up operation from a user, presenting an interface of a launcher for interacting with the user, the interface of the launcher comprising an input box and a plurality of generation functions. The method further comprises: on the basis of an input from the user in the input box, generating an output corresponding to the input. On this basis, the launcher for interacting with the user can be quickly invoked and presented when any interface is displayed on a computer. Thus, user operation can be facilitated, and the processing speed can be increased for the implementation of generation functions.

Description

Computer-implemented method and electronic device TECHNICAL FIELD

[0001] Embodiments of the present disclosure mainly relate to the field of computers, and more particularly, to a computer-implemented method and an electronic device. BACKGROUND

[0002] With the development of large language model (LLM) technology, various models accessible to users have been proposed, for example, users can ask questions and obtain answers. The current large language model usually requires users to access a specific artificial intelligence (AI) dialogue software separately, which leads to cumbersome operation of the user. And the current specific AI dialogue software is independent of the operating system, and the function is relatively limited.

[0003] SUMMARY

[0004] Embodiments of the present disclosure provide a large language model-based solution combined with an operating system, which enables users to quickly wake up the interface of the launcher, facilitates human-computer interaction, and improves user experience.

[0005] In a first aspect of the present disclosure, a computer-implemented method is provided, comprising: in a case where a desktop or an interface of a running application is displayed, presenting, in response to a wake-up operation from a user, an interface of a launcher for interacting with the user, wherein the interface of the launcher comprises an input box and a plurality of generation functions; and generating, based on input from the user in the input box, an output corresponding to the input.

[0006] In a second aspect of the present disclosure, an electronic device is provided, comprising: at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, causes the electronic device to perform actions, the actions comprising: in a case where a desktop or an interface of a running application is displayed, presenting, in response to a wake-up operation from a user, an interface of a launcher for interacting with the user, wherein the interface of the launcher comprises an input box and a plurality of generation functions; and generating, based on input from the user in the input box, an output corresponding to the input.

[0007] In a third aspect of the present disclosure, an apparatus is provided, comprising: a presentation module configured to, in a case where a desktop or an interface of a running application is displayed, present, in response to a wake-up operation from a user, an interface of a launcher for interacting with the user, wherein the interface of the launcher comprises an input box and a plurality of generation functions; and a generation module configured to generate, based on input from the user in the input box, an output corresponding to the input.

[0008] In a fourth aspect of the disclosure, a computer-readable storage medium having stored thereon machine executable instructions, which when executed by a device, cause the device to perform the method described according to the first aspect of the disclosure.

[0009] In a fifth aspect of the disclosure, a computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method described according to the first aspect of the disclosure.

[0010] In a sixth aspect of the disclosure, an electronic device comprising: processing circuitry configured to perform the method described according to the first aspect of the disclosure.

[0011] The summary is provided to introduce a selection of concepts that are further described below in the of the Invention. This summary is not intended to identify key or essential features of the disclosure, nor is it intended to limit the scope of the disclosure. Other features will be more fully described in the following detailed description. The summary is not intended to be used to limit the scope of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other features, aspects and advantages of embodiments of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like reference numerals refer to like elements, wherein:

[0013] FIG. 1 shows a schematic flowchart of a computer-implemented method according to some embodiments of the disclosure;

[0014] FIG. 2 shows a schematic diagram of an interface of an awakened launcher according to some embodiments of the disclosure;

[0015] FIGS. 3A-3C show schematic diagrams of interfaces associated with AI search according to some embodiments of the disclosure;

[0016] FIG. 4 shows a schematic diagram of an interface associated with translation according to some embodiments of the disclosure;

[0017] FIGS. 5A-5C show schematic diagrams of interfaces associated with uploading text according to some embodiments of the disclosure;

[0018] FIGS. 6A-6C show schematic diagrams of interfaces associated with local query according to some embodiments of the disclosure;

[0019] FIG. 7 shows a schematic diagram of a first output according to some embodiments of the disclosure;

[0020] FIG. 8 shows a block diagram of an example apparatus, in accordance with some embodiments of the present disclosure; and

[0021] FIG. 9 shows a block diagram of an example electronic device that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0022] Embodiments of the present disclosure will be described in more detail with reference to the drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It will be appreciated that the drawings of the present disclosure are for illustrative purposes only and are not intended to limit the scope of the present disclosure.

[0023] In the description of embodiments of the present disclosure, the term "includes" and its derivatives, are not intended to be limiting of the embodiments of the present disclosure to a given implementation, but rather are intended to encompass the wide variety of implementations and configurations that can be envisioned by those skilled in the art. The term "based on" is intended to be broadly construed, and refers to the established or current understanding, whether quantitative or qualitative, of the subject matter in which the term is used. The term "one embodiment" or "the embodiment" is intended to be understood as "at least one embodiment." The terms "a" and "an" are intended to be understood as "one or more" of the elements that they introduce. The term "another" is intended to be understood as "at least one other." The term "about" is intended to be understood as "approximately," "substantially," or "essentially." The term "and / or" is intended to be understood as "both" or "at least one of the items that it connects. For example, "A and / or B" is intended to be understood as A, B, or both A and B. Other explicit and implicit definitions can be found below.

[0024] In embodiments of the present disclosure, the term "large language model" is used, which can enable understanding and generation of general language. It should be noted that the use of this term is only illustrative and is not intended to limit the embodiments of the present disclosure. In some scenarios, the term "large language model" can also be replaced by any one or a combination of the following: network model, neural network model, neural network, machine learning (ML) model, deep learning (DL) model, deep network, generative AI model, etc., and the present disclosure does not limit this.

[0025] A large language model can be obtained through a computationally intensive self-supervised and semi-supervised training process. In some examples, the large language model can be based on a Transformer architecture. In some examples, the large language model can include an encoder and a decoder, or only a decoder.

[0026] With the development and popularization of large language models, users can expect to ask questions more conveniently and obtain answers. However, current AI conversation software is relatively independent and needs to be used by users individually, which is not very convenient.

[0027] In view of this, embodiments of the present disclosure provide a solution based on large language model technology. In this solution, in the case of displaying a desktop or an interface of a running application, in response to a wake-up operation from a user, an interface of a launcher for interacting with the user is presented, wherein the interface of the launcher includes an input box and a plurality of generation functions. Then, based on the input from the user in the input box, an output corresponding to the input is generated. Based on this, the launcher for interacting with the user can be quickly invoked and presented when the computer displays any interface. Therefore, the user can be facilitated to operate, and the implementation of the generation function can improve the processing speed. In some examples, the launcher can be combined with the operating system and the search engine, without the user needing to determine the location of the query in advance, saving the user time and effort in advance, and improving the user experience.

[0028] FIG. 1 shows a schematic flowchart of a computer-implemented method 100 according to some embodiments of the present disclosure. The method shown in FIG. 1 can be performed by an electronic device, for example, the electronic device is a computer. At block 110, in the case of displaying a desktop or an interface of a running application, in response to a wake-up operation from a user, an interface of a launcher for interacting with the user is presented, wherein the interface of the launcher includes an input box and a plurality of generation functions. At block 120, based on the input from the user in the input box, an output corresponding to the input is generated.

[0029] The electronic device is installed with an operating system, and is also installed with a plurality of applications, wherein the plurality of applications can include a specific desktop application associated with the launcher. For example, the specific desktop application is implemented as a dedicated browser application, a specific plug-in in at least one search engine, or a specific web page, etc. It should be understood that embodiments of the present disclosure do not limit the name of the specific desktop application, and for the convenience of description, the name of the specific desktop application is taken as “Name-XX” in the following description.

[0030] It should be understood that the term “launcher” used herein refers to a conversational software for interacting with a user to provide the user with desired answers or content, and this name is only exemplary and should not be interpreted as a limitation on embodiments of the present disclosure. In some cases, the launcher can also call other applications to provide the user with desired answers or content, so as to meet the user's generation class needs, interpretation class needs, etc. For example, the launcher can be a generative AI conversational interaction window combined with the operating system and / or the search engine, applying LLM technology.

[0031] It should be understood that the term "wake up" is used herein to mean presenting the interface of the launcher, for example, presenting the interface of the launcher on the top layer of the interface, but the name is only exemplary, for example, the terms "arouse", "trigger" and the like are also used below to mean the same meaning.

[0032] During the use of the electronic device, the user can expect to know the answers to some questions or expect to query some information or content. During the use of the electronic device, the user can at any time present the interface of the launcher through the wake-up operation. In some embodiments, the wake-up operation can be a pre-set or pre-defined operation to arouse the launcher, as described in more detail below in conjunction with examples.

[0033] In some examples, in the case that the electronic device displays the desktop of the operating system, the user can arouse the interface of the launcher through the wake-up operation. In some examples, in the case that the electronic device displays the running interface of any application, the user can arouse the interface of the launcher through the wake-up operation. In some examples, the aroused interface of the launcher can be presented in the form of a floating window. In some examples, the presented interface of the launcher is not full-screen displayed, for example, the presented interface of the launcher can have a predetermined size, and the size of the interface of the launcher can be further adjusted based on the user's adjustment instruction. The embodiments of the present disclosure do not limit the specific size of the predetermined size, for example, it can be 1 / 9 of the entire display screen, for example, it can be determined based on the font size of the display, for example, it can be other sizes, etc. It can be understood that before the interface of the launcher is aroused, the electronic device can display the running interface of any application running on the operating system, which can be an application provided by the operating system or an application installed by the user, etc. For example, when the user is editing a word, the user wants to know the answers to some questions, and when the running interface of the word is displayed, the user arouses the interface of the launcher through the wake-up operation. For another example, when the user is programming in Java, the user wants to know the answers to some questions, and when the running interface of the Java script is displayed, the user arouses the interface of the launcher through the wake-up operation. It should be understood that what is given here is only some examples, and other scenarios are also applicable, which will not be listed herein.

[0034] For example, the user can trigger the shortcut key associated with the launcher to arouse the interface of the launcher, for example, the user can press a group of keys on the keyboard at the same time. For example, the shortcut key can be the default shortcut key set by the application, or can be the shortcut key personalized by the user (such as Fn+Shift+O or others). For example, the user can set the shortcut key on the setting page of the specific desktop application.

[0035] For example, the electronic device can have a voice collection device (e.g., a microphone), and the user can input a voice for invoking the launcher to invoke the interface of the launcher, e.g., the user can speak the voice in a preset language. For example, the voice for invoking the launcher can be a voice that is set by default for the application, or can be a voice that is personalized by the user (such as "open launcher", "pop up launcher", or the like).

[0036] For example, the electronic device can have an image collection device (e.g., a camera), and the user can make a specific gesture for invoking the launcher to invoke the interface of the launcher, e.g., the gesture can be a static gesture or a dynamic gesture. For example, the specific gesture for invoking the launcher can be a gesture that is set by default for the application, or can be a gesture that is personalized by the user (such as a gesture of both hands making an OK sign, or a gesture of a single hand making an OK sign and drawing a circle within a predetermined time period, or the like).

[0037] For example, the user can click, e.g., a right click, on the desktop of the operating system, and select a specific option to invoke the interface of the launcher. For example, the specific option can be "open launcher", "pop up launcher", or the like.

[0038] For example, a shortcut button for invoking the launcher can be presented on the desktop of the operating system or a specific location associated with the desktop (e.g., a toolbar, e.g., a taskbar, or the like), and the user can invoke the launcher by selecting or clicking or double-clicking or long-pressing the shortcut button, or the like.

[0039] For example, the running interface of an application being displayed can present a shortcut button for invoking the launcher, and the user can invoke the launcher by selecting or clicking or double-clicking or long-pressing the shortcut button, or the like. For example, the shortcut button can be set to be displayed in a custom function area of a word document.

[0040] It can be understood that the above gives some examples of the wake-up operation, but embodiments of the present disclosure are not limited thereto, and the wake-up operation can be implemented by other settings or definitions. For example, the launcher that is displayed in a floating window is minimized, and the user can make the interface of the launcher present in a predetermined size by double-clicking or long-pressing or clicking a restore button, or the like.

[0041] FIG. 2 shows a schematic diagram of an interface 200 of a launcher that is invoked, according to some embodiments of the present disclosure. For example, the interface 200 of the launcher can be presented based on a wake-up operation of the user in a case where a desktop of an operating system is displayed. Alternatively, the interface 200 of the launcher can be understood as an initial interface of the launcher, or the like. As shown in FIG. 2, the interface 200 of the launcher includes an input box 210 and a plurality of item generation functions 220.

[0042] As shown, a prompt information can be displayed in the input box 210, for example, in the form of a placeholder, such as "What do you want to search for?", and optionally, the user can input in the input box 210, for example, in the form of text. For example, a voice input button 240 can also be displayed in the interface 200. For example, the voice input button 240 can be displayed in association with the input box 210. For example, the user can click the voice input button 240, and then can input in the form of voice. For example, the user can enter a specific key in the input box 210 to enable the voice input, for example, the specific key can be a space bar, etc.

[0043] For example, the user can enter the voice input mode by selecting (e.g., long-pressing) the voice input button 240, and for example, a voice input window corresponding to the voice input mode can be displayed. For example, the voice input window can include a close button (e.g., X) for closing the voice input window. For example, when the user inputs voice, a dividing line can be displayed, and the dividing line can be a dynamic animation of the voice of the conversation. For example, when the user inputs voice, the voice can be converted into text in real time and displayed, for example, in the input box 210. For example, after completing the voice input (e.g., the user releases the long-pressing of the voice input button 240), a cancel or exit button can also be displayed for deleting the input that has been converted into text, so that the user can re-input voice.

[0044] Optionally, as shown, the link information 250 can be displayed on the interface 200 of the launcher to start the specific desktop application. For example, the link information 250 is a shortcut key for the user to open the main window of the specific desktop application.

[0045] The interface 200 of the launcher displays a plurality of generation functions 220. In some examples, a predetermined number of generation functions can be displayed in the interface 200, such as the 5 generation functions shown in FIG. 2, including: AI search, AskName-XX, translation, upload file, and help me write; and other more generation functions are displayed in a folded manner, such as "More AI Skills" shown in FIG. 2. For example, in response to the user's operation (e.g., clicking) on "More AI Skills", other generation functions can be displayed in an unfolded manner.

[0046] In some examples, more than a predetermined number of generation functions can be displayed in the interface 200 of the launcher, or more than a predetermined number of generation functions can be displayed based on the user's operation on "More AI Skills". For example, the generation functions can be displayed in a scrolling manner, for example, a scroll bar can be displayed on the left or right side, and the user can view the generation functions that cannot be displayed in full by scrolling. It can be understood that the term "generation function" in the embodiments of the present disclosure is only an example, and the term can be replaced by other names such as AI skill, generation skill, etc., which are not limited herein.

[0047] In embodiments of the present disclosure, the multiple generation functions can include some or all of the following: AI search, Ask Name-XX, translation, upload file, help me write, rewrite, expand, grammar correction, writing improvement, link summary, rewrite script, generate picture, generate music, generate weekly report, generate OKR, explain code line by line, code on demand, interpret code error, brainstorm, extract task item, summarize advantages and disadvantages, refine keywords, optimize prompt words, optimize search words, etc. Alternatively, the multiple generation functions can also include summarizing or understanding documents or music or videos, etc., so as to facilitate the user to ask questions about the documents or music or videos, etc. Alternatively, the multiple generation functions and the displayed order can be application default settings, which can be preset in a certain module of the application.

[0048] In some implementations, a triggering operation of the user for a certain specific function (referred to as the first function for the sake of description) in the multiple generation functions can be obtained. For example, the user can select and / or click the first function. In some implementations, based on the triggering operation, an interface associated with the first function can be presented, for example, the interface associated with the first function can include an input box and prompt information associated with the first function.

[0049] In some embodiments, input information of the user can be obtained on the interface associated with the first function, and then an output corresponding to the input information can be generated by using the first function, and the output can be presented. It can be understood that the content, format, etc. of the output depend on the input information of the user and the first function selected by the user.

[0050] As an example, the output can be presented on the interface of the launcher. For example, if the first function is a generation function such as AI search, rewrite, expand, grammar correction, etc., the output can be directly displayed on the interface of the launcher. For example, the output can be displayed on another area associated with the input box, for example, an interactive interface such as below the input box, as shown in FIG. 3B or FIG. 3C.

[0051] As another example, the interface of a specific desktop application (such as Name-XX) can be presented, and the output can be presented on the interface of the specific desktop application (such as Name-XX). For example, if the first function is a generation function such as upload file, generate music, document or video summary, etc., the output can be displayed on the interface of the specific desktop application. For example, the interface of the specific desktop application can be called and opened, and the output can be displayed on the opened interface, as shown in FIG. 5B or FIG. 5C. It can be understood that in the case of displaying the interface of the specific desktop application, the interface of the launcher can also be presented again based on the wake-up operation of the user.

[0052] Some examples of generating and presenting the output after the user selects the first function are described below in connection with FIGS. 3A-5C.

[0053] FIG. 3A illustrates a schematic diagram of an interface 300 associated with the AI search. Exemplarily, the user can enter the interface 300 associated with the first function (i.e., the AI search) by, for example, clicking the first function (i.e., the AI search) in the interface 200 as shown in FIG. 2.

[0054] As shown in FIG. 3A, the interface 300 includes an input box 310. Optionally, the input box 310 can display prompt information associated with the first function. For example, the prompt information can be the name of the first function. For example, the input box 310 can display "AI search" in the form of a placeholder.

[0055] Exemplarily, a return button can also be displayed at a position associated with the input box 310, for example, the return button 302 is displayed at the left side of the input box 310. For example, in response to the user's operation (e.g., clicking) on the return button 302, the interface (e.g., the interface 200 as shown in FIG. 2) that starts the launcher can be returned. Exemplarily, the interface 300 can also display a voice input button 304, which is similar to the voice input button 240 described in connection with FIG. 2, which will not be described here.

[0056] As shown, a "send" button is also displayed, as shown in the lower right corner of the interface 300. Exemplarily, the send button can be disabled in the case that there is no input information of the user in the input box 310. Exemplarily, the send button is enabled in the case that there is input information of the user in the input box 310. As an example, assume that the input information of the user in the input box 310 is "What is the biggest octopus?" 320. After that, the answer (i.e., the output) to the input information of the user can be displayed based on the further confirmation operation of the user, as shown in FIG. 3B or FIG. 3C. Exemplarily, the confirmation operation can be that the user clicks the enter key or the user clicks the send button or the user voice inputs confirmation, etc., which is not limited in the present disclosure.

[0057] Figure 3B illustrates one embodiment of the output displayed for the input information 320. Illustratively, the output can be displayed in the interface of the launcher. Illustratively, one or more operation buttons can also be displayed in association at predetermined locations. As shown in Figure 3B, operation buttons 330 can be displayed at the bottom end, including copy, replace, insert, etc. The "insert" operation in the operation buttons 330 can be used to insert the generated output into the window of the running application before the launcher is invoked. For example, if the launcher is invoked during the display of the word running interface, the generated output can be directly added to the word through the "insert" operation, which can avoid the user to add the content in the word through copy and paste operations, thus more convenient and efficient. Illustratively, link information can also be displayed, such as "open Name-XX to continue" displayed in the lower right corner, for example, the user can trigger the link information to display the interface of a specific desktop application (such as Name-XX) to facilitate the user to perform subsequent operations. It should be noted that several operations in the operation buttons 330 are only illustrative, for example, more or less operations can be included in the actual scenario, for example, a cancel button can be included, the user can trigger the cancel button to return the interface to the interface above or below in Figure 3A.

[0058] Illustratively, the input information in the input box can also be further edited to modify the previous input. During the editing process, the operation buttons 330 and the link information can no longer be displayed, and instead the send button as shown in the lower right corner of Figure 3A is displayed.

[0059] Figure 3C illustrates another embodiment of the output displayed for the input information 320. Illustratively, the output can be displayed in the interface of the launcher. Illustratively, one or more operation buttons and link information can also be displayed in association at predetermined locations (such as the bottom end) similar to Figure 3B. As shown in Figure 3, operation buttons can also be displayed at other locations on the interface, such as at least one operation button 342 displayed near the middle.

[0060] Illustratively, one or more extended questions can also be displayed, such as shown, a plurality of extended questions 351-353 are displayed below the at least one operation button 342. In this way, if the user is interested in the extended questions, the answers can be obtained through the triggering of the extended questions, saving the time and process of re-input, improving the user experience.

[0061] Figure 4 illustrates a schematic diagram of an interface 400 associated with translation. Illustratively, the user can enter the interface 400 associated with the first function (i.e., "translation") by clicking the first function (i.e., "translation") in the interface 200 as shown in Figure 2.

[0062] For example, in the interface associated with the translation, an input box can be displayed, and optionally, a prompt (e.g., the name of the first function, “translation”) associated with the first function can be displayed in the input box in the form of a placeholder. For example, a back button can also be displayed in a position associated with the input box, such as a back button displayed on the left side of the input box. For example, in response to the user’s operation (e.g., clicking) on the back button, the interface (e.g., the interface 200 shown in FIG. 2) displaying the launcher can be returned. For example, a voice input button can also be displayed in the interface, which is similar to the voice input button 240 described in conjunction with FIG. 2, and thus will not be described here.

[0063] For example, in the interface associated with the translation, the source language and the target language can be displayed. For example, the source language can be automatically identified based on the user’s input in the input box, or the source language can be updated or determined based on the user’s selection. For example, the target language can be determined based on the user’s operation (e.g., selecting a specific language) at the target language box 410. For example, the output corresponding to the input, i.e., the translation in the target language, can be generated.

[0064] For example, one or more operation buttons can also be displayed in association with a predetermined position. As shown in FIG. 4, an operation button 420 can be displayed, including copy, insert, regenerate, etc.

[0065] FIG. 5A shows a schematic diagram of an interface 500 associated with uploading a file. For example, the user can enter the interface 500 associated with the first function (i.e., “upload file”) by, for example, clicking the first function (i.e., “upload file”) in the interface 200 shown in FIG. 2.

[0066] As shown, a prompt “select file” 510 can be displayed in a position associated with the generation function “upload file”. For example, when the user selects the generation function by, for example, clicking the upload file, a file selection interface 520 can be displayed, so that the user can select from the files stored locally on the electronic device.

[0067] For example, after the user determines the selected file, the AI reading interface or the AI Q&A interface can be entered. For example, after the user determines the selected file, the interface of a specific desktop application can be presented, such as the interface of a dedicated browser shown in FIG. 5B or the interface of the Name-XX application shown in FIG. 5C.

[0068] Exemplarily, as shown in FIG. 5B, the first region 532 of the interface of the browser can display the opened file, and the second region 534 can display the loading status of the file and the available generation functions. Exemplarily, as shown in FIG. 5C, a region 540 of the interface of the Name-XX application can display the loading status of the file and the available generation functions. For example, the user can input a question in the input box of the region 534 or the region 540, so that the specific desktop application (such as the dedicated browser or the Name-XX application) generates the expected answer.

[0069] The above describes some examples of the present disclosure in combination with FIGS. 3A-5C. The user can have explicitly specified the corresponding generation function and selected the first function, so that the launcher can generate the output based on the input of the user using the first function. It should be understood that the operation for other generation functions other than FIGS. 3A-5C is similar, and will not be listed one by one here. It should be noted that FIGS. 3A-5C only show the interface schematically, and in actual scenarios, more or less content or controls can be presented on the interface, which is not limited by the present disclosure.

[0070] Now returning to FIG. 2, in some implementations, the user can enter the first input information at the input box 210 of the interface 200 of the launcher, for example, the user does not select a specific generation function. In some implementations, in response to the first input information of the user at the input box 210, the first output corresponding to the first input information can be generated.

[0071] In some implementations, the first input information from the user at the input box 210 can be obtained, and then the intention of the user can be determined based on the first input information. Exemplarily, the corresponding query can be determined based on the intention of the user, and the first output can be obtained based on the corresponding query.

[0072] In some embodiments, the intention of the user can represent the interaction scenario expected by the user. For example, the interaction scenario expected by the user is to locally query the local file or the local application, for example, the intention of the user is local search. For example, the interaction scenario expected by the user is to query using at least one search engine, for example, the intention of the user is internet search. For example, the interaction scenario expected by the user is to query using a specific desktop application associated with the launcher, for example, the intention of the user is to solve the demand based on the capability of the large model.

[0073] In some implementations, the user intent can be determined based on the first input information of the user, for example, the scenario of the user intent is local search, search engine search, or solving the demand based on the capability of the large model. Further, the scenario of the user intent can be applied, and the output corresponding to the scenario is presented (or is preferentially presented) to the user. For example, if the determined user intent is to solve the demand based on the capability of the large model, further, one or more matched generation functions can be determined based on the first input information, and then the corresponding output is obtained using the determined one or more matched generation functions.

[0074] Exemplarily, if the determined user intent is local search, the first output corresponding to the first input information can be found by querying the local files or local applications, for example, the first output can include the link of the target file or target application in the local. Exemplarily, when the first output is displayed, the multiple generation functions can also be continuously displayed to facilitate the subsequent operation of the user.

[0075] FIG. 6A shows one embodiment of local search 600. The user can input the first input information 610 such as “terminal” in the input box. As shown, the local files and / or local applications found by local search can be displayed in the first area 612, and further, the multiple generation functions can be displayed in the second area 614.

[0076] The first output displayed in the first area 612 can include multiple entries, exemplarily, a predetermined number (such as 4) of entries with higher matching degree can be displayed, and other entries are displayed by folding (such as through “more applications and files”). For example, if the user clicks “more applications and files”, the local file search page of the system can be opened.

[0077] It can be understood that the first output can include the target application in the local, so that the user can start the target application through operations such as clicking, etc.

[0078] It can be understood that the first output can include the target file in the local, and the target file can have different formats such as pdf, docx, xlsx, pptx, csv, etc. Optionally, for the target file, the user can start (such as in its format) the target file through operations such as clicking, etc. Optionally, for the target file, additional opening modes such as AI reading, AI Q&A, etc. can also be provided.

[0079] Referring to FIG. 6B, for the target file in pdf format, two opening modes of default application opening and AI reading 620 can be provided. Referring to FIG. 6C, for the target file in xlsx format, two opening modes of default application opening and AI Q&A 630 can be provided.

[0080] Exemplarily, if the determined user intent is to solve a demand based on the capability of the large model, one or more matched generation functions can be further determined based on the first input information. Subsequently, the first output corresponding to the first input information can be obtained using the one or more matched generation functions. As an example, assume that the first input information is “Translate the following sentence into English: What is the largest octopus?” At this time, it can be determined through parsing of the first input information that the user intent is to solve a demand based on the capability of the large model, the matched generation function is “translation”, and the target language is English. Then, the first output, i.e., the corresponding English translation, can be displayed similarly to FIG. 4. As another example, assume that the first input information is “Provide summary information of the local PDF <Terminal Use Skills>”, at this time, it can be determined through parsing of the first input information that the user intent is to solve a demand based on the capability of the large model, and the matched generation functions include “upload file” and “document summary”. Subsequently, the electronic device can obtain the target file <Terminal Use Skills> in PDF format from the local file, and obtain the first output using the document summary generation function, for example, a summary of the target file can be displayed on an interface similar to FIG. 5B or FIG. 5C.

[0081] Exemplarily, if the determined user intent is a search engine search, a search engine can be invoked to determine the first output corresponding to the first input information. As an example, assume that the first input information is “What is the address and phone number of restaurant A?” At this time, it can be determined through parsing of the first input information that the user intent is a search engine search. Then, a search can be performed using a search engine to obtain the corresponding first output. As an example, the first output can be a webpage, which can be displayed directly by jumping to the search engine, or which can be displayed as a link. As an example, the first output can be content in the webpage, for example, the corresponding answer.

[0082] In some implementations, the first input information from the user at the input box 210 can be obtained, and two or more of the following can be performed simultaneously based on the first input information: local query, query using at least one search engine, or query using a large language model of a specific desktop application; and the first output can be further obtained based on the results of the query.

[0083] Exemplarily, the first output includes a plurality of entries, for example, one or several entries based on local query, one or several entries based on search engine, and / or one or several entries based on large language model. In some examples, the plurality of entries can be divided into a plurality of parts: a first part based on local query, a second part based on large language model query, and / or a third part based on query using search engine.

[0084] Exemplarily, the plurality of entries can be arranged in an order of priority. For example, one or several entries related to the user intent have a high priority. For example, one or several entries based on the local query have a high priority. For example, at least one entry associated with the running application before invoking the launcher has a high priority. As an example, when displaying the running interface of the java script, the user invokes the interface of the launcher by the wake-up operation, and the first input information of the user is related to the meaning of the while statement. Then, the meaning and application of the while statement in the java script in the first output can be displayed preferentially.

[0085] Exemplarily, the plurality of entries can be arranged in an order based on the location of the query. For example, a first part based on the local query can be displayed in a first region, a second part based on the large language model query can be displayed in a second region, and a third part based on the query using the search engine can be displayed in a third region. As an example, the first region is located above the second region and the third region, and the second region is located above the third region. However, it should be understood that other arrangements are also applicable, which are not listed here. Exemplarily, depending on the size limit of the display region, the number of entries presented in the first region, the second region, and the third region can not exceed a corresponding threshold. For example, the same threshold, such as 4 or 5, is applied to each region. For example, different thresholds are applied to different regions, such as a threshold of 5 for the first region and a threshold of 4 for the second region.

[0086] Optionally, a region identifier can be displayed at the top of each region. Optionally, the region identifier of the region located at the top can be omitted. Optionally, depending on the plurality of entries, part or all of the first part, the second part, and the third part can be displayed.

[0087] FIG. 7 shows a schematic diagram of the first output 700. For the first input information “What is the largest octopus?”, the region 710 and the region 720 can be displayed, where the region 710 displays the second part based on the large language model query, and the third part based on the query using the search engine is displayed in the region 720, and the region 720 is located below the region 710. As shown, the region identifier 722 of the region 720, such as “search engine search”, is also displayed. In the example of FIG. 7, since there is no query result of the local query, the first output does not include the first part based on the local query.

[0088] Exemplarily, for the second part based on the large language model query, the user can activate a certain entry therein by selecting or clicking it. For example, in combination with FIG. 7, the user can select “AI search: What is the largest octopus?” in the region 710 and view the answer by clicking the enter, such as the reply shown in FIG. 3B or FIG. 3C.

[0089] Exemplarily, for the third part based on the query using the search engine, the recommended / suggested information can also be displayed simultaneously. For example, the keywords of the hits can be highlighted or emphasized, while the recommended / suggested information is not highlighted. For example, in combination with FIG. 7, the recommended or suggested information can include “in the middle of the Pacific”, “discovered by humans”, and the like.

[0090] As discussed above, in some scenarios, the first output corresponding to the first input information can be generated using a large language model. For example, the first output can include one or several entries obtained using the large language model. Specifically, in response to the user intent being to solve the demand based on the capability of the large model, the first output can be obtained using the large language model; or, the second part based on the large language model query in the first output can be obtained using the large language model.

[0091] In some examples, generating the first output based on the large language model can include generating the first output based on Retrieval-Augmented Generation (RAG). Exemplarily, when using a specific desktop application (such as Name-XX), the user can establish a personal database, for example, a RAG database called Name-XX. For example, the user can establish the personal database by means of collecting, bookmarking, and the like. For example, the personal database can include documents uploaded by the user, contents collected by the user after viewing through other channels, and the like. It can be understood that the disclosure embodiments do not limit the format, size, attribute, and the like of the contents in the personal database, for example, the contents can be in the format of text (such as files / documents), in the format of multimedia content (such as images / audio / videos), in the format of application programs, in the format of codes, and the like.

[0092] Exemplarily, the first input information and the contents in the personal database can be used as part of the input data of the large language model, so as to obtain the first output. For example, the input data can include the contents in the personal database associated with the first input information, for example, the first input information can be a prompt. For example, the input data can include background knowledge pre-learned by the large language model, and the like. In this way, the obtained first output can be more accurate, for example, the first output more in line with the user demand can be obtained.

[0093] As an example, learning can be performed based on the RAG to obtain learning content, and then a large language model can be used to obtain the first output based on the learning content and the first input information. As another example, content associated with the first input information in the personal database can be obtained, and then the RAG can be used to obtain the first output based on the content associated with the first input information in the personal database and the first input information. As another example, content associated with the first input information in the personal database can be obtained, and then the first output can be obtained based on the first input information, for example, the first output can include part or all of the content associated with the first input information in the personal database, for example, the first output includes part of the content associated with the first input information in the personal database, which has a stronger association with the first input information.

[0094] For example, the personal database includes a sales contract template, and the first input information indicates that the content of Chapter 8 of the sales contract needs to be generated. In one example, the output can include the original content of Chapter 8 in the sales contract template in the personal database. In another example, the original content of Chapter 8 in the sales contract template in the personal database can be revised, for example, updated based on the sales scenario indicated by the first input information, and the output can include the updated content after the revision of the original content of Chapter 8 in the sales contract template.

[0095] Through the exemplary embodiments of the present disclosure, the launcher is associated with a specific desktop application, and can perform AI generation based on a large language model, while the launcher is both a search entry for local queries and a search entry for search engines. In this way, the launcher and the functions of the operating system are combined, the large language model is connected with the operating system and the search engine, the entry is integrated into the input box of the launcher, the user does not need to determine the location to be queried in advance, and the interaction with the user can be quickly and comprehensively realized, thereby improving the user experience.

[0096] It should be understood that in the embodiments of the present disclosure, "first", "second", "third" and the like are only to indicate that the plurality of objects can be different, but at the same time do not exclude that two objects are the same, and should not be interpreted as any limitation on the embodiments of the present disclosure.

[0097] It should also be understood that the manners, cases, categories and divisions of embodiments in the embodiments of the present disclosure are only for the convenience of description, and should not be construed as special limitations. The features in various manners, categories, cases and embodiments can be combined with each other as long as they are logically consistent.

[0098] It should also be understood that all references given above are only to facilitate better understanding of the embodiments of the present disclosure by those skilled in the art, and are not intended to limit the scope of the embodiments of the present disclosure. Those skilled in the art can make various modifications or changes or combinations, etc. according to the above description. Such modified, changed or combined solutions are also within the scope of the embodiments of the present disclosure.

[0099] It should also be understood that the above description focuses on the differences between various embodiments, and the same or similar parts can be referred to or learned from each other, and for brevity, will not be repeated here.

[0100] FIG. 8 shows a schematic block diagram of an example apparatus 800 according to some embodiments of the present disclosure. The apparatus 800 can be implemented by software, hardware, or a combination of both. In some embodiments, the apparatus 800 can be implemented as an electronic device. In embodiments of the present disclosure, the electronic device can be a desktop computer, a tablet computer, etc., which are not limited by the present disclosure.

[0101] As shown in FIG. 8, the apparatus 800 includes a presentation module 810 and a generation module 820. The presentation module 810 is configured to, in a case where a desktop or an interface of a running application is presented, present, in response to a wake-up operation from a user, an interface of a launcher for interacting with the user, wherein the interface of the launcher includes an input box and a plurality of generation functions. The generation module 820 is configured to generate, based on an input from the user in the input box, an output corresponding to the input.

[0102] In some embodiments, the wake-up operation includes any of the following: the user triggers a shortcut key associated with the launcher, the user inputs a voice for invoking the launcher, a specific gesture of the user, or another predefined operation for invoking the launcher. In some embodiments, the interface of the launcher has a predetermined size, and the plurality of generation functions are displayed folded or scrolled based on the predetermined size.

[0103] In some embodiments, the running application includes any application running on an operating system of the computer.

[0104] For example, the apparatus 800 can further include an acquisition module configured to acquire a trigger operation of the user for a first function of the plurality of generation functions. Further, the presentation module 810 can be configured to present, in response to the trigger operation, an interface associated with the first function, wherein the interface associated with the first function includes an input box and prompt information associated with the first function. For example, the prompt information is displayed as a placeholder in the input box.

[0105] Exemplarily, the apparatus 800 can further include an obtaining module configured to obtain first input information in an input box from a user. Further, the generating module 820 is configured to conduct a query based on the first input information to generate a first output corresponding to the first input information, wherein the query comprises at least one of: a local query to a local file or a local application in the computer, a large language model query with a specific desktop application associated with the launcher, or a query with at least one search engine.

[0106] In some examples, the apparatus 800 can include a determining module configured to determine a user intent based on the first input information; and determine the corresponding query based on the user intent.

[0107] In some examples, the specific desktop application comprises a personal database, and the generating module 820 is configured to input the first input information and content of the personal database as part of input data of the large language model to obtain the first output.

[0108] In some examples, the first output comprises a plurality of entries, and the plurality of entries are arranged in an order based on positions of the query.

[0109] In some examples, the first output comprises a plurality of entries, and the plurality of entries comprise at least one of: a first part based on the local query, a second part based on the large language model query, and a third part based on the query with the at least one search engine.

[0110] In some examples, the presenting module 810 is further configured to display the first output by at least one of: displaying the first output in an interface of the launcher; or invoking the specific desktop application to display the first output in the desktop application. For example, at least one of: a copy button, a regenerate button, or a cancel button can be further displayed in association with the first output.

[0111] In some examples, the presenting module 810 is configured to prioritize display of at least one entry in the first output associated with a running application.

[0112] In some embodiments, the specific desktop application is implemented as at least one of: a dedicated browser application, a specific plug-in in the at least one search engine, or a specific web page.

[0113] The apparatus 800 of FIG. 8 can be used to implement the method 100 described above in connection with FIG. 1, and thus will not be described again in detail for brevity.

[0114] The division of modules or units in the embodiments of the present disclosure is illustrative, and is only a logical function division. When actually implemented, another division manner can be used. In addition, each functional unit in the disclosed embodiments can be integrated into one unit, or can be physically separated, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0115] FIG. 9 shows a block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure. It should be understood that the device 900 shown in FIG. 9 is merely an example and should not be construed to limit the functionality and scope of the implementations described herein. For example, the device 900 can be used to perform the method 100 described above.

[0116] As shown in FIG. 9, the device 900 is in the form of a general computing device. Components of the computing device 900 can include, but are not limited to, one or more processors or processing units 910, memory 920, storage 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. The processing unit 910 can be a real or virtual processor and is capable of executing various processing in accordance with programs stored in the memory 920. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 900.

[0117] The computing device 900 typically includes a number of computer storage media. Such media can be any available media that is accessible by the computing device 900 and includes both volatile and non-volatile media, removable and non-removable media. The memory 920 can be a volatile memory (such as a register, cache, or random access memory (RAM)), a non-volatile memory (such as read only memory (ROM), electrically erasable programmable read only memory (EEPROM), or flash memory), or some combination thereof. The storage 930 can be a removable or non-removable media, and can include machine-readable media, such as a flash drive, a disk drive, or any other media that can be used to store information and / or data (e.g., training data for training) and that can be accessed by the computing device 900.

[0118] The computing device 900 can further include additional removable / non-removable, volatile / non-volatile storage devices. Although not shown, a floppy disk drive for reading from or writing to a removable, non-removable magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-removable magnetic disk (e.g., a "floppy disk") can be provided. In such instances, each drive can be connected to the bus by one or more data media interfaces. The storage device(s) 920 can include a computer-program product 925 having one or more program modules that are configured to carry out the various processes or acts of the implementations of the present disclosure.

[0119] The communication unit 940 enables communications with other computing devices over a communication medium. Additionally, the functionality of the components of the computing device 900 can be implemented in a single computing cluster or multiple computer machines that are capable of communicating over a communication connection. Thus, the computing device 900 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0120] The input device(s) 950 can be one or more input device(s) such as a mouse, keyboard, trackball, etc. The output device(s) 960 can be one or more output device(s) such as a display, speakers, printer, etc. The computing device 900 can also include other peripheral components, such as one or more of a processor, a memory, a processor, an input device, an output device, an external device, an input / output (I / O) interface, a communication device, a storage device, a computer program product, etc. The computing device 900 can communicate with one or more external devices using the communication unit 940, such as a storage device, a display device, etc., communicate with one or more devices that enable a user to interact with the computing device 900, or communicate with any devices (e.g., a network card, a modem, etc.) that enable the computing device 900 to communicate with one or more other computing devices. Such communication can be enabled by an I / O interface (not shown).

[0121] According to example implementations of the present disclosure, a computer-readable storage medium, such as a non-transitory computer-readable storage medium, is provided having stored thereon computer-executable instructions that, when executed by a processor, implement the methods described above. According to example implementations of the present disclosure, a computer program product is also provided tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions that, when executed by a processor, implement the methods described above. According to example implementations of the present disclosure, a computer program product is provided having stored thereon a computer program which, when executed by a processor, implements the methods described above.

[0122] The computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all.

[0123] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all.

[0124] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all.

[0125] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0126] Having described various implementations of the disclosure above, the descriptions are not exhaustive and do not limit the disclosure to the disclosed implementations. Numerous modifications and adaptations thereof will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The scope of the disclosure includes all the implementations of which an equivalent would be apparent to those skilled in the art from the disclosure given and the associated drawings. The selection of the terms to be used in the written description is not intended to limit the scope of the present disclosure, but rather to best describe the principles of various implementations in preference to a detailed inventory of all technical terms.

Claims

1. A computer-implemented method, comprising: When displaying the desktop or the interface of a running application, in response to a wake-up operation from the user, a launcher interface for interacting with the user is presented, wherein the launcher interface includes input fields and multiple generation functions; and Based on the user's input in the input box, an output corresponding to the input is generated.

2. The method according to claim 1, wherein the wake-up operation includes: The user triggers the shortcut key associated with the launcher. The user input is the voice used to activate the launcher. The user's specific gesture, or Another predefined operation is used to invoke the launcher.

3. The method of claim 1, wherein the running application includes any application running on the operating system of the computer.

4. The method of claim 1, wherein the interface of the launcher has a predetermined size, and wherein the multiple generation functions are displayed in a collapsed or scrolled manner based on the predetermined size.

5. The method according to claim 1, further comprising: Obtain the user's trigger operation for the first function among the multiple generation functions; as well as In response to the triggering operation, an interface associated with the first function is presented, wherein the interface associated with the first function includes the input box and a prompt message associated with the first function.

6. The method according to claim 5, wherein the prompt information is displayed as a placeholder in the input box.

7. The method according to claim 1, further comprising: Obtain the first input information from the user in the input box; as well as A query is performed based on the first input information to generate a first output corresponding to the first input information, wherein the query includes at least one of the following: Local queries on local files or local applications on the computer. Utilize the large language model query of the specific desktop application associated with the launcher, or A query using at least one search engine.

8. The method according to claim 7, further comprising: Determine the user's intent based on the first input information; as well as The corresponding query is determined based on the user's intent.

9. The method of claim 7, wherein the particular desktop application includes a personal database, and wherein generating the first output includes: The first input information and the content of the personal database are used as part of the input data of the large language model to obtain the first output.

10. The method of claim 7, wherein the first output comprises a plurality of entries, and the plurality of entries are arranged in order based on the position of the query.

11. The method of claim 7, wherein the first output comprises a plurality of entries, and the plurality of entries comprises at least a portion of: Based on the first part of the local query. The second part of the query based on the large language model, and The third part is based on the query using at least one of the search engines.

12. The method of claim 7, further comprising displaying the first output by at least one of the following methods: The first output is displayed in the interface of the launcher; or Invoke the specific desktop application to display the first output in the desktop application.

13. The method of claim 12, further comprising: The following are displayed in association with the first output: a copy button, a regenerate button, or a cancel button.

14. The method of claim 7, further comprising: At least one entry in the first output that is associated with the running application will be displayed preferentially.

15. The method of claim 7, wherein the particular desktop application is implemented as at least one of the following: a dedicated browser application, a specific plugin in the at least one search engine, or a specific webpage.

16. An electronic device comprising: At least one processor; as well as At least one memory having computer-executable instructions stored thereon, which, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1 to 15.

17. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, implement the method according to any one of claims 1 to 15.

18. A computer program product comprising computer-executable instructions, which, when executed, implement the method according to any one of claims 1 to 15.

Images