Computer implementation method and electronic equipment

Abstract

The embodiment of the invention provides a computer implementation method and electronic equipment. The method comprises the steps that under the condition that an interface of a desktop or a running application is displayed, in response to awakening operation from a user, an interface of a starter used for interacting with the user is presented, and the interface of the starter comprises an input box and multiple generation functions. The method further includes generating an output corresponding to an input based on the input in the input box from the user. Based on this, an initiator for interacting with a user may be quickly evoked and presented when the computer displays any interface. Therefore, the user operation can be facilitated, and the processing speed can be improved for the realization of the generation function.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of computers, and more specifically, to a computer-implemented method and an electronic device. Background Art

[0002] With the development of Large Language Model (LLM) technology, a variety of models have been proposed for users to access, such as asking questions and getting answers. Current large language models usually require users to access specific artificial intelligence (AI) dialogue software separately, which makes the user's operation more cumbersome. In addition, the current specific AI dialogue software is independent of the operating system and has relatively limited functions. Summary of the invention

[0003] The embodiments of the present disclosure provide a solution based on a large language model combined with an operating system, so that a user can quickly wake up the interface of a launcher, facilitate human-computer interaction, and improve user experience.

[0004] In a first aspect of the present disclosure, a computer-implemented method is provided, comprising: in response to a wake-up operation from a user, while displaying the interface of a desktop or a running application, presenting an interface of a launcher for interacting with the user, wherein the interface of the launcher includes an input box and multiple generation functions; and based on the input from the user in the input box, generating an output corresponding to the input.

[0005] In a second aspect of the present disclosure, an electronic device is provided, comprising: at least one processing unit; and at least one memory, wherein the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, wherein the instructions, when executed by the at least one processing unit, cause the electronic device to perform actions, the actions comprising: in response to a wake-up operation from a user, while displaying an interface of a desktop or a running application, presenting an interface of a launcher for interacting with the user, wherein the interface of the launcher comprises an input box and a plurality of generation functions; and based on the input from the user in the input box, generating an output corresponding to the input.

[0006] In a third aspect of the present disclosure, a device is provided, comprising: a presentation module configured to present, in response to a wake-up operation from a user, an interface of a launcher for interacting with a user while displaying an interface of a desktop or a running application, wherein the interface of the launcher comprises an input box and a plurality of generation functions; and a generation module configured to generate an output corresponding to the input in the input box based on the input from the user.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, which has machine-executable instructions stored thereon, and when the machine-executable instructions are executed by a device, the device performs the method described in the first aspect of the present disclosure.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising computer executable instructions, wherein the computer executable instructions implement the method described in the first aspect of the present disclosure when executed by a processor.

[0009] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising: a processing circuit configured to execute the method described according to the first aspect of the present disclosure.

[0010] The invention summary is provided to introduce a series of concepts in a simplified form, which will be further described in the specific embodiments below. The invention summary is not intended to identify the key features or essential features of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0012] Figure 1 A schematic flow chart showing a computer-implemented method according to some embodiments of the present disclosure;

[0013] Figure 2 A schematic diagram showing an interface of a evoked launcher according to some embodiments of the present disclosure;

[0014] FIG. 3A to FIG. 3C A schematic diagram showing an interface associated with AI search according to some embodiments of the present disclosure;

[0015] Figure 4 A schematic diagram showing an interface associated with translation according to some embodiments of the present disclosure;

[0016] FIG. 5A to FIG. 5C A schematic diagram showing an interface associated with uploading text according to some embodiments of the present disclosure;

[0017] FIG. 6A to FIG. 6C A schematic diagram showing an interface associated with a local query according to some embodiments of the present disclosure;

[0018] Figure 7 A schematic diagram showing a first output according to some embodiments of the present disclosure;

[0019] Figure 8 A block diagram showing an example apparatus according to some embodiments of the present disclosure; and

[0020] Fig. 9 A block diagram of an example electronic device that may be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or identical objects. The term "and / or" means at least one of the two items associated therewith. For example, "A and / or B" means A, B, or A and B. Other explicit and implicit definitions may also be included below.

[0023] In the embodiments of the present disclosure, the term "big language model" is involved, which can realize the understanding and generation of universal language. It should be noted that the use of this term is only illustrative and is not intended to limit the embodiments of the present disclosure. In some scenarios, the term "big language model" can also be replaced by any one or a combination of the following: network model, neural network model, neural network, machine learning (Machine Learning, ML) model, deep learning (Deep Learning, DL) model, deep network, generative AI model, etc., which is not limited by the present disclosure.

[0024] The large language model can be obtained through a computationally intensive self-supervised and semi-supervised training process. In some examples, the large language model can be based on a transformer architecture. In some examples, the large language model can include an encoder and a decoder, or only a decoder.

[0025] With the development of large language model technology and its widespread application, users may expect to ask questions and get answers more conveniently. However, the current AI dialogue software is relatively independent and requires users to use it alone, which is not convenient enough.

[0026] In view of this, an embodiment of the present disclosure provides a solution based on large language model technology. In this solution, when displaying the interface of a desktop or a running application, in response to a wake-up operation from a user, an interface of a launcher for interacting with the user is presented, wherein the interface of the launcher includes an input box and a plurality of generation functions. Then, based on the input from the user in the input box, an output corresponding to the input is generated. Based on this, a launcher for interacting with a user can be quickly called up and presented when the computer displays any interface. Therefore, it can facilitate user operation, and the processing speed can be increased for the realization of generation functions. In some examples, the launcher can be combined with an operating system and a search engine, without the user having to predetermine the location of the query, saving the user time and energy for prejudgment and improving the user experience.

[0027] Figure 1 A schematic flow chart of a computer-implemented method 100 according to some embodiments of the present disclosure is shown. Figure 1 The method shown can be performed by an electronic device, for example, a computer. In box 110, the electronic device presents a launcher interface for interacting with the user in response to a wake-up operation from the user while displaying the desktop or the interface of the running application, wherein the launcher interface includes an input box and multiple generation functions. In box 120, the electronic device generates an output corresponding to the input based on the input from the user in the input box.

[0028] An operating system is installed on the electronic device, and multiple applications are also installed, wherein the multiple applications may include a specific desktop application associated with a launcher. For example, the specific desktop application is implemented as: a dedicated browser application, a specific plug-in in at least one search engine, or a specific web page, etc. It should be understood that the embodiments of the present disclosure do not limit the name of the specific desktop application. For the sake of convenience, the following description takes the name of the specific desktop application as "Name-XX" as an example.

[0029] It should be understood that the term "launcher" is used herein to refer to a dialog software used to interact with a user to provide the user's desired answer or content. The name is exemplary only and should not be interpreted as a limitation on the embodiments of the present disclosure. In some cases, the launcher may also call other applications to provide the user's desired answer or content, thereby satisfying the user's generation needs, interpretation needs, etc. For example, the launcher may be a generative AI dialog interaction window that is combined with an operating system and / or a search engine and applies LLM technology.

[0030] It should be understood that the term "wake up" is used in this article to represent the presentation of the launcher interface, such as presenting the launcher interface at the top level of the interface, but the name is only exemplary. For example, the terms "evoke", "trigger" and so on are also used below to express the same meaning.

[0031] When using an electronic device, a user may want to know the answers to certain questions or want to query certain information or content. When using an electronic device, the user can use a wake-up operation at any time to make the electronic device present the interface of the launcher. In some embodiments, the wake-up operation can be a pre-set or pre-defined predefined operation for waking up the launcher, as described below in conjunction with a more detailed example.

[0032] In some examples, when the electronic device displays the desktop of the operating system, the user can wake up the interface of the launcher through a wake-up operation. In some examples, when the electronic device displays the running interface of any application, the user can wake up the interface of the launcher through a wake-up operation. In some examples, the interface of the launched launcher can be presented in the form of a floating window. In some examples, the interface of the launched launcher presented is not displayed in full screen, for example, the interface of the launched launcher presented can have a predetermined size, and the size of the interface of the launcher can be further adjusted based on the user's adjustment instruction. The embodiment of the present disclosure does not limit the specific size of the predetermined size, for example, it can be 1 / 9 of the entire display screen, for example, it can be determined based on the size of the displayed font, for example, it can be other sizes, etc. It is understandable that before the interface of the launcher is awakened, the electronic device may display the running interface of any application running on the operating system, where the application can be an application that comes with the operating system, or an application that the user installs independently, and so on. For example, when the user is editing a word, he wants to know the answer to some questions, so when the running interface of the word is displayed, the user wakes up the interface of the launcher through a wake-up operation. For another example, when a user is doing Java programming and wants to know the answers to certain questions, when the Java script running interface is displayed, the user wakes up the launcher interface through a wake-up operation. It should be understood that these are only some examples, and other scenarios are also applicable, which are not listed in this article.

[0033] For example, the user may trigger a shortcut key associated with the launcher to invoke the launcher interface, for example, the user may press a group of keys on the keyboard at the same time. For example, the shortcut key may be a shortcut key set by the application by default, or may be a shortcut key personalized by the user (such as Fn+Shift+O or others). For example, the user may set the shortcut key in the settings page of a specific desktop application.

[0034] For example, the electronic device may have a voice collection device (such as a microphone), and the user may input a voice for waking up the launcher to wake up the launcher interface, for example, the user may speak the voice in a preset language. For example, the voice for waking up the launcher may be a voice set by the application by default, or may be a voice set by the user (such as "open the launcher", "pop up the launcher", or others).

[0035] For example, the electronic device may have an image acquisition device (such as a camera), and the user may make a specific gesture for waking up the launcher to wake up the launcher interface, for example, the gesture may be a static gesture or a dynamic gesture. For example, the specific gesture for waking up the launcher may be a gesture set by default for the application, or may be a gesture personalized by the user (such as making an OK gesture with both hands, or making an OK gesture with one hand and drawing a circle within a predetermined time period, or others).

[0036] For example, the user can right-click on the desktop of the operating system and call up the launcher interface by selecting a specific option. For example, the specific option can be "open launcher", "pop-up launcher" or other similar content.

[0037] For example, a shortcut button for calling up the launcher may be presented on the desktop of the operating system or at a specific location associated with the desktop (such as a toolbar, a taskbar, etc.), and the user may call up the launcher by selecting, clicking, double-clicking, or long pressing the shortcut button.

[0038] For example, the running interface of the application being displayed may present a shortcut button for waving the launcher, and the user may waving the launcher by selecting, clicking, double-clicking, or long pressing the shortcut button, etc. For example, the shortcut button may be set to be displayed in a custom ribbon of a word document.

[0039] It is understandable that some examples of wake-up operations are given above, but the embodiments of the present disclosure are not limited thereto, and the wake-up operation can be implemented through other settings or definitions. Exemplarily, the floating window displays the minimized launcher, and the user can double-click or long press or click the restore button to make the launcher interface appear in a predetermined size.

[0040] Figure 2 Schematic diagram of an interface 200 of a launched launcher according to some embodiments of the present disclosure is shown. Exemplarily, the launcher interface 200 may be presented based on a user's wake-up operation while displaying the desktop of the operating system. Optionally, the launcher interface 200 may be understood as the initial interface of the launcher, etc. Figure 2 As shown, the interface 200 of the launcher includes an input box 210 and a plurality of generating functions 220 .

[0041] As shown, a prompt message may be displayed in the input box 210, such as displaying "What do you want to search for?" in the form of a placeholder. Optionally, the user may input in the input box 210, for example, in the form of text. Exemplarily, a voice input button 240 may also be displayed in the interface 200. For example, it may be displayed in association with the input box 210. For example, the user may click the voice input button 240, and then input in the form of voice. For example, the user may type a specific button in the input box 210 to enable the voice input, such as a space bar, etc.

[0042] Exemplarily, the user can enter the voice input mode by selecting (such as long pressing) the voice input button 240, for example, a voice input window corresponding to the voice input mode can be displayed. For example, the voice input window may include a close button (such as ×) for closing the voice output window. For example, when the user inputs a voice, a dividing line can be displayed, and the dividing line can be a dynamic animation of the voice of the conversation. For example, when the voice input by the user is obtained, the voice can be converted into text in real time and displayed, for example, in the input box 210. For example, after completing the voice input (such as the user long pressing the voice input button 240 and releasing it), a cancel or exit button can also be displayed to delete the input that has been converted into text, so that the user can re-enter the voice.

[0043] Optionally, as shown in the figure, link information 250 can be displayed on the launcher interface 200 to start the specific desktop application. For example, the link information 250 is a shortcut key for the user to open the main window of the specific desktop application.

[0044] A plurality of generating functions 220 are displayed in the interface 200 of the launcher. In some examples, a predetermined number of generating functions may be displayed in the interface 200, such as Figure 2 The five generated functions shown in the figure include: AI search, Ask Name-XX, Translation, Upload File, Help Me Write; and the others are more folded, such as in Figure 2 ”More AI skills” shown in . Exemplarily, in response to a user operation on “More AI skills”, such as clicking, other generation functions may be expanded and displayed.

[0045] In some examples, more than a predetermined number of generation functions may be displayed in the interface 200 of the launcher, or more than a predetermined number of generation functions may be displayed based on the user's operation for "more AI skills". Exemplarily, these generation functions may be displayed in a scrolling manner, such as by displaying a scroll bar on the left or right side, and the user may scroll to view all generation functions that cannot be displayed. It is understandable that the term "generation function" in the embodiments of the present disclosure is only an example, and the term may be replaced by other names such as AI skills, generation skills, etc., which is not limited herein.

[0046] In an embodiment of the present disclosure, multiple generation functions may include some or all of the following: AI search, ask Name-XX, translation, upload file, help me write, rewrite, expand, grammar correction, writing improvement, link summary, rewrite script, generate pictures, generate music, generate weekly report, generate OKR, explain code line by line, urge code, interpret code errors, brainstorm, extract task items, summarize advantages and disadvantages, refine keywords, optimize prompt words, optimize search terms, etc. Optionally, multiple generation functions may also include summaries or understandings of documents, music, videos, etc., so that users can ask questions about documents, music, videos, etc. Optionally, multiple generation functions and the order in which they are displayed may be the default settings of the application, for example, they may be preset in a module of the application.

[0047] In some implementations, a user's trigger operation for a specific function (for convenience of description, referred to as the first function) among the multiple generated functions may be obtained. For example, the user may select and / or click the first function. In some implementations, an interface associated with the first function may be presented based on the trigger operation, for example, the interface associated with the first function may include an input box and prompt information associated with the first function.

[0048] In some embodiments, the user's input information can be obtained on the interface associated with the first function, and then the first function can be used to generate output corresponding to the input information, and the output can be presented. It is understood that the content and format of the output depend on the user's input information and the first function selected by the user.

[0049] As an example, the output may be presented on the interface of the launcher. For example, if the first function is a generating function such as AI search, rewriting, expansion, grammar correction, etc., the output may be directly displayed on the interface of the launcher. Exemplarily, the output may be displayed on another area associated with the input box, such as an interactive interface (such as below the input box), for example, as follows Figure 3B or Figure 3C shown.

[0050] As another example, the interface of a specific desktop application (such as Name-XX) can be presented, and the output can be presented on the interface of the specific desktop application (such as Name-XX). For example, if the first function is a generation function such as uploading a file, generating music, a document, or a video summary, the output can be displayed on the interface of the specific desktop application. Exemplarily, the interface of the specific desktop application can be called and opened, and the output can be displayed on the opened interface, such as the following Figure 5B or Figure 5C It is understandable that, when the interface of the specific desktop application is displayed, the interface of the launcher can be presented again based on the user's wake-up operation.

[0051] Combine the following FIG. 3A to FIG. 5C To describe some examples of generating and presenting output after the user selects the first function.

[0052] Figure 3A FIG. 3 is a schematic diagram of an interface 300 associated with AI search. For example, a user may Figure 2 In the interface 200 shown, by, for example, clicking on the first function (ie, “AI search”), an interface 300 associated with the first function (ie, “AI search”) is entered.

[0053] like Figure 3A As shown, the interface 300 includes an input box 310. Optionally, prompt information associated with the first function may be displayed in the input box 310. For example, the prompt information may be the name of the first function. For example, "AI search" may be displayed in the input box 310 as a placeholder.

[0054] For example, a return button may be displayed at a position associated with the input box 310, for example, a return button 302 may be displayed on the left side of the input box 310. For example, in response to a user operation (such as clicking) on ​​the return button 302, the interface displaying the launcher (such as Figure 2 For example, the interface 300 may also display a voice input button 304, which is combined with Figure 2 The voice input button 240 is similar and will not be described again here.

[0055] As shown in the figure, a "Send" button is also displayed, as shown in the lower right corner of the interface 300. For example, in the case where the user's input information is not typed in the input box 310, the send button can be disabled. For example, in the case where the user's input information already exists in the input box 310, the send button is enabled. As an example, assume that the user's input information in the input box 310 is "What is the largest octopus?" 320. After that, the answer (i.e., output) to the user's input information can be displayed based on the user's further confirmation operation, such as Figure 3B or Figure 3C As shown. Exemplarily, the confirmation operation may be the user clicking the enter key or the user clicking the send key or the user entering a voice input to confirm, etc., which is not limited in the present disclosure.

[0056] Figure 3B An embodiment of an output displayed for input information 320 is shown. For example, the output may be displayed in the interface of the launcher. For example, one or more operation buttons may be displayed in association at a predetermined position. Figure 3B As shown, operation buttons 330 can be displayed at the bottom, including copy, replace, insert, etc. The "Insert" operation in the operation button 330 can be used to insert the generated output into the window of the running application before the launcher is invoked. For example, if the launcher is invoked during the display of the word running interface, the generated output can be directly added to the word through the "Insert" operation, which can avoid the user from adding content to the word through copy and paste operations, making it more convenient and efficient. Exemplarily, link information can also be displayed, such as "Open Name-XX to continue" displayed in the lower right corner. For example, the user can trigger the link information to display the interface of a specific desktop application (such as Name-XX), which is convenient for the user to perform subsequent operations. It should be noted that the several operations in the operation button 330 are only for illustration. For example, more or fewer operations may be included in actual scenarios, such as a cancel button. The user can trigger the cancel button to return the interface to the following state. Figure 3A Upper interface or lower interface.

[0057] Exemplarily, the input information in the input box can also be further edited to modify the previous input. In this editing process, the operation button 330 and the link information can no longer be displayed, but instead displayed as follows: Figure 3A The Send button shown in the lower right corner.

[0058] Figure 3C Another embodiment of the output displayed for the input information 320 is shown. For example, the output can be displayed in the interface of the launcher. For example, it can also be displayed with Figure 3BSimilarly, one or more operation buttons and link information are displayed in association at a predetermined position (such as the bottom). As shown in Figure 3, operation buttons can also be displayed at other positions on the interface, such as displaying at least one operation button 342 near the middle position.

[0059] Exemplarily, one or more extended questions may also be displayed, as shown in the figure, multiple extended questions 351-353 are displayed below at least one operation button 342. In this way, if the user is interested in these extended questions, the answers can be obtained by triggering the extended questions, which saves the time and process of re-entering and improves the user experience.

[0060] Figure 4 400 is a schematic diagram of an interface associated with translation. Figure 2 In the illustrated interface 200 , by clicking on the first function (ie, “Translate”), an interface 400 associated with the first function (ie, “Translate”) is entered.

[0061] Exemplarily, in the interface associated with translation, an input box may be displayed. Optionally, the input box may display prompt information associated with the first function (e.g., the name of the first function "translation") in the form of a placeholder. Exemplarily, a return button may also be displayed at a position associated with the input box, such as displaying a return button on the left side of the input box. For example, in response to a user operation (e.g., clicking) on ​​the return button, the interface displaying the launcher (e.g., Figure 2 As shown in the interface 200). For example, the interface may also display a voice input button, which is combined with Figure 2 The voice input button 240 is similar and will not be described again here.

[0062] Exemplarily, in the interface associated with the translation, the original language and the target language can be displayed. Exemplarily, the original language can be automatically identified based on the user's input in the input box, or the original language can be updated or determined based on the user's selection. Exemplarily, the target language can be determined based on the user's operation at the target language box 410 (such as selecting a specific language). Exemplarily, an output corresponding to the input, i.e., a translation in the target language, can be generated.

[0063] Exemplarily, one or more operation buttons may also be displayed in association at a predetermined position. Figure 4 As shown, operation buttons 420 may be displayed, including copy, insert, regenerate, etc.

[0064] Figure 5A FIG. 5 is a schematic diagram of an interface 500 associated with uploading a file. For example, a user may Figure 2In the interface 200 shown, by, for example, clicking on the first function (ie, “upload file”), an interface 500 associated with the first function (ie, “upload file”) is entered.

[0065] As shown, at a location associated with the generation function "upload file", a prompt message "select file" 510 may be displayed. Exemplarily, when the user selects the generation function by, for example, clicking to upload a file, a file selection interface 520 may be displayed so that the user can select from files stored locally on the electronic device.

[0066] For example, after the user confirms the selected file, the user can enter the AI ​​reading companion interface or the AI ​​question and answer interface. For example, after the user confirms the selected file, the user can present an interface with a specific desktop application, such as Figure 5B The interface of the dedicated browser shown or Figure 5C The interface of the Name-XX application is shown.

[0067] For example, Figure 5B As shown, the first area 532 of the browser interface may display the opened files, and the second area 534 may display the loading status of the files and the available generation functions. Figure 5C As shown, a certain area 540 of the Name-XX application interface can display the loading status of the file and the available generation functions. For example, the user can enter a question in the input box of area 534 or area 540 so that the specific desktop application (such as a dedicated browser or Name-XX application) generates a desired answer.

[0068] Combined with the above FIG. 3A to FIG. 5C Some examples of the present disclosure are described, and the user may have specified the corresponding generation function and selected the first function, so that the launcher can generate output using the first function based on the user's input. FIG. 3A to FIG. 5C The operations of other generation functions are similar and will not be listed here one by one. It should be noted that FIG. 3A to FIG. 5C What is shown is only a schematic diagram of the interface. In actual scenarios, more or less content or controls may be presented on the interface, and the present disclosure is not limited to this.

[0069] Return now Figure 2 In some implementations, the user may enter first input information in the input box 210 of the launcher interface 200, for example, the user has not selected a specific generation function. In some implementations, in response to the user's first input information in the input box 210, a first output corresponding to the first input information may be generated.

[0070] In some implementations, first input information from the user at the input box 210 may be obtained, and then the user's intention may be determined based on the first input information. For example, a corresponding query may be determined based on the user's intention, and a first output may be obtained based on the corresponding query.

[0071] In some embodiments, the user's intention may represent the interaction scenario that the user expects. For example, the interaction scenario that the user expects is to perform a local query on a local file or a local application, for example, the user's intention is a local search. For example, the interaction scenario that the user expects is to perform a query using at least one search engine, for example, the user's intention is an Internet search. For example, the interaction scenario that the user expects is to perform a large language model query using a specific desktop application associated with the launcher, for example, the user's intention is to solve the demand based on the ability of the large model.

[0072] In some implementations, the user intent may be determined based on the user's first input information, for example, the scenario of the user intent is local search, search engine search, or solving a demand based on the capabilities of a large model. Further, the scenario of the user intent may be applied, and the output corresponding to the scenario may be presented (or presented preferentially) to the user. For example, if the determined user intent is to solve a demand based on the capabilities of a large model, further, one or more matching generation functions may be determined based on the first input information, and then the determined one or more matching generation functions may be used to obtain the corresponding output.

[0073] For example, if the determined user intention is a local search, then a first output corresponding to the first input information can be found by querying a local file or a local application. For example, the first output can include a link to a local target file or a target application. For example, when displaying the first output, multiple generation functions can continue to be displayed to facilitate subsequent operations by the user.

[0074] Fig. 6A An embodiment of a local search 600 is shown. A user may enter first input information 610, such as "terminal" in an input box. As shown, local files and / or local applications found through local search may be displayed in a first area 612, and multiple generated functions may be further displayed in a second area 614.

[0075] The first output displayed in the first area 612 may include multiple items. For example, a predetermined number (such as 4) of items with higher matching degrees may be displayed, while other items are folded and displayed (such as through "more applications and files"). For example, if the user clicks "more applications and files", the local file search page of the system may be opened.

[0076] It is understandable that the first output may include a local target application, so that the user can start the target application through an operation such as clicking.

[0077] It is understandable that the first output may include a local target file, which may have different formats, such as pdf, docx, xlsx, pptx, csv, etc. Optionally, for the target file, the user can start (such as in its format) the target file by operations such as clicking. Optionally, for the target file, additional opening methods may be provided, such as AI reading companion, AI question and answer, etc.

[0078] Reference Figure 6B For target files in pdf format, two opening methods can be provided: default application opening and AI companion reading 620. Figure 6C For target files in xlsx format, two opening methods can be provided: default application opening and AI Q&A 630 opening.

[0079] Exemplarily, if the determined user intention is to solve the demand based on the ability of the large model, then one or more matching generation functions can be further determined based on the first input information. Subsequently, one or more matching generation functions can be used to obtain a first output corresponding to the first input information. As an example, assuming that the first input information is "Translate this sentence into English: What is the largest octopus?" At this time, by parsing the first input information, it can be determined that the user intention is to solve the demand based on the ability of the large model, the matching generation function is "translation", and the target language is English. Then, it can be matched with Figure 4 Similarly, the first output, i.e., the corresponding English translation, is displayed. As another example, assuming that the first input information is "provide summary information of the local PDF "Terminal Usage Tips"," at this time, the first input information can be parsed to determine that the user's intention is to solve the needs based on the capabilities of the large model, and the matching generation functions include "upload file" and "document summary". Subsequently, the electronic device can obtain the target file "Terminal Usage Tips" in PDF format from the local file, and use the document summary generation function to obtain the first output, for example, in Figure 5B or Figure 5C A summary of the target file is displayed on a similar interface.

[0080] Exemplarily, if the determined user intention is a search engine search, then the search engine can be called to determine the first output corresponding to the first input information. As an example, suppose the first input information is "What is the address and phone number of Restaurant A?" At this time, the user intention can be determined to be a search engine search by parsing the first input information. Then, the search engine can be used to search and obtain the corresponding first output. As an example, the first output can be a web page, and optionally, it can jump directly to the search engine and display the web page, or optionally, a link to the web page can be displayed. As an example, the first output can be the content in the web page, such as the corresponding answer.

[0081] In some implementations, first input information from the user at the input box 210 can be obtained, and based on the first input information, two or more of the following can be performed simultaneously: local query, query using at least one search engine, or large language model query using a specific desktop application; further, a first output can be obtained based on the query result.

[0082] Exemplarily, the first output includes multiple entries, for example, one or more entries based on a local query, one or more entries based on a search engine, and / or one or more entries based on a large language model. In some examples, the multiple entries can be divided into multiple parts: a first part based on a local query, a second part based on a large language model query, and / or a third part based on a query using a search engine.

[0083] Exemplarily, multiple entries can be arranged and displayed in order of priority. For example, one or more entries related to user intent have a high priority. For example, one or more entries based on local queries have a high priority. For example, at least one entry associated with a running application before the launcher is invoked has a high priority. As an example, when displaying the running interface of a javascript, the user invokes the launcher interface through a wake-up operation, and the user's first input information involves the meaning of a while statement, then the meaning and application of the while statement in javascript in the first output can be displayed with priority.

[0084] Exemplarily, multiple entries may be arranged in order based on the location of the query. For example, the first part based on the local query may be displayed in the first area, the second part based on the large language model query may be displayed in the second area, and the third part based on the query using the search engine may be displayed in the third area. For example, the first area is located above the second area and the third area, and the second area is located above the third area. However, it should be understood that other arrangements are also applicable and are not listed here. Exemplarily, depending on the size limit of the display area, the number of entries presented in the first area, the second area, and the third area may not exceed the corresponding threshold. For example, each area applies the same threshold, such as 4 or 5. For example, different areas apply different thresholds, such as a threshold of 5 for the first area and a threshold of 4 for the second area.

[0085] Optionally, a region identification may be displayed at the top of each region. Optionally, the region identification of the region located at the top may be omitted. Optionally, depending on the number of entries, part or all of the first part, the second part, and the third part may be displayed.

[0086] Figure 7 A schematic diagram of the first output 700 is shown. For the first input information "What is the largest octopus?", regions 710 and 720 may be displayed, wherein region 710 displays the second part based on the query using the large language model, and region 720 displays the third part based on the query using the search engine, and region 720 is located below region 710. As shown in the figure, a region identifier 722 of region 720 is also displayed, such as "search engine search". Figure 7 In the example, since there is no query result of the local query, the first output does not include the first part based on the local query.

[0087] For example, for the second part of the query based on the large language model, the user can select or click an item in it to activate it. Figure 7 The user can select "AI Search: What is the largest octopus?" in area 710 and click Enter to view the answer, such as the answer Figure 3B or Figure 3C shown.

[0088] For example, for the third part based on the query using the search engine, recommended / suggested information can also be displayed at the same time. For example, the hit keyword can be highlighted or emphasized, while the recommended / suggested information is not highlighted. Figure 7 , recommended or suggested information may include "in the Pacific Ocean", "discovered by humans", etc.

[0089] As discussed above, in some scenarios, a large language model can be used to generate a first output corresponding to the first input information, for example, the first output can include one or more items obtained using the large language model. Specifically, if the user intent in response is to solve the demand based on the ability of the large model, the large language model can be used to obtain the first output; or, the large language model can be used to obtain the second part of the first output based on the large language model query.

[0090] In some examples, generating a first output based on a large language model may include generating a first output based on retrieval-augmented generation (RAG). Exemplarily, a user may establish a personal database when using a specific desktop application (such as Name-XX), such as a RAG database called Name-XX. For example, a user may establish a personal database by means of collection, bookmarking, etc. For example, a personal database may include documents uploaded by a user, content collected by a user after viewing through other channels, etc. It is understandable that the embodiments of the present disclosure do not limit the format, size, attributes, etc. of the content in the personal database. For example, the content may be in the format of text (such as files / documents), in the format of multimedia content (such as images / audio / video), in the format of an application, in the format of a code, and so on.

[0091] Exemplarily, the first input information and the content in the personal database can be used as part of the input data of the large language model to obtain the first output. For example, the input data can include the content in the personal database associated with the first input information, for example, the first input information can be a prompt. For example, the input data can include background knowledge pre-learned by the large language model, so that the first output can be more accurate, for example, the first output that better meets the user's needs can be obtained.

[0092] As an example, learning can be performed based on RAG to obtain learning content, and then a large language model can be used to obtain a first output based on the learning content and the first input information. As another example, the content associated with the first input information in a personal database can be obtained, and then the first output can be obtained based on RAG, according to the content associated with the first input information in the personal database and the first input information. As another example, the content associated with the first input information in the personal database can be obtained, and then the first output can be obtained based on the first input information. For example, the first output may include part or all of the content associated with the first input information in the personal database, for example, the first output includes part of the content associated with the first input information in the personal database that has a strong correlation with the first input information.

[0093] For example, the personal database includes a sales contract template, and the first input information indicates that the content of Chapter 8 of the sales contract needs to be generated. In one example, the output may include the original text of the content of Chapter 8 in the sales contract template in the personal database. In another example, the original text of the content of Chapter 8 in the sales contract template in the personal database may be revised, for example, updated based on the sales scenario indicated by the first input information, and the output may include the updated content after the original text of the content of Chapter 8 in the sales contract template is revised.

[0094] Through the exemplary embodiments of the present disclosure, the launcher is associated with a specific desktop application, and can generate AI based on the large language model. At the same time, the launcher is both a search entry for local queries and a search engine. In this way, the launcher and the operating system are combined, the large language model is connected with the operating system and the search engine, and the entry is integrated into the input box of the launcher. The user does not need to predetermine the location to be queried, and the interaction with the user can be quickly and comprehensively realized, thereby improving the user experience.

[0095] It should be understood that in the embodiments of the present disclosure, "first", "second", "third", etc. are only used to indicate that multiple objects may be different, but at the same time do not exclude that two objects are the same, and should not be interpreted as any limitation on the embodiments of the present disclosure.

[0096] It should also be understood that the methods, situations, categories and divisions of the embodiments in the present disclosure are only for the convenience of description and should not constitute special limitations. The features in various methods, categories, situations and embodiments can be combined with each other when it is logical.

[0097] It should also be understood that the above content is only to help those skilled in the art better understand the embodiments of the present disclosure, rather than to limit the scope of the embodiments of the present disclosure. Those skilled in the art may make various modifications, changes or combinations based on the above content. Such modifications, changes or combinations are also within the scope of the embodiments of the present disclosure.

[0098] It should also be understood that the description of the above content focuses on emphasizing the differences between the various embodiments, and the same or similar points can be referenced or borrowed from each other. For the sake of brevity, they will not be repeated here.

[0099] Figure 8 A schematic block diagram of an example device 800 according to some embodiments of the present disclosure is shown. The device 800 can be implemented by software, hardware, or a combination of both. In some embodiments, the device 800 can be implemented as an electronic device. In the embodiments of the present disclosure, the electronic device can be a desktop computer, a tablet computer, etc., which is not limited by the present disclosure.

[0100] like Figure 8 As shown, the device 800 includes a presentation module 810 and a generation module 820. The presentation module 810 is configured to present a launcher interface for interacting with the user in response to a wake-up operation from the user, while displaying the interface of the desktop or the running application, wherein the launcher interface includes an input box and multiple generation functions. The generation module 820 is configured to generate an output corresponding to the input based on the input from the user in the input box.

[0101] In some embodiments, the wake-up operation includes any of the following: a user triggering a shortcut key associated with the launcher, a user inputting a voice for waking up the launcher, a user's specific gesture, or another predefined operation for waking up the launcher. In some embodiments, the interface of the launcher has a predetermined size, and the plurality of generated functions are folded or scrolled based on the predetermined size.

[0102] In some embodiments, the running application includes any application running on an operating system of the computer.

[0103] Exemplarily, the device 800 may further include an acquisition module configured to acquire a user's trigger operation for a first function among the multiple generated functions. Further, the presentation module 810 may be configured to present an interface associated with the first function in response to the trigger operation, wherein the interface associated with the first function includes an input box and prompt information associated with the first function. For example, the prompt information is displayed in the input box as a placeholder.

[0104] Exemplarily, the apparatus 800 may further include an acquisition module configured to acquire first input information from a user in an input box. Further, the generation module 820 is configured to perform a query based on the first input information to generate a first output corresponding to the first input information, wherein the query includes at least one of the following: a local query of a local file or a local application in the computer, a query using a large language model of a specific desktop application associated with the launcher, or a query using at least one search engine.

[0105] In some examples, the apparatus 800 may include a determination module configured to determine a user intent based on the first input information; and determine a corresponding query based on the user intent.

[0106] In some examples, the specific desktop application includes a personal database, and the generation module 820 is configured to use the first input information and the content of the personal database as partial input data of the large language model to obtain a first output.

[0107] In some examples, the first output includes a plurality of entries, and the plurality of entries are arranged in order based on a location of the query.

[0108] In some examples, the first output includes a plurality of entries, and the plurality of entries includes at least a first portion based on a local query, a second portion based on a large language model query, and a third portion based on a query utilizing at least one search engine.

[0109] In some examples, the presentation module 810 is further configured to display the first output in at least one of the following ways: displaying the first output in the interface of the launcher; or calling a specific desktop application to display the first output in the desktop application. For example, at least one of the following may also be displayed in association with the first output: a copy button, a regenerate button, or a cancel button.

[0110] In some examples, the presentation module 810 is configured to preferentially display at least one entry in the first output that is associated with a running application.

[0111] In some embodiments, the specific desktop application is implemented as at least one of the following: a dedicated browser application, a specific plug-in in the at least one search engine, or a specific web page.

[0112] Figure 8 The device 800 can be used to achieve the above combination Figure 1 For the sake of brevity, the method 100 is not described in detail here.

[0113] The division of modules or units in the embodiments of the present disclosure is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in the disclosed embodiments may be integrated into one unit, or may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] Fig. 9 900 is a block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure. It should be understood that Fig. 9 The device 900 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the implementation described herein. For example, the device 900 can be used to perform the method 100 described above.

[0115] like Fig. 9As shown, the device 900 is in the form of a general-purpose computing device. The components of the computing device 900 may include, but are not limited to, one or more processors or processing units 910, a memory 920, a storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. The processing unit 910 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 920. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to increase the parallel processing capabilities of the computing device 900.

[0116] The computing device 900 typically includes a plurality of computer storage media. Such media may be any available media accessible to the computing device 900, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 920 may be a volatile memory (e.g., register, cache, random access memory (Random Access Memory, RAM)), a non-volatile memory (e.g., read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read Only Memory, EEPROM), flash memory) or some combination thereof. The storage device 930 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the computing device 900.

[0117] The computing device 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Fig. 9 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. Memory 920 may include a computer program product 925 having one or more program modules that are configured to perform various methods or actions of various implementations of the present disclosure.

[0118] The communication unit 940 implements communication with other computing devices through a communication medium. Additionally, the functions of the components of the computing device 900 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the computing device 900 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0119] Input device 950 may be one or more input devices, such as a mouse, keyboard, tracking ball, etc. Output device 960 may be one or more output devices, such as a display, a speaker, a printer, etc. Computing device 900 may also communicate with one or more external devices (not shown) through communication unit 940 as needed, such as storage devices, display devices, etc., communicate with one or more devices that allow users to interact with computing device 900, or communicate with any device that allows computing device 900 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0120] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, such as a non-transitory computer-readable storage medium, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, the computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0121] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0122] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0123] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0124] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0125] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A computer-implemented method comprising: In the case of displaying the interface of a desktop or a running application, in response to a wake-up operation from a user, presenting a launcher interface for interacting with the user, wherein the launcher interface includes an input box and a plurality of generation functions; and Based on the input from the user in the input box, an output corresponding to the input is generated.

2. The method according to claim 1, wherein the wake-up operation comprises: The user triggers a shortcut key associated with the launcher, The user input is used to invoke the voice of the launcher, a specific gesture of the user, or Another predefined operation for invoking the launcher. The method according to claim 1 , wherein the running application comprises any application running on an operating system of the computer. 4 . The method according to claim 1 , wherein the interface of the launcher has a predetermined size, and wherein the plurality of generating functions are displayed in a folded state or in a scrolled state based on the predetermined size.

5. The method according to claim 1, further comprising: Acquire a trigger operation of the user on a first function among the multiple generation functions; as well as In response to the trigger operation, an interface associated with the first function is presented, wherein the interface associated with the first function includes the input box and prompt information associated with the first function. The method according to claim 5 , wherein the prompt information is displayed in the input box as a placeholder.

7. The method according to claim 1, further comprising: Acquire first input information from the user in the input box; as well as Performing a query based on the first input information to generate a first output corresponding to the first input information, wherein the query includes at least one of the following: local query of local files or local applications in the computer, querying using a large language model of a specific desktop application associated with the launcher, or A query is performed using at least one search engine.

8. The method according to claim 7, further comprising: determining a user intention based on the first input information; as well as The corresponding query is determined based on the user intent.

9. The method of claim 7, wherein the specific desktop application comprises a personal database, and wherein generating the first output comprises: The first input information and the content of the personal database are used as partial input data of the large language model to obtain the first output.

10. The method of claim 7, wherein the first output comprises a plurality of entries, and the plurality of entries are arranged in order based on a location of a query.

11. The method of claim 7, wherein the first output comprises a plurality of entries, and the plurality of entries comprises at least a portion of: Based on the first part of the local query, a second part based on the large language model query, and A third portion is based on a query utilizing the at least one search engine.

12. The method of claim 7, further comprising displaying the first output in at least one of the following ways: displaying the first output in the interface of the launcher; or The specific desktop application is called to display the first output in the desktop application.

13. The method according to claim 12, further comprising: At least one of the following is displayed in association with the first output: a copy button, a regenerate button, or a cancel button.

14. The method according to claim 7, further comprising: At least one item in the first output that is associated with the running application is preferentially displayed.

15. The method according to claim 7, wherein the specific desktop application is implemented as at least one of the following: a dedicated browser application, a specific plug-in in the at least one search engine, or a specific web page.

16. An electronic device, comprising: at least one processor; as well as At least one memory having computer executable instructions stored thereon, the computer executable instructions, when executed by the at least one processor, causing the electronic device to implement the method according to any one of claims 1 to 15. 17 . A non-transitory computer-readable storage medium having computer-executable instructions stored thereon, wherein when the computer-executable instructions are executed by a processor, the method according to claim 1 is implemented.

18. A computer program product, comprising computer executable instructions, which when executed, implement the method according to any one of claims 1 to 15.

Images