Electronic equipment, information processing method thereof and medium

Abstract

The invention relates to the technical field of application programs of intelligent terminals, and relates to electronic equipment, an information processing method thereof and a medium, and the method comprises the steps: displaying a first interface comprising to-be-processed information, displaying a sharing interface comprising an icon of an intelligent assistant, and displaying a second interface comprising a control related to the input of a user intention and the to-be-processed information, and in response to the detected input of the user intention, displaying a processing result corresponding to the user intention. It can be seen that through the information processing method, the second interface of the intelligent assistant is aroused on the first interface; then, the to-be-processed information is processed through the intelligent assistant according to the intention of the user, that is, the processing result of the to-be-processed information can be obtained on the first interface, the continuity of processing the to-be-processed information between the application program and the intelligent assistant is improved, and the user can more conveniently use the intelligent assistant in the application program.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent terminals, and in particular to an electronic device and an information processing method and medium thereof. Background Art

[0002] With the development of electronic devices, users use electronic devices to process information more and more frequently, such as processing photos, editing documents, etc. The operating system of the electronic device also provides an intelligent assistant to provide auxiliary functions to the user in the process of using the electronic device to process information. For example, the intelligent assistant can retrieve photos and documents stored in the storage space of the electronic device, quickly open the search engine for query according to the content input by the user, etc.

[0003] As artificial intelligence (AI) technology matures, users can process information more accurately and efficiently using AI technology. Currently, most AI technology usage methods require opening an application corresponding to the AI ​​technology to initiate use. For example, if a user needs to use an AI question-and-answer tool to process information, he can search for the URL corresponding to the AI ​​question-and-answer tool through an intelligent assistant, and then open the web page corresponding to the AI ​​question-and-answer tool based on the search results and enter the information for use to obtain the processing results corresponding to the information. It can be seen that if a user needs to further process information with the help of AI technology while using an electronic device to process information, it is necessary to open additional applications under the operating system of the electronic device, such as a browser, which causes the user to switch between multiple applications and lacks interactive continuity. Summary of the invention

[0004] The present application provides an electronic device and an information processing method and medium thereof.

[0005] In a first aspect, an embodiment of the present application provides an information processing method, which is applied to an electronic device, and the method includes:

[0006] Displaying a first interface, wherein the first interface includes information to be processed;

[0007] In response to detecting an operation that triggers a sharing interface, displaying a sharing interface, the sharing interface including icons of multiple applications, the icons of the multiple applications including an icon of a smart assistant;

[0008] Detecting an operation of triggering the icon of the intelligent assistant, displaying a second interface, wherein the second interface includes controls and information to be processed related to the input of the user's intention;

[0009] In response to detecting input of user intention, a processing result corresponding to the user intention is displayed, wherein the user intention is used to instruct the intelligent assistant to process the information to be processed.

[0010] In this specification, an electronic device may be any terminal device. The first interface may be an application interface of an application or service installed on the electronic device, or an application interface of an application or service not installed on the electronic device retrieved through a quick service center. For example: the application interface of a photo application. The information to be processed here may be information data corresponding to the application or service to which the first interface belongs, such as: the corresponding first interface is an application interface of a photo application, and the information to be processed may be an image; the corresponding first interface is a text editing application, and the information to be processed may be text. The sharing interface here may be used by the user to select the sharing object or sharing method of the information to be processed, and the icon of the application here may be the sharing object or sharing method corresponding to the information to be processed, for example: sharing the information to be processed to contacts in an instant messaging application or sharing the information to be processed to an email application. The second interface here may be an application interface of an intelligent assistant, and the application interface is used for the user to input the user's intention, and the intelligent assistant obtains the processing result corresponding to the information to be processed according to the user's intention, that is, the user command.

[0011] It can be seen that through the above information processing method, an interactive mode of invoking the intelligent assistant is set for the first interface. By invoking the second interface of the intelligent assistant on the first interface, the second interface can be displayed on the first interface; then, the intelligent assistant processes the information to be processed according to the user's intention, that is, the processing result of the information to be processed can be obtained on the first interface, increasing the consistency of processing the information to be processed between the application and the intelligent assistant. By invoking the intelligent assistant in the application to call the application / application service of AI technology based on the information to be processed and the user's intention, processing the information to be processed, and displaying the processing result on the application interface of the intelligent assistant, users can use the intelligent assistant in the application more conveniently.

[0012] In a possible implementation of the first aspect, in response to detecting an operation that triggers the sharing interface, displaying the sharing interface includes:

[0013] In response to detecting a first operation on a first control in the first interface, displaying a first operation interface;

[0014] In response to detecting a first operation on a second control in the first operation interface, a sharing interface is displayed, wherein the second control includes a sharing control.

[0015] In this specification, the first control here may be more controls. Depending on the type and / or amount of information to be processed, the first operation interface may include a sharing control, a renaming control, etc.

[0016] In a possible implementation of the first aspect, in response to detecting an operation that triggers the sharing interface, displaying the sharing interface includes:

[0017] In response to detecting a second operation on the information to be processed, displaying a sharing interface.

[0018] In this specification, the second operation here can be a click operation, a long press operation or a right click operation on the information to be processed.

[0019] In a possible implementation of the first aspect, displaying the sharing interface includes displaying the sharing interface on the first interface in at least one of a window form, a pop-up menu, and a semi-modal suspension form; or

[0020] Displaying the second interface includes displaying the second interface on the first interface in at least one of a window form, a pop-up menu, and a semi-modal suspension form.

[0021] In this specification, the sharing interface and the second interface of the smart assistant can be displayed on the first interface, such as by overlaying the first interface, which increases the coherence of the interaction between the first interface, the sharing interface and the second interface.

[0022] In a possible implementation of the first aspect above, controls related to the input of user intention include a user intention input box and / or a user intention prompt box, wherein the user intention input box is used for user input instructions, and the user intention prompt box includes user intention generated based on the type or content of the information to be processed, or generated based on the user's historical behavior.

[0023] In a possible implementation of the first aspect above, the input of the user intention includes an instruction input operation for a user intention input box or a selection operation for a user intention prompt box.

[0024] In this specification, the user intention prompt box here may include a bubble prompt instruction control. The user intention prompt box may correspond to the type of information to be processed. The smart assistant may pre-generate at least one user intention based on the type or content of the information to be processed, and / or the user's historical behavior, and display it on the second interface of the smart assistant in the form of a user intention prompt box to provide user selection, thereby avoiding the need for the user to enter the user intention every time the smart assistant is used.

[0025] In a possible implementation of the first aspect, displaying a processing result corresponding to the user intention includes:

[0026] The processing result is displayed on the first interface, or the second interface, or another interface different from the first interface or the second interface.

[0027] In this specification, the processing results can be directly displayed on the second interface of the smart assistant, or directly returned to the first interface, or directly sent to the application interface of the third-party application.

[0028] In a possible implementation of the first aspect, the present invention further includes:

[0029] determining a second application from a plurality of generative AI applications based on a type of information to be processed;

[0030] Based on the information to be processed and the input of the user's intention, a second application is used to obtain a processing result.

[0031] In this specification, the second application here can be a generative AI application determined according to the type of information to be processed.

[0032] In a possible implementation of the first aspect above, corresponding to the information to be processed being an image, processing the information to be processed includes at least one of the following: beautifying, cutting out, or coding the image.

[0033] In a possible implementation of the first aspect, corresponding to the information to be processed being text, processing the information to be processed includes at least one of the following: translating the text or extracting key points.

[0034] In this specification, the intelligent assistant can process different pending information according to the user's intention.

[0035] In a second aspect, an embodiment of the present application provides an electronic device, including:

[0036] a memory for storing instructions to be executed by one or more processors of the electronic device, and

[0037] A processor is used to read the instruction to execute the information processing method under the first aspect or any one of the implementation modes.

[0038] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the information processing method in the first aspect or any one of the implementation modes.

[0039] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising: a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium containing a computer program code for executing the information processing method under the first aspect or any one of the implementation modes.

[0040] The beneficial effects of the above aspects can be referenced to each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1(a) to Figure 1(d) The schematic diagram of the scenario in which a user uses a mobile phone to process information provided in this specification is exemplarily shown;

[0042] Figure 2(a) to Figure 2(e) A schematic diagram exemplarily shows a scenario to which the information processing method in this specification is applicable;

[0043] Figure 3 A schematic diagram of a method flow of an information processing method in this specification is exemplarily shown;

[0044] Figure 4(a) to Figure 4(e) A schematic diagram exemplarily shows a scenario to which the information processing method in this specification is applicable;

[0045] Figure 5(a) to Figure 5(e) A schematic diagram of a scenario in which a method for processing information in which a plurality of target information types are photos is applicable is exemplarily shown in this specification;

[0046] Figure 6(a) to Figure 6(f) The schematic diagram of the scenario in which the controls of the smart assistant are arranged on top in this specification is exemplarily shown;

[0047] Figure 7(a) to Figure 7(c) A schematic diagram of a scenario in which a user uses a personal computer to perform an information processing method on target information in this specification is exemplarily shown;

[0048] Figure 8(a) to Figure 8(c) A schematic diagram of a scenario in which a user uses a personal computer to perform an information processing method on target information in this specification is exemplarily shown;

[0049] Fig. 9 The structural diagram of the electronic device in this specification is exemplarily shown;

[0050] Fig.10 The software structure block diagram of the electronic device in this specification is exemplarily shown. DETAILED DESCRIPTION

[0051] The technical solutions in this specification will be described clearly and in detail below in conjunction with the accompanying drawings. The following embodiments can be referenced to each other.

[0052] It can be understood that the technical method of the present application is applicable to any electronic device, which can specifically be a non-folding screen mobile phone, a folding screen mobile phone, a wearable device (such as a smart watch, a smart bracelet, etc.), a tablet computer, a laptop computer, a smart screen, a car terminal, a computer, a personal computer (personal computer, PC), an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), a virtual reality (virtual reality, VR) device / augmented reality (augmented reality, AR) device, an artificial intelligence (artificial intelligence, AI) device, and other terminal devices with display functions. The operating system installed in the above electronic device 100 includes but is not limited to Or other operating systems. This application does not limit the specific type of the electronic device 100 and the installed operating system.

[0053] Figure 1(a) to Figure 1(d) The schematic diagram of a scenario in which a user uses a mobile phone 100 to process information provided in this specification is exemplarily shown.

[0054] As shown in FIG. 1( a ), the screen of the mobile phone 100 displays an application interface 101 of a photo application. The application interface 101 of the photo application here can be a photo overview interface. The application interface 101 of the photo application includes multiple photos and a target photo 1011, that is, the photo selected by the user by clicking. The user can perform a user operation on the side power button 102 of the mobile phone 100, such as: a long press operation. In response to the user operation, as shown in FIG. 1( b ), the application interface 101 of the photo application on the screen of the mobile phone 100 displays an application interface 103 of an intelligent assistant (also referred to as an AI assistant), such as: an intelligent assistant "Xiao Yi". That is, the application interface 103 of the intelligent assistant is called up in the application interface 101 of the photo application. The user can enter a user command (also referred to as a user intention) in the application interface 103 of the intelligent assistant, such as: "Cut out the person in the photo and copy it to the memo". In response to the user command, as shown in FIG. 1( c ), the intelligent assistant opens the search engine application to query based on the user command, and displays the search results in the application interface 104 of the search engine application, wherein the search results may include multiple AI technology websites corresponding to the user command, that is, AI technology websites corresponding to the cutout. Further, referring to FIG1(c), the user can determine the AI ​​technology website to be used from the search results and open it, such as the AI ​​technology website of the cutout tool, as shown in FIG1(d). Through the AI ​​technology website of the cutout tool, the user can upload a photo and click the "one-key cutout" button to obtain the corresponding processing result. The smart assistant here can be a system application or a third-party application configured by the mobile phone 100 that has a search processing function and calls a third-party application.

[0055] It can be seen that when the user uses the photo application of the mobile phone 100 to process the target photo 1011, if the user is ready to use the AI ​​technology application with the help of the smart assistant, the user can search for multiple AI technology applications corresponding to the user command through the smart assistant, and then select the AI ​​technology application to be used to obtain the processing result of the target photo 1011. In this process, the user needs to manually select the required AI technology application, and the mobile phone 100 also needs to switch between the application interface of the photo application and the application interface of the AI ​​technology application, that is, the user cannot directly obtain the processing result of the target photo 1011 corresponding to the user command by calling the smart assistant. Therefore, the usage connection between the smart assistant and the AI ​​technology application is weak. If the user wants to use the AI ​​technology application through the smart assistant, he needs to switch between multiple applications, and lacks interactive coherence.

[0056] In order to solve the incoherent problem caused by the application using AI technology through the intelligent assistant in the process of the above-mentioned user using the application, this specification provides an information processing method. Through this information processing method, in the process of the user using the application of the electronic device, if the user wants to process the target information (such as photos, documents, etc.), the user can wake up the intelligent assistant. In the application interface of the intelligent assistant, the intelligent assistant can respond to the user command (i.e., AI instruction) for the target information, and call / access the corresponding AI technology application to process the target information according to the type of target information and the content of the user command, and obtain the processing result corresponding to the target information based on the user command and prompt the above processing result through the application interface of the intelligent assistant. That is, the intelligent assistant's wake-up control can be added to the application interface of the application of the electronic device, and the user command can be sent to the intelligent assistant. After the intelligent assistant obtains the processing result through the application of AI technology, it is directly displayed in the application interface of the application.

[0057] In this specification, after the user uses an electronic device to wake up the smart assistant, the application interface of the smart assistant can be displayed superimposed on the application interface of the current application. The superimposed display method may include: opening a new application interface of the smart assistant, displaying the application interface of the smart assistant in the form of a semi-modal window or a non-modal window suspended on the application interface of the current application, or jumping from the application interface of the current application to the application interface of the smart assistant, and returning to the application interface of the current application after the user obtains the processing result. Among them, the application interface of the smart assistant corresponding to the non-modal window indicates that the user is allowed to interact with the application interface of the smart assistant and the application interface of the current application at the same time; the application interface of the smart assistant in the semi-modal window indicates that only the user can interact with the application interface of the smart assistant.

[0058] It can be seen that, through the information processing method in this specification, an interactive method for invoking the intelligent assistant is set in the application interface of the application of the electronic device, by invoking the application interface of the intelligent assistant on the application interface of the application; then, processing the user command through the application interface of the intelligent assistant to increase the consistency of the interaction between the application and the intelligent assistant, by invoking the intelligent assistant in the application to call the application / application service of AI technology based on the user command to process the target information according to the user command, and displaying the processing result in the application interface of the intelligent assistant, so that the user can use the intelligent assistant in the application more conveniently.

[0059] This application takes the voice intelligent assistant Xiaoyi as an example. It is understandable that other voice intelligent assistants or non-voice intelligent assistants can also apply this application. It should be understood that the appearance of the display interface shown in the drawings of this application is only an example and is not intended to limit the method provided in this application.

[0060] Below through Figure 2(a) to Figure 2(e) The scenarios to which the information processing method in this specification is applicable are further described by way of example.

[0061] As shown in FIG. 2( a ), the screen of the mobile phone 100 displays an application interface 201 of a photo application. The application interface 201 of the photo application includes a plurality of photos and a target photo 2011. The user can perform a user operation on the “more” control 202 in the application interface 201, such as a click operation. In response to the user operation, as shown in FIG. 2( b ), the application interface 201 of the photo application in the screen of the mobile phone 100 displays an operation interface 203, and the operation interface 203 includes a “share” control 2031 and the like. The user can perform a user operation on the “share” control 2031 in the operation interface 203, such as a click operation. In response to the user operation, as shown in FIG. 2( c ), an application interface 204 of a sharing application is displayed on the application interface 201 of the photo application. The application interface 204 (sharing interface) of the sharing application on the application interface 201 of the photo application can be displayed in a pop-up window (window form), a pop-up menu, a semi-modal suspension, and the like. The application interface 204 of the sharing application includes controls of multiple shareable devices and controls of shareable applications, etc., wherein the controls of the shareable applications include: a control 205 of the smart assistant "Xiao Yi". The user can perform a user operation on the control 205 of the smart assistant "Xiao Yi", such as a click operation. In response to the user operation, as shown in FIG2(d), the application interface 206 of the smart assistant "Xiao Yi" (i.e., the smart assistant) can be displayed in a semi-modal suspended form on the application interface 201 of the photo application, that is, the application interface 206 of the smart assistant is evoked in the application interface 201 of the photo application. Continuing to refer to FIG2(d), the user can enter a user command in the user command input box 2062 of the application interface 206 of the smart assistant, such as: "Cut out the person in the photo". In response to the user command, the smart assistant can call the application / application service of the AI ​​technology to process the target photo 211 according to the user command. As shown in FIG. 2( e ), the application interface 206 of the smart assistant displays the processing result of the target photo 2011 that has completed the user command, that is, the result photo 2063 in which the person is cut out from the target photo 2011 .

[0062] In some embodiments, the application interface 206 of the smart assistant "Xiaoyi" shown in FIG. 2(d) may also display a bubble prompt command control 2061 corresponding to the target photo 2011, such as: "Automatic beautification", "Cutout", "Face coding", etc. The bubble prompt command control here may correspond to the type of target information. The smart assistant may pre-generate at least one default commonly used user command based on the type of target information and / or user operation habits, and display it in the application interface of the smart assistant in the form of a bubble prompt command control to provide users with choices, so as to avoid the need for users to enter user commands every time they use the smart assistant to process target information. For example, if the target information is a text file, the bubble prompt command may include: "Extract key points" and "Translate into Chinese", etc.

[0063] In another embodiment, a control for sharing or sending to "Xiao Yi" may be included in the user interface shown in FIG2(b). Exemplarily, a control "Share to Xiao Yi" may be added under the "Share" control 2031, and the user clicks the "Share to Xiao Yi" control, and the electronic device displays the application interface of the smart assistant in response to the click operation (see FIG2(d) 206). Other embodiments of the present application may also refer to this implementation.

[0064] Combine the following Figure 3 The method flow chart shown is used to describe the information processing method of this specification. Figure 3 The method shown can be implemented by a processor of a terminal device (taking mobile phone 100 as an example) executing relevant instructions.

[0065] refer to Figure 3 , the information processing method may include the following steps:

[0066] S301: The mobile phone 100 runs a first application and displays a first application interface of the first application.

[0067] Exemplarily, the first application here may be the photo application shown in FIG. 2( a ), and the first application interface may be the application interface 201 of the photo application.

[0068] S302: The mobile phone 100 determines first target information in a first application interface of a first application.

[0069] For example, referring to FIG. 2( a ), the first target information here may be the first application interface of the first application program, that is, the target photo 2011 displayed in the application interface 201 of the photo application. The user may click on the target photo 2011 in the application interface 201 of the photo application to select the target photo 2011.

[0070] S303: In response to the user operation, the mobile phone 100 invokes the smart assistant and displays the second application interface of the smart assistant.

[0071] Exemplarily, the user operation here can be that the user performs a user operation on the first application interface, that is, the "more" control 202 in the application interface 201, such as a click operation. In response to the user operation, the application interface 101 displays the operation interface 203. Then, the user performs a click operation on the "share" control 2031 in the operation interface 203. As shown in FIG2(c), the application interface 201 displays the application interface 204 of the sharing application. The application interface 204 includes a control 205 of the smart assistant "Xiao Yi". Finally, the user performs a click operation on the control 205 of the smart assistant "Xiao Yi". As shown in FIG2(d), the application interface 201 of the photo application displays the application interface 206 of the smart assistant "Xiao Yi", that is, the second application interface of the smart assistant.

[0072] In this specification, the mobile phone 100 can control the second application interface to be displayed in a partial display manner in the first application interface, that is, the second application interface can be located in a partial area of ​​the first application interface, and the partial area corresponding to the second application interface overlays / covers the first application interface. Alternatively, in this specification, the second application interface can also be displayed on the first application interface in a full-screen overlay manner, that is, the second application interface is opened on the first application interface, and after the user obtains the processing result, it returns to the first application interface.

[0073] In this specification, the method of invoking the smart assistant in the first application interface by the mobile phone 100 is not limited to the method of performing user operations on controls. For example, the mobile phone 100 can also invoke the smart assistant in the first application interface in response to user operations that the user can perform on the buttons of the mobile phone 100.

[0074] S304: The mobile phone 100 sends the first target information to the smart assistant.

[0075] Exemplarily, as shown in FIG. 2( d ), the target photo 2011 may be displayed in the application interface 206 of the smart assistant “Xiaoyi”, that is, the second application interface of the smart assistant.

[0076] It is understandable that the execution order of step S304 here is not necessarily after step S303. In this specification, when mobile phone 100 responds to user operation, wakes up the smart assistant and displays the second application interface, mobile phone 100 can send the first target information to the smart assistant.

[0077] S305: The mobile phone 100 displays a user command corresponding to the type of the first target information in the second application interface.

[0078] For example, referring to the application interface 206 in FIG. 2( d ), the second application interface may further display a bubble prompt instruction control 2031 corresponding to the first target information, i.e., the target photo 2011 , i.e., a user command, such as “automatically beautify”, etc. The user command here may correspond to the type of the first target information.

[0079] In other embodiments, step S305 here is not necessary, that is, in the second application interface of the smart assistant, the mobile phone 100 may not display the user command in the second application interface, but wait to receive the user command input by the user through voice or keyboard.

[0080] S306: The mobile phone 100 responds to the user command input by the user. According to the type of the first target information and the content of the user command, the intelligent assistant calls the corresponding AI technology application to process the first target information and obtain a processing result.

[0081] For example, referring to FIG. 2(d), the user command here may be a user command that the user may input in the second application interface, i.e., the application interface 206 of the intelligent assistant, such as "cut out the person in the photo" input by voice or other means, or an instruction input by selecting the bubble prompt control "automatically beautify". In response to the user command, the intelligent assistant may call an application program of AI technology, such as a photo processing AI tool, etc., to process the first target information based on the user command.

[0082] In this specification, the AI ​​technology application here can be an application that has been installed by the mobile phone 100, or an AI technology-related service subscribed by the mobile phone 100, or an AI technology-related service deployed on a server for access, such as a generative AI application. The intelligent assistant can determine the AI ​​technology application from multiple generative AI applications according to the type of the first target information, and obtain the processing result using the AI ​​technology application based on the first target information and the user command.

[0083] S307: The mobile phone 100 displays the processing result of the first target information in the second application interface.

[0084] Exemplarily, as shown in FIG. 2( e ), the second application interface, i.e., the application interface 206 of the smart assistant, may display the first target information that has executed the user command, i.e., the processing result of the target photo 2011 .

[0085] In other embodiments, step S307 here may not be necessary, that is, after the intelligent assistant completes processing of the first target information and obtains the processing result, the processing result of the first target information can be directly saved or sent to other applications, such as: saving the processing result in the storage space of the mobile phone 100 or directly sending the processing result to the instant messaging application, without displaying the processing result in the second application interface.

[0086] Through the above Figure 2(a) to Figure 2(e) as well as Figure 3 , introduces the applicable scenarios of the information processing method for target information type of photos involved in this specification, and the following will be Figure 4(a) to Figure 4(e) Describes scenarios in which information processing methods are applicable when the target information type is text.

[0087] Exemplarily, as shown in FIG4(a), the screen of the mobile phone 100 displays an application interface 401 of a folder application. The application interface 401 of the folder application includes multiple files and a target text file 4011. Exemplarily, the user can click on the "more" control 402 in the application interface 401, and as shown in FIG4(b), the application interface 401 in the screen of the mobile phone 100 displays an operation interface 403, and the operation interface 403 includes a "share" control 4031, etc. In other embodiments, the user can also trigger the display of the operation interface 403 by long pressing the target text file 4011, etc.

[0088] The user can click on the "Share" control 4031 in the operation interface 403, as shown in FIG4 (c), and the screen of the mobile phone 100 displays the application interface 404 of the sharing application. The application interface 404 of the sharing application can be displayed on top of the application interface 401, or in other display modes such as full screen display. The application interface 404 includes controls for multiple shareable devices and a control 405 of the smart assistant "Xiao Yi". The user can click on the control 405 of the smart assistant "Xiao Yi", as shown in FIG4 (d), and the application interface 406 of the smart assistant "Xiao Yi" is displayed on the application interface 401 of the photo application. Continuing to refer to FIG4 (d), the application interface 406 may include a default bubble prompt instruction control 4061, and the user can enter a user command in the application interface 406 of the smart assistant, such as: "Extract the main points of this article". In response to the user command, the smart assistant can call an application of AI technology, such as: a text processing AI tool, to process the target text file 4011 according to the user command. As shown in FIG. 4( e ), the application interface 406 of the intelligent assistant displays the processing result of the target text file 4011 that has completed the user command, that is, the key points 4062 are extracted from the target text file 4011 .

[0089] Optionally, the application interface 406 may be a dialogue interface that supports multiple rounds of dialogue, and the user may continue to have a dialogue with the intelligent assistant on the application interface 406 and continue to send instructions to the intelligent assistant through dialogue.

[0090] pass Figure 4(a) to Figure 4(e) The scenario where the information processing method is applicable to the target information type described as text, it can be seen that the intelligent assistant can provide different user commands, i.e., AI prompt words, for different types of target information. For example: if the target information is a document, the user command will be related to document processing, and if the target information is a photo, the user command will be related to photo processing, so that the intelligent assistant has better adaptability when processing the target information.

[0091] The following will be Figure 5(a) to Figure 5(e) The present invention describes a scenario in which a method for processing information of multiple target information at the same time, wherein the type of the target information is a photo, is applicable.

[0092] Exemplarily, as shown in FIG5(a), the screen of the mobile phone 100 displays an application interface 501 of a photo application, including multiple photos and selected target photos 5011 and target photos 5012. The user can click on the "more" control 502 in the application interface 501. As shown in FIG5(b), the application interface 501 in the screen of the mobile phone 100 displays an operation interface 503, and the operation interface 503 includes a "share" control 5031 and the like. The user can click on the "share" control 5031. As shown in FIG5(c), the screen of the mobile phone 100 displays an application interface 504 of a sharing application, and the application interface 504 can be displayed superimposed on the application interface 501. The application interface 504 includes controls for multiple shareable devices and a control 505 of the smart assistant "Xiaoyi". The user can click on the control 505 of the smart assistant "Xiaoyi". As shown in FIG5(d), the application interface 506 of the smart assistant "Xiaoyi" is displayed on the application interface 501. Continuing to refer to FIG. 5( d ), the user can enter a user command in the application interface 506 , such as: “Batch blur the eyes of these photos”. In response to the user command, the intelligent assistant can call an application / application service of AI technology, such as: an image processing AI tool, to process the target photo 5011 and the target photo 5012 according to the user command. As shown in FIG. 5( e ), the application interface 506 displays the processing result of the target photo 5011 on which the user command has been executed, that is, the result photo 5013 after blurring the eyes of the person in the target photo 5011. For the processing result of the target photo 5012, the user can click the “Next” control in the application interface 506 , and the application interface 506 will display the processing result of the target photo 5012.

[0093] pass Figure 5(a) to Figure 5(e)The types of multiple target information described are scenarios in which the information processing method of photos is applicable. It can be seen that the user commands supported by the intelligent assistant can act on multiple target information at the same time, and can process the target information more efficiently and conveniently.

[0094] In this specification, it can be seen from the above figures that the electronic device can wake up the smart assistant by responding to the user's operation on the smart assistant's control in the application interface corresponding to the "Share" control. It can be seen that there may be multiple controls of other applications in the application interface corresponding to the "Share" control. In the screen of an electronic device with a relatively limited display area, if the smart assistant's control is arranged at the back among multiple controls, the user needs to perform additional search operations, which brings inconvenience to the user. Figure 6(a) to Figure 6(f) The present invention further describes in detail a scenario in which, in response to a user operation, the control of the smart assistant is arranged on top of the application interface corresponding to the "share" control during the execution of the information processing method by the electronic device.

[0095] As shown in FIG6(a), the screen of the mobile phone 100 displays an application interface 601 of a photo application, including multiple photos and selected target photos 6011 and target photos 6012. The user can click on the "more" control 602 in the application interface 601. As shown in FIG6(b), the application interface 601 in the screen of the mobile phone 100 displays an operation interface 603, and the operation interface 603 includes a "share" control 6031 and the like. The user can click on the "share" control 6031. As shown in FIG6(c), the screen of the mobile phone 100 displays an application interface 604 of a sharing application, and the application interface 604 can be displayed superimposed on the application interface 601. The application interface 604 includes controls for multiple shareable devices and a control 605 of the smart assistant "Xiaoyi". It can be seen that in the application interface 604, the control 605 of the smart assistant "Xiaoyi" is arranged at the back of the controls of multiple shareable devices. The user can perform a long press operation on the control 605 of the smart assistant "Xiao Yi", as shown in Figure 6 (d), the application interface 601 in the screen of the mobile phone 100 displays the operation interface 606, and the operation interface 606 includes the "top" control 6031 and the like. The user can perform a click operation on the "top" control 6031, as shown in Figure 6 (e), the screen of the mobile phone 100 displays the application interface 607 of the control management, in which the control 605 of the smart assistant "Xiao Yi" can be arranged on the top, that is, the control 605 of the smart assistant "Xiao Yi" is arranged before the controls of other shareable devices. At this time, when the user re-performs a click operation on the "share" control 6031 as shown in Figure 6 (b), as shown in Figure 6 (f), in the application interface 604 displayed on the screen of the mobile phone 100, the control 605 of the smart assistant "Xiao Yi" is arranged in front of the controls of multiple shareable devices.

[0096] The information processing method of this specification is applicable not only to electronic devices such as mobile terminals such as mobile phones and tablets, but also to electronic devices such as desktop computers and personal computers. The following will take a personal computer as an example. Figure 7(a) to Figure 7(c) as well as Figure 8(a) to Figure 8(c) A scenario in which a user uses a personal computer to execute an information processing method on target information is further described in detail.

[0097] As shown in FIG. 7( a), the screen of the personal computer 700 displays an application interface 701 of a file application, including multiple selected folders. The user can right-click one of them, and the screen of the personal computer 700 displays an operation interface 702. The user can click the left mouse button on the "Share to Xiaoyi" control in the operation interface 702. As shown in FIG. 7( b), the application interface 703 of the intelligent assistant "Xiaoyi" is displayed on the application interface 701. Continuing to refer to FIG. 7( b), the user can enter a user command in the application interface 703, such as: "Generate a PPT based on these materials". In response to the user command, the intelligent assistant can call an application / application service of AI technology, such as: PPT processing AI tool, to process the information stored in multiple selected folders according to the user command. As shown in FIG. 7( c), the application interface 703 displays the processing results of multiple selected folders that have completed the user command, that is, the PPT files generated according to the information stored in the multiple selected folders.

[0098] And in Figure 8(a) to Figure 8(c) In the described scenario, as shown in FIG8(a), the screen of the personal computer 700 displays the application interface 801 of the music application, including the music file being played. The user can click the left mouse button on the "Share" control 8011 corresponding to the music file, as shown in FIG8(b), and the screen of the personal computer 700 displays the application interface 802 of the sharing application, and the application interface 802 can be superimposed on the application interface 801. The application interface 802 includes controls for multiple shareable devices and a control 8021 of the smart assistant "Xiao Yi". The user can click the left mouse button on the control 8021 of the smart assistant "Xiao Yi", as shown in FIG8(c), and the application interface 803 of the smart assistant "Xiao Yi" is displayed on the application interface 801. The user can enter a user command in the application interface 803, such as: "Share to XX, XX, XX". In response to the user command, the smart assistant can call the application / application service of AI technology to share the music file to "XX, XX, XX".

[0099] It can be seen that in Figure 7(a) to Figure 7(c) In the described scenario, the user can directly perform a user operation on the selected target information to call up the intelligent assistant in the current application interface, that is, display the application interface of the intelligent assistant on the current application interface; Figure 8(a) to Figure 8(c) In the described scenario, the user can also first open the application interface of the shared application, and then call up the smart assistant through the smart assistant control in the application interface of the shared application. A variety of convenient ways to use the smart assistant are provided for different scenarios.

[0100] Fig. 9 This is a schematic diagram of the structure of an electronic device provided in this specification. Fig. 9 As shown, the electronic device (such as a mobile phone, a car computer, etc.) may include: a processor 910, an external memory interface 920, an internal memory 921, a universal serial bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna 1, an antenna 2, a mobile communication module 950, a wireless communication module 960, an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, an earphone interface 970D, a sensor module 980, a button 990, a motor 991, an indicator 992, a camera 993, a display screen 994, and a subscriber identification module (SIM) card interface 995, etc.

[0101] Among them, the above-mentioned sensor module 980 may include sensors such as a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor and an ambient light sensor.

[0102] The processor 910 may include one or more processing units, for example, the processor 910 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0103] In this specification, the processor 910 can be used to respond to user operations performed by a user during the process of a user using an application of an electronic device, invoke an intelligent assistant, display the application interface of the intelligent assistant on the application interface of the application, and respond to user commands for target information, call the application of AI technology to process the target information, obtain processing results corresponding to the target information based on the user command, and prompt the above processing results through the application interface of the intelligent assistant.

[0104] The electronic device implements the display function through a GPU, a display screen 994, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 994 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 910 may include one or more GPUs that execute program instructions to generate or change display information.

[0105] The display screen 994 is used to display images, videos, etc. The display screen 994 includes a display panel. In this specification, the display screen 994 can display the application program and the application interface of the smart assistant to the user; the display screen 994 can also receive user operations performed by the user.

[0106] The electronic device can realize the shooting function through the ISP, camera 993, video codec, GPU, display screen 994 and application processor. The ISP is used to process the data fed back by the camera 993. The camera 993 is used to capture static images or videos. The external memory interface 920 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 910 through the external memory interface 920 to realize the data storage function. For example, files such as music and videos are saved in the external memory card.

[0107] The internal memory 921 may be used to store computer executable program codes, which include instructions. The processor 910 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 921. For example, in this specification, the processor 910 may execute instructions stored in the internal memory 921, and the internal memory 921 may include a program storage area and a data storage area. In this specification, the internal memory 921 may store instructions for executing an information processing method.

[0108] It is understood that the structure illustrated in this specification does not constitute a specific limitation on the electronic device. In this specification, the electronic device (such as a mobile phone, a car computer, etc.) may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0109] Fig.10 It is a software structure block diagram of the electronic device of this manual.

[0110] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In this manual, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, hardware abstraction layer, and kernel layer.

[0111] like Fig.10 As shown, the application layer may include applications such as life, video, reading, shopping, gallery, calendar, call, navigation and music. It is understood that the application here may be an application or service installed by the electronic device 100, or an application or service not installed by the electronic device 100 retrieved by the quick service center.

[0112] The application framework layer may include layout services, window management services, event input services, call processing services, etc. Among them, the layout service and the window management service are used to display the application interface of the application on the screen of the electronic device. The event input service is used to respond to the user operation performed by the user on the electronic device. The call processing service is used to deliver the incoming call message to the call application and establish a hang-up monitoring corresponding to the hang-up state corresponding to the hang-up message.

[0113] The hardware abstraction layer may include a drawing service module (SurfaceFlinger), a call module, etc. Among them, the drawing service module (SurfaceFlinger) is used to draw and synthesize one or more layers in the windows of one or more applications to obtain frame data. The call module can be used to transfer the received incoming call message to the call processing module, process and transfer the received hang-up message, etc.

[0114] The kernel layer includes display driver, event driver, sensor driver, etc.

[0115] In this specification, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When a computer program instruction is loaded and executed on a computer, a process or function according to this specification is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. Computer-readable storage media can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state hard disk), etc.

[0116] A person skilled in the art can understand that to implement all or part of the processes in this specification, the processes can be completed by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes in this specification. The aforementioned storage medium includes: read-only memory (ROM) or random access memory (RAM), a disk or an optical disk, and other media that can store program codes.

[0117] It should be understood that although the terms "first", "second", etc. may be used herein to describe various features, these features should not be limited by these terms. These terms are used only to distinguish and should not be understood as indicating or implying relative importance. For example, a first feature may be referred to as a second feature, and similarly a second feature may be referred to as a first feature without departing from the scope of this specification.

[0118] In addition, various operations will be described as multiple separate operations in a manner that is most helpful for understanding this specification; however, the order of description should not be interpreted as implying that these operations must rely on the order of description, and many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can also be rearranged. When the described operations are completed, the processing can be terminated, but there can also be additional operations not included in the drawings. The processing can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0119] Unless the context dictates otherwise, the terms "comprising," "having," and "including" are synonymous. The phrase "A / B" means "A or B." The phrase "A and / or B" means "(A), (B), or (A and B)."

[0120] As used herein, the term "module" may refer to, be part of, or include: a memory (shared, dedicated, or group) for running one or more software or firmware programs, an application-specific integrated circuit (ASIC), an electronic circuit and / or processor (shared, dedicated, or group), a combinational logic circuit, and / or other suitable components that provide the described functionality.

[0121] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order is not required. Instead, in this specification, these features may be described in a manner and / or order different from that shown in the illustrative drawings. In addition, the structural or method features included in a specific figure do not mean that such features are required to be included. In this specification, these features may not be included, or these features may be combined with other features.

[0122] The above is a detailed description of the specification in conjunction with the drawings, but the use of the technical solution of the present application is not limited to the various applications mentioned in the specification. Various structures and variations can be easily implemented with reference to the technical solution of the present application to achieve the various beneficial effects mentioned in this article. Within the knowledge of ordinary technicians in this field, various changes made without departing from the purpose of the present application should be included in the scope of the patent application.

Claims

1. An information processing method, applied to an electronic device, characterized in that: The method comprises: Displaying a first interface, wherein the first interface includes information to be processed; In response to detecting an operation of triggering a sharing interface, displaying a sharing interface, the sharing interface including icons of multiple applications, the icons of the multiple applications including an icon of a smart assistant; Detecting an operation of triggering the icon of the intelligent assistant, and displaying a second interface, wherein the second interface includes controls related to the input of the user's intention and the information to be processed; In response to detecting input of user intention, displaying a processing result corresponding to the user intention, wherein the user intention is used to instruct the intelligent assistant to process the information to be processed.

2. The method according to claim 1, characterized in that The step of displaying the sharing interface in response to detecting an operation that triggers the sharing interface includes: In response to detecting a first operation on a first control in the first interface, displaying a first operation interface; In response to detecting a first operation on a second control in the first operation interface, the sharing interface is displayed, wherein the second control includes a sharing control.

3. The method according to claim 1, characterized in that The step of displaying the sharing interface in response to detecting an operation that triggers the sharing interface includes: In response to detecting a second operation on the information to be processed, displaying the sharing interface.

4. The method according to any one of claims 1 to 3, characterized in that The displaying of the sharing interface includes displaying the sharing interface on the first interface in at least one of a window form, a pop-up menu, and a semi-modal suspension form; or Displaying the second interface includes displaying the second interface on the first interface in at least one of a window form, a pop-up menu, and a semi-modal suspension form.

5. The method according to claim 1, characterized in that The controls related to the input of user intention include a user intention input box and / or a user intention prompt box, wherein the user intention input box is used for the user to input instructions, and the user intention prompt box includes a user intention generated based on the type or content of the information to be processed, or generated based on the user's historical behavior.

6. The method according to claim 5, characterized in that The input of the user intention includes an instruction input operation for the user intention input box or a selection operation for the user intention prompt box.

7. The method according to any one of claims 1 to 6, characterized in that: The displaying of the processing result corresponding to the user intention includes: The processing result is displayed on the first interface, or the second interface, or another interface different from the first interface or the second interface.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes: Determining a second application from a plurality of generative AI applications according to the type of the information to be processed; Based on the information to be processed and the input of the user's intention, the processing result is obtained using the second application.

9. The method according to any one of claims 1 to 8, characterized in that: Corresponding to the information to be processed being an image, the processing of the information to be processed includes at least one of the following: The image is beautified, cut out or coded.

10. The method according to claim 7, characterized in that Corresponding to the information to be processed being text, the processing of the information to be processed includes at least one of the following: Translate or extract the main points of the text.

11. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device, and A processor, used for reading instructions to execute the information processing method according to any one of claims 1 to 10.

12. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is enabled to execute the information processing method according to any one of claims 1 to 10.

13. A computer program product, characterized in that include: A non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium contains a computer program code for executing the information processing method according to any one of claims 1 to 10.

Images