An application interaction method and an interaction device

Abstract

The application provides an application interaction method and device. The interaction method is applied to a target application. When the target application runs in the background, a floating control is displayed on an interaction interface of a first application running in the foreground in a terminal. The interaction method comprises the following steps: in response to a triggering operation on the floating control, the floating control is switched to an active state; a target application service request is received through the floating control in the active state; in response to the target application service request, to-be-processed information in the terminal is acquired; and the to-be-processed information is processed according to the target application service request. In this way, the user can conveniently use the service of the target application under the page of other applications, so as to expand the information acquisition mode of the target application and reduce the application switching cost.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an interactive method and device for an application. Background Technology

[0002] In practice, users have a need for convenient use of a particular application in different application scenarios. Taking AI assistants as an example, most existing AI assistants exist as standalone applications (APPs) or browser plugins (robot assistants), and the prototypes of AI assistants on some AI devices are generally tied to specific computer / mobile phone brands (such as Siri).

[0003] This leads to significant switching costs when using AI assistants across different applications. For example, copying text in App A requires switching to another app to send it and receive AI assistance; or, taking a screenshot requires switching to another app for recognition and AI help. On the other hand, there's the issue of contextual recognition limitations. AI assistants can only receive information sent by the user. For instance, in a chat log, if the user only sends a specific section, the limited information makes it difficult for the AI ​​assistant to provide a response appropriate to the specific context. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an interaction method and device for an application. When the target application is running in the background, it is displayed as a floating control on the interactive interface of the first application running in the foreground on the terminal. The user can activate the floating control on the interactive interface, enabling the target application to receive service requests and automatically obtain pending information from the terminal, thereby expanding the information acquisition methods of the target application. The target application then processes the information and provides application services to the user. In this way, users can conveniently use the services of the target application even from other application pages, thus reducing the cost of switching applications.

[0005] This application provides an interaction method for an application, which is applied to a target application; when the target application is running in the background, it is displayed as a floating control on the interactive interface of a first application running in the foreground on the terminal; the interaction method includes:

[0006] In response to a trigger operation on the floating control, the floating control is switched to an active state;

[0007] The floating control, when activated, receives requests for services from the target application.

[0008] In response to the target application service request, obtain the information to be processed in the terminal;

[0009] The information to be processed is processed according to the target application service request.

[0010] Furthermore, the information to be processed includes at least one of the following: request information included in the target application service request, page information presented in the interactive interface, application information in the second application, and real physical world information collected by the terminal.

[0011] Furthermore, when the information to be processed includes page information presented in the interactive interface, the step of responding to the target application service request and obtaining the information to be processed in the terminal includes:

[0012] In response to the target application service request, obtain the page information presented in the interactive interface through the accessibility interface; or, call the page recording program or page monitoring program to obtain the page information presented in the interactive interface.

[0013] Furthermore, when the information to be processed includes application information in the second application, the target application service request includes the application identifier of the second application and the information identifier of the application information in the second application.

[0014] The step of responding to the target application service request and obtaining the pending information from the terminal includes:

[0015] In response to the target application service request, the second application is determined based on the application identifier;

[0016] According to the information identifier, the application information is requested from the second application through the program interface to obtain the application information fed back by the second application.

[0017] Furthermore, when the information to be processed includes real-world information collected through the terminal, the step of obtaining the information to be processed from the terminal in response to the target application service request includes:

[0018] In response to the target application service request, the target information type is determined, and the corresponding acquisition program is invoked to control the hardware device in the terminal to acquire the real physical world information.

[0019] Furthermore, based on the target application service request, the information to be processed is processed, including:

[0020] Determine the information modalities included in the information to be processed;

[0021] If the information modality includes an image, then the image information in the information to be processed is extracted through the image recognition module;

[0022] If the information modality includes a webpage link, then the webpage information is obtained through the webpage link in the information to be processed;

[0023] If the information modality includes text, then the text information in the information to be processed is extracted;

[0024] If the information modality includes speech, then the speech in the information to be processed is converted into text information through the speech-to-text conversion module;

[0025] By combining the image information and / or the webpage information and / or the text information, the combined information is obtained.

[0026] The spliced ​​information is processed according to the target application service request.

[0027] Furthermore, processing the information to be processed according to the target application service request includes:

[0028] Based on the target application service request and the application information in the second application, output information is generated;

[0029] The output information can be displayed via the floating control; or the output information can be sent to the second application so that the output information can be output through the program functions of the second application.

[0030] Furthermore, in response to a trigger operation on the floating control, switching the floating control to an active state includes:

[0031] When a click operation on the floating control is detected on the page, the floating control is switched to an active state;

[0032] Alternatively, when a preset activation word is recognized from the received voice command, the floating control is switched to an active state;

[0033] Alternatively, when a preset operation is detected on the page, a prompt window is displayed; in response to the user's confirmation of the prompt window, the floating control is switched to an active state.

[0034] This application also provides an interactive device for an application, which is applied to a target application; when the target application is running in the background, it is displayed as a floating control on the interactive interface of a first application running in the foreground on the terminal; the interactive device includes:

[0035] A switching module is used to switch the floating control to an active state in response to a trigger operation on the floating control;

[0036] The receiving module is used to receive target application service requests through the floating control in the active state;

[0037] The acquisition module is used to acquire the information to be processed in the terminal in response to the target application service request;

[0038] The processing module is used to process the information to be processed according to the target application service request.

[0039] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the application interaction method described above are performed.

[0040] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the interactive method of the application described above.

[0041] This application provides an application interaction method and device. When the target application is running in the background, it is displayed as a floating control on the interactive interface of the first application running in the foreground on the terminal. The user can activate the floating control on the interactive interface, enabling the target application to receive service requests and automatically obtain pending information from the terminal, thus expanding the target application's information acquisition methods. The target application then processes the information and provides application services to the user. In this way, users can conveniently use the target application's services even from other application pages, reducing application switching costs.

[0042] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating an interaction method for an application provided in an embodiment of this application is shown;

[0045] Figure 2This illustration shows a structural schematic diagram of an interactive device for an application provided in an embodiment of this application;

[0046] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0048] Research has found that users have a need for convenient use of certain applications in different application scenarios. Taking AI assistants as an example, most existing AI assistants exist as standalone applications (APPs) or browser plugins (robot assistants), and the prototypes of AI assistants on some AI devices are generally tied to specific computer / mobile phone brands (such as Siri).

[0049] This leads to significant switching costs when using AI assistants across different applications. For example, copying text in App A requires switching to another app to send it and receive AI assistance; or, taking a screenshot requires switching to another app for recognition and AI help. On the other hand, there's the issue of contextual recognition limitations. AI assistants can only receive information sent by the user. For instance, in a chat log, if the user only sends a specific section, the limited information makes it difficult for the AI ​​assistant to provide a response appropriate to the specific context.

[0050] Based on this, the embodiments of this application provide an interaction method for an application, which allows users to conveniently use the services of the target application on other application pages, thereby expanding the information acquisition methods of the target application and reducing the cost of switching applications.

[0051] The interaction method provided in this application is applied to a target application installed on a terminal device. The terminal device can be a local electronic device that has downloaded and can run the target application, such as a smartphone, personal computer, or tablet computer.

[0052] When the target application runs in the background, it appears as a floating control on the interactive interface of the first application running in the foreground in the terminal. A foreground program is one that is currently interacting directly with the user. The user can interact with the program using input devices such as a keyboard, mouse, touchscreen, microphone, and camera, and the program's output is displayed directly on the screen. A background program is one that continues to run without direct user interaction. This means these programs are still running, but not displayed in the foreground. The first application refers to any application other than the target application, which may include system applications, such as the system desktop, or installed third-party applications, such as music or shopping applications.

[0053] In this embodiment, the target application running in the background is displayed as a floating control on the interactive interface of the first application. A floating control is a user interface element that can be flexibly moved and resized, such as a "floating ball." Through this floating control, the user can directly interact with the target application on the interactive interface of the first application, including sending information to the target application and receiving feedback information from the target application.

[0054] The following section details the implementation steps of this application, using an AI application (AI assistant) as an example.

[0055] It should be noted that user authorization is required beforehand for the target application to be displayed as a floating control. In one possible implementation, based on the installed AI application, user authorization is obtained to enable the floating ball (window), so that the AI ​​application's floating ball (window) always floats on the page.

[0056] When or after the floating ball (window) is enabled, screen recording and / or listening permissions can also be enabled simultaneously based on user authorization, so as to obtain a more comprehensive and intelligent AI service experience in real time through the AI ​​application floating ball (window).

[0057] Please see Figure 1 , Figure 1 This is a flowchart illustrating an interaction method for an application provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the interaction method includes:

[0058] S101. In response to a trigger operation on the floating control, the floating control is switched to an active state.

[0059] It should be noted that the state of a floating control can include an active state, a waiting-to-be-activated state, etc. (e.g., a semi-transparent state and a collapsed state).

[0060] In one possible implementation, the display effect of the floating control can differ depending on the state. For example, in the collapsed state, a portion of the floating ball (window) can be hidden off-screen, with only a part visible, to minimize interference with the user's use of other primary applications. In the semi-transparent state, the transparency of the floating ball (window) can be set to semi-transparent, again minimizing interference with the user's use of other primary applications. Furthermore, the floating control can also be set to not display on the interfaces of certain applications.

[0061] After detecting a trigger operation on the floating control, the floating control can be switched to the active state. In the active state, the floating control can open an interactive window that floats on the interactive interface. The target application can interact with the user through this interactive window, such as receiving or sending text, voice or video information.

[0062] In another possible implementation, the display effect of the floating control can be the same in different states. In this case, the different states of the floating control determine the different functions it can perform. When it is active, it can interact with the user, but the display effect in the interactive interface remains unchanged. For example, the interactive window is not opened, but the interaction is conducted by voice.

[0063] In one possible implementation, step S101 may include:

[0064] Method 1: When a click operation on the floating control is detected on the page, the floating control is switched to the active state.

[0065] In this method, users can click on the floating control displayed on the interactive interface; at this time, the target application listens for the click operation on the floating control on the page, that is, listens for the trigger operation on the floating control, and responds to this trigger operation by switching the floating control to the active state.

[0066] Alternatively, in method two: when a preset activation word is recognized from the received voice command, the floating control is switched to the active state.

[0067] In this approach, the target application has a pre-defined activation word library. Based on pre-authorized user access, the target application can monitor voice information in real time. When the pre-defined activation word is identified from the voice command indicated by the voice information, a trigger operation for the floating control is determined, and in response to this trigger operation, the floating control is switched to the active state.

[0068] Alternatively, method three: when a preset operation is detected on the page, a prompt window is displayed; in response to the user's confirmation operation of the prompt window, the floating control is switched to the active state.

[0069] In this approach, the target application pre-defines a specific action. When this action is detected on the page, a prompt window is displayed. After confirming the user's response to the prompt window, a trigger action for the floating control is determined, and in response to this trigger action, the floating control is switched to an active state. For example, the pre-defined specific action could be a screenshot operation. Upon detecting the user's screenshot operation, a corresponding prompt window pops up, indicating that the target application can perform image recognition processing on the screenshot. If the user confirms the image recognition processing, the floating control is switched to an active state. Thus, through proactive prompts in the prompt window and user confirmation, the application guides the user to use its functions and prevents accidental touches.

[0070] Thus, this application provides multiple activation methods for the floating control, allowing users to freely activate the floating control and use subsequent functions in various application scenarios according to their needs, thereby improving user convenience.

[0071] S102. Receive the target application service request through the floating control in the active state.

[0072] Among them, through the interactive functions of the floating control in the active state, users can send a target application service request to the floating control in the active state; the target application service request can indicate the relevant information of the required application service.

[0073] S103. In response to the target application service request, obtain the information to be processed in the terminal.

[0074] Here, the information to be processed includes at least one of the following: request information included in the target application service request, page information presented in the interactive interface, application information in the second application, and real physical world information collected by the terminal.

[0075] In this step, the target application can determine which of the aforementioned items(s) of information to be processed is required by parsing the target application service request. For example, parsing the target application service request and using keywords in the request can directly determine whether the information to be processed includes its own request information, page information, application information, or real physical world information. Another example is parsing the target application service request to determine the service scenario, and then determining the information to be processed based on the required information type within the preset service scenario. In one example, the target application service request might be an empathy request in a TV show scenario (watching a show with me) or a praise request in a game scenario (playing a game with me). In this case, the information to be processed should at least include the page information presented in the interactive interface (i.e., TV show video information or game screen information).

[0076] In a first possible implementation, when the information to be processed includes request information included in the target application service request, step S103 may include: determining the parsed request information as the information to be processed by parsing the target application service request.

[0077] In this approach, the request for the target application service carries information related to the required application service (i.e., the request information). For example, a user can send a piece of text or an image to an AI assistant via a floating control, requesting the AI ​​assistant to reply with the text or output a caption that matches the image.

[0078] In a second possible implementation, when the information to be processed includes page information presented in the interactive interface, step S103 may include:

[0079] In response to the target application service request, obtain the page information presented in the interactive interface through the accessibility interface; or, call the page recording program or page monitoring program to obtain the page information presented in the interactive interface.

[0080] Accessibility interfaces refer to a set of functions and tools designed within software, applications, or operating systems to help people with disabilities use computers and mobile devices more easily, such as screen readers. Page recording or monitoring programs can record page information from the interactive interface.

[0081] In this step, the page information presented in the current interactive interface can be easily obtained through the accessibility interface or by calling the page recording program or page monitoring program. Compared with the method of manual copying by the user, the target application can automatically obtain the page context information in a wider range, in a more convenient and efficient manner, and can solve the problem of context recognition obstacles in the existing technology.

[0082] In a third possible implementation, when the information to be processed includes application information in a second application, the target application service request includes the application identifier of the second application and the access address of the application information in the second application.

[0083] Accordingly, step S103 may include:

[0084] In response to the target application service request, the second application is determined based on the application identifier; the application information is requested from the second application through the program interface according to the information identifier, so as to obtain the application information fed back by the second application.

[0085] Here, the second application refers to the first application corresponding to the current interactive interface, as well as other applications besides the target application. In this step, the target application can determine the second application from multiple applications installed on the terminal based on the application identifier, and then request application information from the program interface exposed by the second application based on the information identifier, and obtain the application information fed back by the second application through this program interface. The information identifier can be information encoding or information address, etc.

[0086] In one example, while a user is watching a video through a video-sharing app, a notification appears in the notification bar showing a chat message sent by contact A via a chat app. Therefore, the video-sharing app is the first application, and chat app A is the second application. The user can then activate a floating icon in the video-sharing app's interface and send a request to the target application service: "Help me reply to A's message in chat app A." Therefore, the application is identified as "chat app A," and the message is identified as "the message sent by A." By parsing this target application service request, the AI ​​assistant can request the chat history between A and the user from chat app A.

[0087] In a fourth possible implementation, when the information to be processed includes real-world information collected through the terminal, step S103 may include:

[0088] In response to the target application service request, the target information type is determined, and the corresponding acquisition program is invoked to control the hardware device in the terminal to acquire the real physical world information.

[0089] In this approach, users can also request the collection of real-world information through the target application service; the type of target information can include audio or video, such as collecting user audio by calling the microphone or collecting user video by calling the camera, that is, users can make voice or video calls with the AI ​​assistant through a floating window.

[0090] S104. Process the information to be processed according to the target application service request.

[0091] Here, the information about the desired application service carried in the target application service request can indicate the user's intention to process the information to be processed. For example, requesting an AI assistant to reply with a piece of text or output a caption that matches an image. The target application can determine the processing intention by parsing the target application service request, and then process the information to be processed according to the processing intention, thereby enabling it to provide AI services to the user.

[0092] In one possible implementation, step S104 may include:

[0093] First, determine the information modality included in the information to be processed. Here, information modality typically refers to the way information is presented. It should be understood that the information to be processed may contain one or more information modalities, such as text, images, and audio.

[0094] If the information modality includes images, then the image information in the information to be processed is extracted using the image recognition module. Here, the image information can be extracted from the image using image processing techniques employed by the image recognition module in the target application.

[0095] If the information modality includes a webpage link, then the webpage information is obtained through the webpage link in the information to be processed; here, the corresponding webpage can be accessed and the webpage information can be obtained through the URL webpage link.

[0096] If the information modality includes text, then the text information in the information to be processed is extracted. Here, the text information can be extracted using natural language processing techniques through the text processing module in the target application.

[0097] If the information modality includes speech, then the speech in the information to be processed is converted into text information through a speech-to-text conversion module; here, the speech-to-text conversion module in the target application can be used to convert speech into text information using speech recognition technology.

[0098] Then, the image information and / or the webpage information and / or the text information are concatenated to obtain concatenated information; and the concatenated information is processed according to the target application service request. That is, the concatenated information is processed using the functions of the target application according to the user's processing intention; for example, the large language model in the target application can be called to process the concatenated information to obtain a response.

[0099] The processed results can be displayed to the user via a floating control. For example, after generating a caption, the user can copy the result using the floating control and then manually edit and paste it.

[0100] Furthermore, corresponding to the above implementation, when the information to be processed includes application information in the second application, S104 may include:

[0101] Based on the target application service request and the application information in the second application, output information is generated; the output information is then displayed through the floating control.

[0102] Here, the output information is displayed to the user via a floating control, allowing for subsequent manual operations. For example, the user can copy the output information, then manually open the chat software, paste the output information, and send it.

[0103] Alternatively, the output information can be sent to the second application so that the output information can be output through the program functions of the second application.

[0104] Here, the target application can directly send the output information to the second application and directly control the program functions of the second application to automatically output the output information according to the user's processing intentions. Corresponding to the example above, the AI ​​assistant generates the user's reply to Xiao A, and then sends the reply information directly to Xiao A through chat software A.

[0105] In this approach, users do not need to manually switch between the first and second applications (video software and chat software). The second application can be operated automatically by using the function of the target application, which is displayed as a floating control.

[0106] The various activation methods and methods for acquiring pending information provided in this application greatly expand the application scenarios of this interaction method. For example, a floating control can be used to achieve the following:

[0107] 1. Assist users with writing: For example, when there is already input content, revise and expand the user input in different styles; when there is no input content, obtain the dialogue context as information to be processed and help the user draft a reply.

[0108] 2. Help users summarize and analyze: For example, summarize article content; analyze financial reports; create mind maps.

[0109] 3. Help users find: For example, provide original video information based on video screenshots; find similar low-priced products based on product screenshots.

[0110] 4. Help users process images: For example, provide corresponding answers to identified questions; automatically retouch and edit captured images and videos.

[0111] 5. Empathize with users: For example, watch videos with users and leave comments; play games with users and praise their performance.

[0112] This application provides an interaction method for an application, which is applied to a target application. When the target application is running in the background, it is displayed as a floating control on the interactive interface of a first application running in the foreground on a terminal. The interaction method includes: switching the floating control to an active state in response to a trigger operation on the floating control; receiving a target application service request through the floating control in the active state; obtaining pending information in the terminal in response to the target application service request; and processing the pending information according to the target application service request.

[0113] In this way, when the target application is running in the background, it appears as a floating control on the interface of the first application running in the foreground on the terminal. Users can activate the floating control on the interface, allowing the target application to receive service requests and automatically retrieve pending information from the terminal, thus expanding the target application's information acquisition methods. The target application then processes the information and provides application services to the user. In this way, users can conveniently use the target application's services from other application pages, reducing the cost of switching applications.

[0114] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an interactive device for an application provided in an embodiment of this application. The interactive device is applied to a target application; when the target application is running in the background, it is displayed as a floating control on the interactive interface of the first application running in the foreground on the terminal; such as... Figure 2 As shown, the interactive device 200 includes:

[0115] The switching module 210 is used to switch the floating control to an active state in response to a trigger operation on the floating control;

[0116] The receiving module 220 is used to receive a target application service request through the floating control in the active state;

[0117] The acquisition module 230 is used to acquire the pending information in the terminal in response to the target application service request;

[0118] The processing module 240 is used to process the information to be processed according to the target application service request.

[0119] Furthermore, the information to be processed includes at least one of the following: request information included in the target application service request, page information presented in the interactive interface, application information in the second application, and real physical world information collected by the terminal.

[0120] Furthermore, when the information to be processed includes page information presented in the interactive interface, when the acquisition module 230 is used to acquire the information to be processed in the terminal in response to the target application service request, the acquisition module 230 is used to:

[0121] In response to the target application service request, obtain the page information presented in the interactive interface through the accessibility interface; or, call the page recording program or page monitoring program to obtain the page information presented in the interactive interface.

[0122] Furthermore, when the information to be processed includes application information in the second application, the target application service request includes the application identifier of the second application and the information identifier of the application information in the second application.

[0123] When the acquisition module 230 is used to acquire pending information in the terminal in response to the target application service request, the acquisition module 230 is used to:

[0124] In response to the target application service request, the second application is determined based on the application identifier;

[0125] According to the information identifier, the application information is requested from the second application through the program interface to obtain the application information fed back by the second application.

[0126] Furthermore, when the information to be processed includes real-world information collected through the terminal, when the acquisition module 230 is used to acquire the information to be processed from the terminal in response to the target application service request, the acquisition module 230 is used to:

[0127] In response to the target application service request, the target information type is determined, and the corresponding acquisition program is invoked to control the hardware device in the terminal to acquire the real physical world information.

[0128] Furthermore, when the processing module 240 processes the information to be processed according to the target application service request, the processing module 240 is used to:

[0129] Determine the information modalities included in the information to be processed;

[0130] If the information modality includes an image, then the image information in the information to be processed is extracted through the image recognition module;

[0131] If the information modality includes a webpage link, then the webpage information is obtained through the webpage link in the information to be processed;

[0132] If the information modality includes text, then the text information in the information to be processed is extracted;

[0133] If the information modality includes speech, then the speech in the information to be processed is converted into text information through the speech-to-text conversion module;

[0134] By combining the image information and / or the webpage information and / or the text information, the combined information is obtained.

[0135] The spliced ​​information is processed according to the target application service request.

[0136] Furthermore, when the processing module 240 processes the information to be processed according to the target application service request, the processing module 240 is used to:

[0137] Based on the target application service request and the application information in the second application, output information is generated;

[0138] The output information can be displayed via the floating control; or the output information can be sent to the second application so that the output information can be output through the program functions of the second application.

[0139] Furthermore, when the switching module 210 switches the floating control to an active state in response to a trigger operation on the floating control, the switching module 210 is used to:

[0140] When a click operation on the floating control is detected on the page, the floating control is switched to an active state;

[0141] Alternatively, when a preset activation word is recognized from the received voice command, the floating control is switched to an active state;

[0142] Alternatively, when a preset operation is detected on the page, a prompt window is displayed; in response to the user's confirmation of the prompt window, the floating control is switched to an active state.

[0143] In this way, when the target application is running in the background, it appears as a floating control on the interface of the first application running in the foreground on the terminal. Users can activate the floating control on the interface, allowing the target application to receive service requests and automatically retrieve pending information from the terminal, thus expanding the target application's information acquisition methods. The target application then processes the information and provides application services to the user. In this way, users can conveniently use the target application's services from other application pages, reducing the cost of switching applications.

[0144] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 300 includes a processor 310, a memory 320, and a bus 330.

[0145] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 and the memory 320 communicate via the bus 330. When the machine-readable instructions are executed by the processor 310, they can perform the operations described above. Figure 1The steps of the application interaction method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0146] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the application interaction method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0147] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An interaction method for an application, characterized in that, The interaction method is applied to the target application; when the target application is running in the background, it is always displayed as a floating control on the interactive interface of the first application running in the foreground in the terminal; the state of the floating control includes an active state and a waiting-to-activate state; in the waiting-to-activate state, the floating control is presented as a partially hidden and partially exposed collapsed state, or a semi-transparent state. The target application is an AI application; the interaction method includes: In response to a trigger operation on the floating control, the floating control is switched to an active state; The floating control, when activated, receives requests for services from the target application. In response to the target application service request, obtain the information to be processed in the terminal; The information to be processed is processed according to the target application service request; The step of switching the floating control to an active state in response to a trigger operation on the floating control includes: When a preset operation is detected on the page, a prompt window is displayed; in response to the user's confirmation of the prompt window, the floating control is switched to an active state; the preset operation includes a screenshot operation; the prompt window indicates that the target application can perform image recognition processing on the screenshot; In response to the target application service request, obtain the pending information from the terminal, including: The target application service request is parsed to determine the type of information to be processed and the information to be processed in the terminal is obtained. When the information to be processed includes page information presented in the interactive interface, the step of responding to the target application service request and obtaining the information to be processed in the terminal includes: In response to the target application service request, obtain the page information presented in the interactive interface through the accessibility interface; or, call the page recording program or page monitoring program to obtain the page information presented in the interactive interface. When the information to be processed includes application information in the second application, the target application service request includes the application identifier of the second application and the information identifier of the application information in the second application; The step of responding to the target application service request and obtaining the pending information from the terminal includes: In response to the target application service request, the second application is determined based on the application identifier; According to the information identifier, the application information is requested from the second application through the program interface to obtain the application information fed back by the second application; when the target application service request is to help the user reply to the target contact's information in the chat software, the application identifier is the identifier of the chat software, the information identifier is the information sent by the target contact; the application information is the chat history between the target contact and the user; Based on the target application service request, the information to be processed is processed, including: The information modality to be processed is determined; the information to be processed also includes at least one of the following: request information included in the target application service request, application information in the second application, and real physical world information collected by the terminal; If the information modality includes an image, then the image information in the information to be processed is extracted through the image recognition module; If the information modality includes a webpage link, then the webpage information is obtained through the webpage link in the information to be processed; If the information modality includes text, then the text information in the information to be processed is extracted; If the information modality includes speech, then the speech in the information to be processed is converted into text information through the speech-to-text conversion module; By combining the image information and / or the webpage information and / or the text information, the combined information is obtained. The spliced ​​information is processed according to the target application service request; The step of processing the information to be processed according to the target application service request further includes: Based on the target application service request and the application information in the second application, output information is generated. Without the user switching between the first application and the second application, the output information is directly sent to the second application through the target application, and the program function of the second application is directly controlled to output the output information according to the user's processing intention; controlling the program function of the second application to output the output information includes: controlling the chat software to send reply information to the target contact.

2. The interaction method according to claim 1, characterized in that, When the information to be processed includes real-world information collected through the terminal, the step of obtaining the information to be processed from the terminal in response to the target application service request includes: In response to the target application service request, the target information type is determined, and the corresponding acquisition program is invoked to control the hardware device in the terminal to acquire the real physical world information.

3. The interaction method according to claim 1, characterized in that, The step of processing the information to be processed according to the target application service request further includes: The output information is displayed via the floating control.

4. The interaction method according to claim 1, characterized in that, In response to a trigger operation on the floating control, switching the floating control to an active state includes: When a click operation on the floating control is detected on the page, the floating control is switched to an active state; Alternatively, when a preset activation word is recognized from the received voice command, the floating control is switched to an active state.

5. An interactive device for an application, characterized in that, The interactive device is applied to the target application; when the target application is running in the background, it is always displayed as a floating control on the interactive interface of the first application running in the foreground in the terminal; the state of the floating control includes an active state and a waiting-to-activate state; in the waiting-to-activate state, the floating control is presented as a partially hidden and partially exposed collapsed state, or a semi-transparent state. The target application is an AI application; the interactive device includes: A switching module is used to switch the floating control to an active state in response to a trigger operation on the floating control; The receiving module is used to receive target application service requests through the floating control in the active state; The acquisition module is used to acquire the information to be processed in the terminal in response to the target application service request; The processing module is used to process the information to be processed according to the target application service request; When the switching module switches the floating control to an active state in response to a trigger operation on the floating control, the switching module is used to: When a preset operation is detected on the page, a prompt window is displayed; in response to the user's confirmation of the prompt window, the floating control is switched to an active state; the preset operation includes a screenshot operation; the prompt window indicates that the target application can perform image recognition processing on the screenshot; When the acquisition module is used to acquire pending information from the terminal in response to the target application service request, the acquisition module is used to: The target application service request is parsed to determine the type of information to be processed and the information to be processed in the terminal is obtained. When the information to be processed includes page information presented in the interactive interface, the acquisition module is further configured to: In response to the target application service request, obtain the page information presented in the interactive interface through the accessibility interface; or, call the page recording program or page monitoring program to obtain the page information presented in the interactive interface. When the information to be processed includes application information in the second application, the target application service request includes the application identifier of the second application and the information identifier of the application information in the second application; When the acquisition module is used to acquire pending information from the terminal in response to the target application service request, the acquisition module is used to: In response to the target application service request, the second application is determined based on the application identifier; According to the information identifier, the application information is requested from the second application through the program interface to obtain the application information fed back by the second application; when the target application service request is to help the user reply to the target contact's information in the chat software, the application identifier is the identifier of the chat software, the information identifier is the information sent by the target contact; the application information is the chat history between the target contact and the user; When the processing module processes the information to be processed according to the target application service request, the processing module is used to: The information modality to be processed is determined; the information to be processed also includes at least one of the following: request information included in the target application service request, application information in the second application, and real physical world information collected by the terminal; If the information modality includes an image, then the image information in the information to be processed is extracted through the image recognition module; If the information modality includes a webpage link, then the webpage information is obtained through the webpage link in the information to be processed; If the information modality includes text, then the text information in the information to be processed is extracted; If the information modality includes speech, then the speech in the information to be processed is converted into text information through the speech-to-text conversion module; By combining the image information and / or the webpage information and / or the text information, the combined information is obtained. The spliced ​​information is processed according to the target application service request; When the processing module processes the information to be processed according to the target application service request, the processing module is further configured to: Based on the target application service request and the application information in the second application, output information is generated. Without the user switching between the first application and the second application, the output information is directly sent to the second application through the target application, and the program function of the second application is directly controlled to output the output information according to the user's processing intention; controlling the program function of the second application to output the output information includes: controlling the chat software to send reply information to the target contact.

6. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the interactive method of the application as described in any one of claims 1 to 4.

Images