Window operation method, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a window operation method, electronic equipment, a storage medium and a program product, and relates to the technical field of communication, so that a plurality of focus windows (such as a first focus window and a second focus window) can exist in a terminal, and when a user processes a task in the first focus window, the user experience is improved. The intelligent agent can autonomously execute the operation in the second focus window, so that the operation performance and the task processing efficiency of the terminal are improved. The method is applied to the terminal and comprises the steps of displaying a first focus window and a second focus window; in response to a first interaction event triggered by the user for the first focus window, executing the first interaction event; in response to a second interaction event triggered by the agent for the second focus window, executing the second interaction event; wherein the intelligent agent responds to a user instruction or autonomously executes the operation in the second focus window according to a user behavior.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a window operation method, electronic device, storage medium, and program product. Background Technology

[0002] With the continuous development of the smart terminal industry, people's reliance on and demand for mobile phones and other terminal devices are constantly increasing. Typically, to avoid task conflicts, the user interface operating system of the terminal (such as Windows, macOS, Linux, and other desktop operating systems on mobile phones and computers) adopts a single-focus operation mode for task processing. That is, only one active window exists in a given interface operating system at any given time.

[0003] However, this operating method limits the number of tasks a terminal can handle, resulting in low operational performance and low task processing efficiency. Improving the terminal's operational performance and task processing efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0004] This disclosure provides a window operation method, electronic device, storage medium, and program product, which enable multiple focus windows (e.g., a first focus window and a second focus window) to exist in a terminal, and allow an intelligent agent to autonomously perform operations in the second focus window while the user is processing tasks in the first focus window, thereby improving the terminal's operational performance and task processing efficiency.

[0005] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0006] On the one hand, this disclosure provides a window operation method applied to a terminal, the method including:

[0007] In response to a first interaction event triggered by the user in relation to the first focused window, execute the first interaction event;

[0008] In response to a second interaction event triggered by the agent in relation to the second focus window, the agent executes the second interaction event; wherein the agent responds to user instructions or autonomously performs operations in the second focus window based on user behavior.

[0009] On the other hand, an electronic device is provided, comprising: a processor and a memory for storing processor-executable instructions; the processor is configured to execute the instructions such that the electronic device implements any of the window operation methods provided in the embodiments of this disclosure.

[0010] In another aspect, a computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed on a computer, cause the computer to implement any of the window operation methods provided in the embodiments of this disclosure.

[0011] In another aspect, a computer program product is provided, which includes computer program instructions that, when executed on a computer, cause the computer to implement any of the window operation methods provided in the embodiments of this disclosure.

[0012] The window operation method provided in this disclosure, on the one hand, allows multiple focus windows (e.g., a first focus window and a second focus window) to exist on the terminal, enabling the terminal to process multiple tasks in parallel; on the other hand, it allows the user to process tasks in the first focus window while the intelligent agent can autonomously execute operations in the second focus window, realizing collaborative task processing between the user and the intelligent agent, saving user time and reducing the user's task processing burden; furthermore, considering that artificial intelligence has more efficient and accurate processing capabilities than users, it improves the accuracy and efficiency of task processing. In summary, the window operation method provided in this disclosure improves the terminal's operational performance and task processing efficiency. Attached Figure Description

[0013] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0014] Figure 1 This is one of the structural schematic diagrams of a terminal provided in an embodiment of this disclosure;

[0015] Figure 2 A flowchart illustrating a window operation method provided in an embodiment of this disclosure;

[0016] Figure 3 A schematic diagram of a terminal display interface provided in an embodiment of this disclosure;

[0017] Figure 4 This is a second schematic diagram of the structure of a terminal provided in an embodiment of this disclosure;

[0018] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0019] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0020] Unless the context otherwise requires, throughout the specification and claims, the term "comprise" and its other forms, such as the third-person singular "comprises" and the present participle "comprising," are interpreted as open-ended and encompassing, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific example," or "some examples," etc., are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0022] In this disclosure, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] In addition, the use of “based on” implies openness and inclusivity, because processes, steps, calculations or other actions “based on” one or more of the stated conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated.

[0024] With the continuous development of the smart terminal industry, people's reliance on and demand for mobile phones and other terminal devices are constantly increasing. Typically, to avoid task conflicts, the user interface operating system of the terminal (such as Windows, macOS, Linux, and other desktop operating systems on mobile phones and computers) adopts a single-focus operation mode for task processing. That is, only one active window exists in a given interface operating system at any given time.

[0025] However, this operating method limits the number of tasks a terminal can handle, resulting in low operational performance and low task processing efficiency. Improving the terminal's operational performance and task processing efficiency has become an urgent technical problem to be solved.

[0026] To address this issue, this disclosure provides a window operation method applied to a terminal. The method includes: displaying a first focus window and a second focus window; executing a first interaction event in response to a first interaction event triggered by a user on the first focus window; and executing a second interaction event in response to a second interaction event triggered by an intelligent agent on the second focus window. The intelligent agent autonomously performs operations in the second focus window in response to user commands or based on user behavior. This improves the terminal's operational performance and task processing efficiency.

[0027] For ease of understanding, one type of terminal involved in this disclosure will be introduced below.

[0028] like Figure 1 As shown, the terminal includes an artificial intelligence (AI) subsystem 110 and a multi-focus management subsystem 120. In some embodiments, the terminal further includes a user operation detection subsystem 130 and / or a control subsystem 140.

[0029] The AI ​​subsystem 110 employs AI technology, possesses AI computing resources, or has a communication connection with the network side, enabling it to perform AI calculations based on the network-side AI computing resources. The AI ​​subsystem 110 includes intelligent agents, which may also be referred to as AI modules, AI assistants, AI agents, AI units, or AI intelligent agents, etc., without specific limitations in this disclosure. The intelligent agent must at least have interface operation capabilities, such as simulating a real person browsing the web.

[0030] In some embodiments, the intelligent agent has at least one of the following capabilities: predicting the user's operational needs for the second focus window based on user behavior; responding to user instructions or autonomously performing operations in the second focus window based on user behavior; simulating real-person operations and triggering interactive events (such as the second interactive events described below) in the second focus window; outputting the task processing result in the second focus window; and outputting prompt information to indicate that the task in the second focus window has been completed. The meaning of the second focus window is detailed in the description of the method embodiments below.

[0031] The multi-focus management subsystem 120 is used to manage or maintain one or more focus windows in a terminal (e.g., a first focus window and at least one second focus window, or at least one second focus window).

[0032] In some embodiments, the multi-focus management subsystem 120 has at least one of the following functions: separating the event source of an interactive event; distributing the interactive event to the corresponding focus window; and maintaining focus mapping information, wherein the focus mapping information includes at least one focus identifier and at least one window handle of a second focus window, and each focus identifier corresponds to a window handle of a second focus window.

[0033] The user operation detection subsystem 130 has at least one of the following functions: detecting user operations and / or detecting simulated operations of intelligent agents; acquiring interaction events (e.g., first interaction event, second interaction event, third interaction event, fourth interaction event); and monitoring the status and / or position of each focus window (e.g., second focus window).

[0034] The control subsystem 140 has at least one of the following functions: acquiring user instructions; forwarding user instructions to the AI ​​subsystem 110 (e.g., the agent in the AI ​​subsystem 110) or the multi-focus management subsystem 120; and issuing control instructions to at least one of the AI ​​subsystem 110 (e.g., the agent in the AI ​​subsystem 110), the multi-focus management subsystem 120, or the user operation detection subsystem 130.

[0035] In some embodiments, the terminal has at least AI computing capabilities to support the operation of the intelligent agent. Specifically, the terminal has local AI computing capabilities; and / or, the terminal implements AI computing capabilities based on AI computing resources located at the network end.

[0036] In some embodiments, the terminal also has display capabilities. Or, in other words, the terminal has display space.

[0037] It should be noted that in the window operation method provided in this disclosure, the terminal can be a personal computer (PC) device, user equipment (UE), handheld device with various communication functions, vehicle device, wearable device, computer, smart home device or smart office device, etc., and this disclosure does not specifically limit it.

[0038] It should be understood that the terminal examples described above are merely for illustrating the technical solutions of this disclosure more clearly and do not constitute a limitation of this disclosure. Those skilled in the art will recognize that, with the evolution of network architecture and the emergence of new service scenarios, the technical solutions provided in this disclosure are equally applicable to similar technical problems.

[0039] To more clearly illustrate the window operation method provided in this disclosure, the window transmission method provided in this disclosure is described below with reference to the accompanying drawings. It should be noted that the various embodiments of this disclosure can be referenced or understood from each other. For example, the same or similar steps, method embodiments, and device embodiments can be referenced from each other without limitation.

[0040] like Figure 2 As shown, this disclosure provides a window operation method applied to a terminal, the method including:

[0041] S101, Display the first focus window and the second focus window.

[0042] The first focus window receives at least the actual user-triggered interaction events (such as the first interaction event described below); for example, it can receive at least the interaction events in which the user interacts with the system via mouse, touchscreen, and keyboard. It should be understood that the first focus window corresponds to input from a real mouse and a real keyboard.

[0043] It should be noted that at least one primary focus window is present and displayed by default in the terminal to ensure that there is at least one focus window that the user can directly interact with.

[0044] The second focus window receives at least the virtual interaction events triggered by the agent (such as the second interaction events described below). The agent has interface operation capabilities and can trigger virtual mouse events, virtual touch events, or other virtual input events to replace or assist the user in performing operations. The agent's capabilities can be referred to above. Figure 1 For related textual descriptions, please refer to the descriptions in other method embodiments below.

[0045] There can be one or more secondary focus windows. When there are multiple secondary focus windows, there are multiple focuses in the terminal for the agent to operate on. The agent can process tasks in multiple secondary focus windows simultaneously, thereby replacing or assisting the user in completing more tasks and improving task processing efficiency.

[0046] It should be noted that the number of second focus windows mentioned above is based on the premise that a second focus window has already been created. If a second focus window is not actively created, it may not exist on the terminal. The terminal only creates and displays a second focus window when the focus creation conditions are met. The focus creation conditions and the method for creating the second focus window can be found in the descriptions in the method embodiments below.

[0047] To more clearly illustrate the window operation methods provided in this disclosure, the following is a brief introduction to the concepts of focus window (e.g., first focus window and / or second focus window) and focus.

[0048] The first and second focus windows mentioned above are both focus windows, which are application windows in an active state. This active state can also be understood as having focus or being in a focus state. Focus typically refers to the user's current activity center or point of interest. When an application window has focus, it can be considered that the application window or the program corresponding to the application window is in a focus state and can be directly operated by the user or intelligent agent, for example, it can directly receive and execute interactive events triggered by the user or intelligent agent. It should be understood that when the terminal displays the first and second focus windows, at least two focus windows exist simultaneously on the terminal, thus enabling multi-focus operation.

[0049] It is understood that, in addition to the aforementioned focused window, there may be other application windows in the terminal that do not have focus, such as background applications that are inactive. This disclosure does not specifically limit this.

[0050] In some embodiments, each focused window corresponds to a unique focus identifier and window handle. The terminal maintains the focus identifier and window handle of each focused window in real time.

[0051] The focus identifier of the focus window can also be called the window identifier, focus number, etc., and is used to uniquely identify the focus of the focus window. The concept of focus can be referred to the description above.

[0052] The window handle of the focused window is used to uniquely identify the program corresponding to that focused window.

[0053] To accurately distribute interactive events, the terminal can also maintain relevant information about each focused window (such as the window handle and / or focus marker of the focused window) to accurately distribute interactive events based on this information. For example, the state of each focused window can be monitored or tracked in real time to update the window handle of each focused window. Specific update and maintenance methods are described below.

[0054] S102. In response to the first interaction event triggered by the user for the first focus window, execute the first interaction event.

[0055] The first interaction event, also known as the first focus event, is the actual interaction event triggered by the user.

[0056] In some embodiments, the terminal sends a first interaction event to the program corresponding to the first focus window based on the window handle of the first focus window, and the program corresponding to the first focus window executes the first interaction event. It should be understood that the program corresponding to the first focus window is an active application that only receives interaction events triggered on that first focus window.

[0057] Furthermore, the terminal can execute the first interaction event based on its type. For example, if the first interaction event is a scrolling event, the terminal controls the scrolling of the display page in the program corresponding to the first focus window. As another example, if the first interaction event is a keyboard event, the terminal receives the text input from the keyboard event and transmits the text input to the program corresponding to the first focus window.

[0058] Based on step S102, the first interactive event can be accurately sent to the corresponding program execution through the window handle.

[0059] S103. In response to a second interaction event triggered by the agent for the second focus window, execute the second interaction event; wherein, the agent responds to user instructions or autonomously performs operations in the second focus window based on user behavior.

[0060] Here, the intelligent agent uses AI technology and has AI computing capabilities. It can also be called an AI module, AI assistant, AI agent, AI unit, or AI intelligent agent, etc., and can autonomously perform operations in the second focus window.

[0061] In some embodiments, the intelligent agent realizes AI computing capabilities based on the terminal's local AI computing resources (e.g., AI resources in the terminal's AI subsystem) or based on network-side AI computing resources.

[0062] In some embodiments, the intelligent agent has interface operation capabilities. For example, the intelligent agent can trigger interactive events (such as a second interactive event) to perform actions such as swiping, selecting, and scrolling down on web pages, simulating human operation for web browsing or web browsing. As another example, the intelligent agent can use image recognition, text recognition, or similarity comparison to filter online resources such as items, services, or stores that meet certain requirements from a web page; furthermore, it can determine the filtered results and output them.

[0063] In some embodiments, the intelligent agent has at least one of the following capabilities: predicting the user's operational needs for the second focus window based on user behavior; responding to user instructions or autonomously performing operations in the second focus window based on user behavior; simulating real-person operations and triggering interactive events (e.g., second interactive events) in the focus window; outputting the task processing results in the second focus window; and outputting prompt information to indicate that the task in the second focus window has been completed.

[0064] In some embodiments, the intelligent agent can also acquire local data from the terminal for task processing. For example, it can read and generate the data table required by the user in the background from the local computer file system of the terminal, based on the data storage location that has been informed in advance or automatically recorded before.

[0065] It should be understood that intelligent agents can simulate human actions within a focused window (e.g., a second focused window) without altering third-party applications. In some cases, because some software does not publicly expose its data query interface (e.g., real-time product prices in some software are not directly accessible for querying), intelligent agents can simulate human actions to browse web pages and filter desired data, thereby increasing data acquisition channels and improving task processing efficiency. In some examples, intelligent agents simulate human actions by triggering interaction events (e.g., a second interaction event).

[0066] Here, the second interaction event can also be called the second focus event, which is a virtual interaction event triggered by the agent.

[0067] In some embodiments, the second interaction event is triggered by the agent in response to a user command and / or based on user behavior. The triggering methods of the second interaction event are briefly described below.

[0068] (1) The second interaction event is triggered by the agent responding to the user's command.

[0069] For example, the user instruction includes a first operation instruction; the method further includes: the terminal acquiring the first operation instruction input by the user to the intelligent agent, the first operation instruction being used to instruct the intelligent agent to perform an operation on the second focus window; in response to the first operation instruction, triggering a second interaction event through the intelligent agent on the second focus window.

[0070] In a specific example, the first operation instruction can be input to the terminal in any form, such as via keyboard, mouse, text, instruction button, or voice command.

[0071] In a specific example, the terminal displays an application interface for controlling the intelligent agent, allowing the user to select or input a first operation command based on this interface.

[0072] In a specific example, the application interface described above for controlling the agent can be used to orchestrate the agent's tasks based on user operations. For instance, the application interface can display multiple draggable components and an orchestration area, with each component corresponding to a functional node; the agent's tasks are orchestrated based on at least one component dragged by the user into the orchestration area; the agent's tasks are then sent to the agent via a first operation command, enabling the agent to perform operations in a second focus window based on the agent's tasks.

[0073] In a specific example, the second focus window indicated by the first operation instruction already exists in the terminal.

[0074] In another specific example, if the second focus window indicated by the first operation instruction does not exist on the terminal, then the terminal generates a focus identifier and creates a second focus window corresponding to the focus identifier so that the agent can perform operations on the second focus window.

[0075] Based on this, the operation of the intelligent agent can be flexibly determined based on user instructions, so as to flexibly control the intelligent agent to complete the task on behalf of the user in the second focus window, thereby improving the efficiency of task processing.

[0076] (2) The second interaction event is triggered based on user behavior.

[0077] For example, the above method further includes: the terminal determining the user's behavior in the first focus window; predicting the user's operation needs for the second focus window through an intelligent agent based on the user's behavior in the first focus window; and triggering a second interaction event for the second focus window through the intelligent agent based on the operation needs.

[0078] In a specific example, the terminal determines user behavior based on at least one of the following: a first interaction event triggered by the user in the first focus window, and / or the display information of the first focus window.

[0079] In a specific example, user behavior can also be understood as the user's behavioral intent or the user's behavioral purpose.

[0080] For example, a user action could be: completing a presentation document for a quarterly report. Further, the predicted operational needs based on this user action could be: retrieving and analyzing market price information for the current and previous quarters, and / or generating data charts of market price information for the current and previous quarters. Thus, while the user fills in the content framework and writes the page in the first focus window, the terminal can automatically perform operations in the second focus window through an intelligent agent, thereby improving the efficiency of the user in completing the quarterly report presentation document.

[0081] For example, a user behavior could be: writing a paper manuscript. Further, the operational requirement predicted from this user behavior could be: collecting reference materials.

[0082] For example, user behavior could be: developing a team-building plan. Further, the operational need predicted from this user behavior could be: finding and filtering the most cost-effective vendors.

[0083] Based on this, the terminal can predict operational needs based on user behavior through an intelligent agent, thereby processing tasks synchronously through the intelligent agent in the second focus window, improving task processing efficiency and reducing the user's task processing pressure.

[0084] It should be noted that the above-mentioned triggering method for the second interactive event is only an example, and there may be other triggering methods, such as the agent triggering the second interactive event based on a preset target, etc. This disclosure does not impose specific restrictions on this.

[0085] In some embodiments, step S103 includes: in response to a second interaction event triggered by the agent for the second focus window, determining the window handle of the second focus window; based on the window handle of the second focus window, sending the second interaction event to the program corresponding to the second focus window, and executing the second interaction event through the program corresponding to the second focus window. Based on this, the second interaction event can be accurately sent to the corresponding program for execution through the window handle.

[0086] It should be understood that the program corresponding to the second focus window is an active application that only receives interactive events triggered on that second focus window. Furthermore, the terminal can also execute the second interactive event based on its event type.

[0087] In some embodiments, the number of second focus windows is at least one. When there are multiple second focus windows, before executing the second interaction event, the terminal (e.g., a multi-focus management system in the terminal) determines the target focus window corresponding to the second interaction event from among the multiple second focus windows and distributes the second interaction event to the target focus window so that it can be executed by the program corresponding to the target focus window.

[0088] For example, when step S101 includes displaying a first focus window and at least one second focus window, step S103 includes: in response to a second interaction event triggered by the agent for at least one second focus window, determining the window handle of a target focus window based on the focus identifier of the second interaction event and preset focus mapping information; wherein, the target focus window is the second focus window corresponding to the window handle of at least one second focus window and the focus identifier of the second interaction event; based on the window handle of the target focus window, sending the second interaction event to the program corresponding to the target focus window, and executing the second interaction event through the program corresponding to the target focus window; wherein, the focus mapping information includes at least one focus identifier and at least one window handle of a second focus window, and each focus identifier corresponds to a window handle of a second focus window.

[0089] Based on this, the second interactive event can be accurately sent to the corresponding program in the second focus window for execution by using the focus identifier, thus improving the accuracy of task execution. This allows the terminal to handle tasks in multiple focus windows simultaneously without switching focus windows, improving task processing efficiency.

[0090] It should be noted that steps S102 and S103 above can be executed in parallel, allowing the user to perform tasks in the first focus window while the intelligent agent operates simultaneously with the user in the second focus window. This allows the intelligent agent to simulate a real person handling tasks on behalf of the user in the second focus window. Thus, without affecting the user's normal use of the terminal, the user can also observe the intelligent agent's task processing process through the second focus window, improving the user experience.

[0091] The window operation method provided in this disclosure, on the one hand, allows multiple focus windows (e.g., a first focus window and a second focus window) to exist on the terminal, enabling the terminal to process multiple tasks in parallel; on the other hand, it allows the user to process tasks in the first focus window while the intelligent agent can autonomously execute operations in the second focus window, realizing collaborative task processing between the user and the intelligent agent, saving user time and reducing the user's task processing burden; furthermore, considering that artificial intelligence has more efficient and accurate processing capabilities than users, it improves the accuracy and efficiency of task processing. In summary, the window operation method provided in this disclosure improves the terminal's operational performance and task processing efficiency.

[0092] In some embodiments, the first focus window can also receive virtual interaction events triggered by the agent, meaning the agent can autonomously perform operations within the first focus window. This allows the agent to perform operations in various focus windows (e.g., the first focus window and / or the second focus window) on behalf of or assist the user. Consequently, task processing can be completely or partially transferred to the agent, reducing the user's task processing burden and improving task processing efficiency.

[0093] For example, the above method further includes: obtaining a second operation instruction input by the user to the agent, the second operation instruction being used to instruct the agent to perform an operation on the first focus window; in response to the second operation instruction, triggering a fourth interaction event through the agent on the first focus window; and in response to the fourth interaction event triggered by the agent on the first focus window, executing the fourth interaction event.

[0094] Furthermore, the terminal can also display an application interface for controlling the intelligent agent, allowing the user to select or input a second operation command based on this interface. In some examples, this application interface for controlling the intelligent agent can also be used to orchestrate tasks for the intelligent agent based on user actions.

[0095] Based on this, an intelligent agent can also be implemented in the first focus window to replace or assist the user in performing operations, thereby reducing the user's task processing pressure and improving processing efficiency.

[0096] It should be understood that when a terminal's agent can autonomously execute operations in both the first and second focus windows, after receiving user commands input by the user, the agent can first determine whether to execute the operation directly in the first or second focus window to decide whether a new focus needs to be created. It should be noted that the focus mentioned here refers to a point of interest that can be manipulated by the user and is created in the form of a focus window.

[0097] For example, if a user instruction (such as the second operation instruction mentioned above) instructs the agent to perform an operation on an existing first focus window, the terminal does not need to create a new focus window.

[0098] For example, if a user command instructs an agent to perform an operation on a second focus window, and that second focus window does not exist on the terminal, the terminal creates a new focus window.

[0099] This enables the agent to operate in both the first and second focus windows, allowing users to flexibly manage the agent and enriching the user experience.

[0100] Furthermore, considering that the agent operates in the second focus window, this window may contain content of interest to the user. Therefore, in some embodiments, the second focus window can also be configured to receive real interaction events triggered by the user, allowing the user to operate directly within the second focus window, thereby improving the user experience.

[0101] To avoid operational conflicts when the user and the agent simultaneously perform operations in the second focus window, this disclosure also provides a window operation method, including: acquiring a third interaction event triggered by the user in the second focus window during the execution of a second interaction event; stopping the execution of the second interaction event and executing the third interaction event in response to the third interaction event triggered by the user in the second focus window. Based on this, the user can interfere with the agent's operations in the second focus window, allowing them to intervene in the second focus window to process tasks when they discover content of interest there; and the user-triggered interaction events are executed first, avoiding operational conflicts and ensuring the stability of the terminal system.

[0102] In some embodiments, the method further includes: issuing a prompt message after the second interactive event is executed, the prompt message indicating that the second interactive event has been completed. Exemplarily, the prompt message may be in the form of text, voice, pop-up window, etc. Exemplarily, the prompt message may also include: the task execution result corresponding to the second interactive event. Based on this, without affecting the user's normal use of the terminal, the intelligent agent can simultaneously execute tasks in the second focus window and obtain task execution results, thereby reducing the user's task processing burden and improving task processing efficiency.

[0103] As can be seen, the above mainly describes the window operation method provided in this disclosure from the perspectives of triggering, distributing, and executing interactive events. To more clearly illustrate the method provided in this disclosure, the creation of the focus window and the maintenance of focus mapping information are briefly introduced below.

[0104] In some embodiments, the method further includes: generating at least one focus identifier when the focus creation conditions are met, and creating at least one second focus window corresponding to the at least one focus identifier, each second focus window corresponding to one focus identifier; and maintaining focus mapping information based on the at least one focus identifier and the window handle of the at least one second focus window.

[0105] The focus creation conditions include at least one of the following: obtaining a focus creation instruction from the user; obtaining a focus request instruction sent by the agent; receiving a first operation instruction input by the user to the agent, and there is no second focus window on the terminal corresponding to the first operation instruction, wherein the first operation instruction is used to instruct the agent to perform an operation on the second focus window.

[0106] The details regarding the window handle can be found in the description of step S101. The window handle of the second focus window can be determined as follows: determine the program corresponding to the second focus window; and determine the window handle of the second focus window based on the program corresponding to the second focus window.

[0107] The focus mapping information includes at least one focus identifier and at least one window handle of a second focus window, with each focus identifier corresponding to a window handle of a second focus window. In some examples, the focus mapping information also includes the focus identifier of a first focus window and the window handle of the first focus window.

[0108] It should be understood that generating at least one focus identifier and creating at least one second focus window corresponding to at least one focus identifier can also be understood as creating one or more sets of focus. A focus usually refers to the central point or focus of a user's current activity. By generating a focus identifier and creating a second focus window corresponding to the focus identifier, at least one such focus can be formed.

[0109] It should be noted that creating a second focus window corresponding to the aforementioned focus identifier does not mean that the terminal must create a new application window.

[0110] For example, the terminal receives a user instruction input by the user to the agent, which instructs the agent to operate on a browser window. If a browser window already exists on the terminal and does not have focus, the terminal generates a focus identifier and sets the browser window to focus. The browser window in focus is then the second focus window corresponding to the focus identifier, and the terminal can create the second focus window without creating a new application window. Alternatively, if no browser window exists on the terminal, the terminal generates a focus identifier, creates a browser window, and sets its focus to focus. The browser window in focus is then the second focus window corresponding to the focus identifier, and the terminal creates the second focus window by creating a new application window.

[0111] Based on this, when creating the second focus window, the window handle and focus identifier of the second focus window can be maintained in the focus mapping information. This allows for the accurate sending of each interactive event to the corresponding second focus window based on the focus mapping information, improving the accuracy of task processing. It also enables the terminal to handle multiple tasks simultaneously without switching focus windows, thus improving task processing efficiency.

[0112] In some embodiments, focus mapping information is stored in the terminal and maintained by the terminal's multifocus management subsystem.

[0113] Considering that the state of the focused window changes as the focus changes (e.g., focus creation, transfer, clearing, etc.), the window handles corresponding to each focus identifier may also change. Therefore, to ensure the accuracy of focus mapping information, this disclosure also provides a window operation method to maintain focus mapping information in real time.

[0114] For example, the above method further includes: monitoring at least one second focus window corresponding to at least one focus identifier; obtaining the updated at least one second focus window when the focus state of the at least one second focus window corresponding to at least one focus identifier changes; and maintaining focus mapping information based on the window handle of the updated at least one second focus window.

[0115] The monitoring of at least one second focus window corresponding to at least one focus identifier can also be understood as tracking the focus state of at least one second focus window corresponding to at least one focus identifier.

[0116] In a specific example, the terminal monitors at least one second focus window corresponding to at least one focus identifier by listening to, detecting, or tracking interaction events.

[0117] For example, if a program is switched by clicking the mouse or switching the keyboard, the terminal updates at least one second focus window based on the switched application and determines the window handle of the updated at least one second focus window in order to maintain focus mapping information.

[0118] For example, for the target focus identifier and the second focus window corresponding to the target focus identifier in the focus mapping information, if the second focus window is the first application window with focus, and it is detected that the agent moves the focus from the first application window to the second application window by simulating a mouse click, then the terminal uses the second application window with focus as the second focus window corresponding to the target focus identifier, and updates the window handle of the second focus window with the window handle of the second application window.

[0119] For example, if the terminal detects that a task in a second focus window has been completed, it responds to the user's command or closes the second focus window by default, and clears the window handle and focus identifier of the second focus window in the focus mapping information.

[0120] Based on this, the focus state of the focus window can be monitored or tracked in real time, and the window handle can be updated in a timely manner to improve the accuracy of the terminal's execution of interactive events, thereby improving the accuracy of task execution.

[0121] To more clearly illustrate the window operation methods provided in this disclosure, Figure 3 The image also shows a terminal display interface that displays a first focus window 310 and a second focus window 320.

[0122] The first focus window 310 contains a first pointer 311, which moves based on user-triggered interaction events, and real mouse events triggered by the user are bound to the first pointer 311.

[0123] The second focus window 320 contains a second pointer 321. The second pointer 321 moves based on interaction events triggered by the agent, and virtual mouse events triggered by the agent are bound to the second pointer 321. It should be noted that there can be one or more second focus windows 320. Figure 3 The quantities shown do not constitute a specific limitation of this disclosure.

[0124] In some embodiments, when it is detected that a user switches applications via mouse clicks or keyboard input, the terminal updates the actually active application corresponding to the first focus window 310 and updates the window handle of the first focus window 310 based on the switched application. Similarly, when it is detected that an agent switches applications via simulated mouse clicks or keyboard input, the terminal updates the actually active application corresponding to the second focus window 320 and updates the window handle of the second focus window 320 based on the switched application.

[0125] In some embodiments, the terminal only allows the user to trigger interaction events in the first focus window 310, and the user-triggered interaction events are always sent to the first focus window 310; the terminal only allows the agent to trigger interaction events in the second focus window 320, and the agent-triggered interaction events are always sent to the second focus window 320.

[0126] Based on this, the operations of the user and the intelligent agent do not interfere with each other, reducing task conflicts. This allows the intelligent agent to perform operations simultaneously in the second focus window 320 while the user is processing tasks, thereby improving task processing efficiency.

[0127] In other embodiments, the terminal allows the user to trigger an interaction event in either focus window, and always sends the interaction event triggered by the user in the first focus window 310 (e.g., the first interaction event) to the first focus window 310, and always sends the interaction event triggered by the user in the second focus window 320 (e.g., the third interaction event) to the second focus window 320; the terminal also allows the agent to trigger an interaction event in the second focus window 320, and always sends the interaction event triggered by the agent in the second focus window 320 (e.g., the second interaction event) to the second focus window 320. In the event of a conflict between the second and third interaction events, the execution of the second interaction event is stopped, and the execution of the third interaction event is performed.

[0128] Based on this, users can intervene in the operation of the agent in the second focus window, and the third interaction event triggered by the user has a higher priority than the second interaction event, so that when interesting content is found in the second focus window, the user can intervene in the second focus window to process the task, thereby improving the flexibility of task processing.

[0129] In some other embodiments, the terminal also allows the agent to trigger interactive events in the first focus window 310, and always sends the interactive events triggered by the agent in the first focus window 310 (such as the fourth interactive event mentioned above) to the first focus window 310. For example, upon receiving a second operation instruction input by the user to the agent, the agent is allowed to trigger interactive events in the first focus window 310, and the second operation instruction is used to instruct the agent to perform an operation on the first focus window. Based on this, the user does not need to operate directly in the first focus window, but instead instructs the agent to complete the task on their behalf, reducing the user's task processing burden and improving processing efficiency.

[0130] In some embodiments, a first focus area 330 and a second focus area 340 may also be displayed on the terminal. The first focus area includes a first focus window 310, and the second focus area includes at least one second focus window 320, so that the focus windows are arranged and displayed in an orderly manner.

[0131] For ease of understanding, a specific embodiment is used below to describe the window operation method provided in this disclosure. The method includes the following steps one through four:

[0132] Step 1: Respond to the first interaction event triggered by the user on the first focused window and execute the first interaction event.

[0133] For example, a user triggers the first interaction event in the first focus window to complete the quarterly report presentation document.

[0134] Step 2: The terminal acquires the user's input command to the agent and determines whether the command is a first operation command or a second operation command. The first operation command instructs the agent to perform an operation on the second focus window, while the second operation command instructs the agent to perform an operation on the first focus window.

[0135] Step 3: If the user instruction is determined to be the first operation instruction and a second focus window corresponding to the first operation instruction exists in the terminal, the intelligent agent performs an operation on the second focus window based on the second operation instruction; or, if the user instruction is determined to be the first operation instruction and a second focus window corresponding to the first operation instruction does not exist in the terminal, the terminal obtains the focus request instruction sent by the intelligent agent and creates a second focus window for the intelligent agent to operate.

[0136] In some examples, the first operation instruction also instructs the agent to collect, retrieve, and analyze market price information for the current and previous quarters by operating on the second focus window, and / or generate market price data charts for the current and previous quarters.

[0137] In other examples, the agent can also determine the action to be taken in the focus window based on user behavior. For instance, the agent can predict the user's action needs based on user behavior in the first focus window, such as retrieving and analyzing market price information for the current and previous quarters, and / or generating data charts of market price information for the current and previous quarters.

[0138] Therefore, while the user fills in the content framework and writes the page in the first focus window, the terminal can automatically perform operations in the second focus window through the intelligent agent, thereby improving the efficiency of the user in completing the quarterly report presentation document.

[0139] Step 4: After the second interactive event is completed, a prompt message is issued to indicate that the second interactive event has been completed.

[0140] In some examples, the prompt message may also include the task execution result corresponding to the second interactive event. For example, it may include charts showing market price data for the current and previous quarters.

[0141] Therefore, without affecting the user's normal use of the terminal, the intelligent agent can simultaneously execute tasks in the second focus window and obtain the task execution results, thereby reducing the user's task processing pressure and improving task processing efficiency.

[0142] It is understood that, in order to achieve the aforementioned functions, the device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0143] like Figure 4 As shown in the figure, this disclosure also provides a schematic diagram of a terminal structure, including a window display module 410, a task processing module 420, and an intelligent agent 430. In some embodiments, the terminal further includes a focus maintenance module 440.

[0144] The window display module 410 is used to display the first focus window and the second focus window.

[0145] The task processing module 420 is used to respond to the first interaction event triggered by the user on the first focus window and execute the first interaction event.

[0146] The task processing module 420 is also used to execute the second interaction event in response to the second interaction event triggered by the agent 430 for the second focus window.

[0147] The intelligent agent 430 is used to respond to user commands or autonomously perform operations in the second focus window based on user behavior.

[0148] In some embodiments, the task processing module 420 is further configured to acquire a first operation instruction input by the user to the agent 430, the first operation instruction being used to instruct the agent 430 to perform an operation on the second focus window; in response to the first operation instruction, triggering a second interaction event for the second focus window through the agent 430.

[0149] In some embodiments, the task processing module 420 is further configured to determine the user's behavior in the first focus window; predict the user's operation needs for the second focus window through the intelligent agent 430 based on the user's behavior in the first focus window; and trigger a second interaction event for the second focus window through the intelligent agent 430 based on the operation needs.

[0150] In some embodiments, the task processing module 420 is further configured to, during the execution of the second interactive event, acquire a third interactive event triggered by the user in the second focus window; in response to the third interactive event triggered by the user in the second focus window, stop executing the second interactive event and execute the third interactive event.

[0151] In some embodiments, the task processing module 420 is further configured to issue a prompt message after the second interactive event is executed, the prompt message being used to indicate that the second interactive event has been completed.

[0152] In some embodiments, the task processing module 420 is specifically configured to, in response to a second interaction event triggered by the agent 430 for the second focus window, determine the window handle of the second focus window; based on the window handle of the second focus window, send the second interaction event to the program corresponding to the second focus window, and execute the second interaction event through the program corresponding to the second focus window.

[0153] In some embodiments, the window display module 410 is specifically used to display a first focus window and at least one second focus window; the task processing module 420 is specifically used to respond to a second interaction event triggered by the agent 430 for at least one second focus window, and determine the window handle of the target focus window based on the focus identifier of the second interaction event and preset focus mapping information; wherein, the target focus window is the second focus window corresponding to the window handle of at least one second focus window and the focus identifier of the second interaction event; based on the window handle of the target focus window, the second interaction event is sent to the program corresponding to the target focus window, and the second interaction event is executed by the program corresponding to the target focus window; wherein, the focus mapping information includes at least one focus identifier and at least one window handle of the second focus window, and each focus identifier corresponds to a window handle of the second focus window.

[0154] In some embodiments, the focus maintenance module 440 is used to generate at least one focus identifier and create at least one second focus window corresponding to the at least one focus identifier when the focus creation conditions are met, with each second focus window corresponding to one focus identifier; and maintain focus mapping information based on the at least one focus identifier and the window handle of the at least one second focus window; wherein the focus creation conditions can be referred to the description above.

[0155] In some embodiments, the focus maintenance module 440 is further configured to monitor at least one second focus window corresponding to at least one focus identifier; when the focus state of at least one second focus window corresponding to at least one focus identifier changes, obtain the updated at least one second focus window; and maintain focus mapping information based on the window handle of the updated at least one second focus window.

[0156] In some embodiments, the task processing module 420 is further configured to acquire a second operation instruction input by the user to the agent 430, the second operation instruction being used to instruct the agent 430 to perform an operation on the first focus window; in response to the second operation instruction, trigger a fourth interaction event for the first focus window through the agent 430; and in response to the fourth interaction event triggered by the agent 430 for the first focus window, execute the fourth interaction event.

[0157] It should be noted that, Figure 4 The module division shown is illustrative and represents only one logical functional division; in actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single processing module. These integrated modules can be implemented either in hardware or as software functional modules.

[0158] In implementing the functions of the integrated modules described above using hardware, this disclosure also provides a possible structure for an electronic device used to execute the window operation method provided in this disclosure. Similarly, the electronic device and the window operation method described above can be referred to in correspondence with each other.

[0159] like Figure 5 As shown, the electronic device includes a processor 502 and a communication interface 503. In some examples, the electronic device may also include at least one of a bus 504 and a memory 501.

[0160] Processor 502 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 502 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 502 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0161] Communication interface 503 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0162] The memory 501 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0163] In one possible implementation, the memory 501 can exist independently of the processor 502. The memory 501 can be connected to the processor 502 via a bus 504 and is used to store instructions or program code executable by the processor 502, such as computer program instructions. When the processor 502 calls and executes the instructions or program code stored in the memory 501, it can implement the window operation method provided in this embodiment.

[0164] In another possible implementation, the memory 501 can also be integrated with the processor 502.

[0165] Bus 504 can be an extended industry standard architecture (EISA) bus, etc. Bus 504 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0166] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform the window operation method as described in any of the above embodiments. It should be understood that this disclosure does not limit the specific form of the computer.

[0167] In some examples, the aforementioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage media" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0168] This disclosure provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the window operation method described in any of the above embodiments.

Claims

1. A window operation method, characterized by, The method applied to a terminal comprises: displaying a first focus window and a second focus window; in response to a first interaction event triggered by a user for the first focus window, executing the first interaction event; in response to a second interaction event triggered by an intelligent agent for the second focus window, executing the second interaction event; wherein the intelligent agent executes operations in the second focus window in response to a user instruction or autonomously according to user behavior.

2. The method of claim 1, wherein, The method further comprises: obtaining a first operation instruction input by the user to the intelligent agent, the first operation instruction being used to instruct the intelligent agent to operate on the second focus window; in response to the first operation instruction, triggering a second interaction event by the intelligent agent for the second focus window.

3. The method of claim 1, wherein, The method further comprises: determining user behavior of the user in the first focus window; predicting, by the intelligent agent, operation requirements of the user for the second focus window according to the user behavior of the user in the first focus window; based on the operation requirements, triggering a second interaction event by the intelligent agent for the second focus window.

4. The method of claim 1, wherein, The method further comprises: in a process of executing the second interaction event, obtaining a third interaction event triggered by the user for the second focus window; in response to the third interaction event triggered by the user for the second focus window, stopping execution of the second interaction event and executing the third interaction event.

5. The method of claim 1, wherein, The method further comprises: after the second interaction event is executed, issuing a prompt information, the prompt information being used to prompt that the second interaction event has been executed.

6. The method of claim 1, wherein, The response to the second interaction event triggered by the intelligent agent for the second focus window, executing the second interaction event, comprises: in response to the second interaction event triggered by the intelligent agent for the second focus window, determining a window handle of the second focus window; based on the window handle of the second focus window, sending the second interaction event to a program corresponding to the second focus window, and executing the second interaction event by the program corresponding to the second focus window.

7. The method of claim 1, wherein, The display of the first focus window and the second focus window comprises: displaying a first focus window and at least one second focus window; The response to the second interaction event triggered by the intelligent agent for the second focus window, executing the second interaction event, comprises: in response to the second interaction event triggered by the intelligent agent for the at least one second focus window, based on a focus identifier of the second interaction event and preset focus mapping information, determining a window handle of a target focus window; wherein the target focus window is a second focus window in the at least one second focus window, the window handle of which corresponds to the focus identifier of the second interaction event; based on the window handle of the target focus window, sending the second interaction event to a program corresponding to the target focus window, and executing the second interaction event by the program corresponding to the target focus window; The focus mapping information includes the at least one focus identifier and a window handle of the at least one second focus window, and each focus identifier corresponds to a window handle of a second focus window.

8. The method of claim 7, wherein, The method further includes: In a case where a focus creation condition is met, generating at least one focus identifier, and creating the at least one second focus window corresponding to the at least one focus identifier, each second focus window corresponding to a focus identifier; Based on the at least one focus identifier and a window handle of the at least one second focus window, maintaining the focus mapping information; The focus creation condition includes at least one of the following: Obtaining a focus creation instruction of a user; Obtaining a focus application instruction sent by the agent; Receiving a first operation instruction input by a user to the agent, and the terminal does not have a second focus window corresponding to the first operation instruction, the first operation instruction being used to instruct the agent to perform an operation on a second focus window.

9. The method of claim 8, wherein, The method further includes: Monitoring the at least one second focus window corresponding to the at least one focus identifier; In a case where a focus state of the at least one second focus window corresponding to the at least one focus identifier changes, obtaining updated at least one second focus window; Based on a window handle of the updated at least one second focus window, maintaining the focus mapping information.

10. The method of claim 1, wherein, The method further includes: Obtaining a second operation instruction input by a user to the agent, the second operation instruction being used to instruct the agent to perform an operation on the first focus window; In response to the second operation instruction, triggering, by the agent, a fourth interaction event for the first focus window; In response to the fourth interaction event triggered by the agent for the first focus window, executing the fourth interaction event.

11. An electronic device, comprising: includes: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instructions, so that the electronic device performs the window operation method according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and when the computer instructions run on a computer, the computer executes the window operation method according to any one of claims 1-10.

13. A computer program product, characterised in that, The computer program product contains computer instructions, and when the computer instructions run on a computer, the computer executes the window operation method according to any one of claims 1-10.

Images