Window operation method, electronic device, storage medium and program product

Abstract

Provided in the embodiments of the present disclosure are a window operation method, an electronic device, a storage medium and a program product. The method is applied to a terminal, and comprises: displaying a first focus window and a second focus window; in response to a first interaction event triggered by a user for the first focus window, executing the first interaction event; and in response to a second interaction event triggered by an agent for the second focus window, executing the second interaction event, wherein the agent autonomously executes an operation in the second focus window in response to a user instruction or on the basis of a user behavior.

Description

Window operation methods, electronic devices, storage media and program products

[0001] This disclosure claims priority to Chinese patent application No. 202410954120.9, filed on July 15, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of communication technology, and in particular to a window operation method, electronic device, storage medium, and program product. Background Technology

[0003] With the continuous development of the smart terminal industry, people's reliance on and demand for mobile phones and other terminal devices are constantly increasing. To avoid task conflicts, the user interface operating system of the terminal (such as Windows, macOS, Linux, and other desktop operating systems on mobile phones and computers) usually adopts a single-focus operation mode for task processing. That is, only one active window exists in a single interface operating system at any given time. Summary of the Invention

[0004] This disclosure provides a window operation method, an electronic device, a storage medium, and a program product.

[0005] The technical solution adopted in this disclosure is as follows:

[0006] On one hand, embodiments of this disclosure provide a window operation method applied to a terminal, the method comprising:

[0007] In response to a first interaction event triggered by the user in relation to the first focused window, execute the first interaction event;

[0008] In response to a second interaction event triggered by the agent in relation to the second focus window, the agent executes the second interaction event; the agent responds to user instructions or autonomously performs operations in the second focus window based on user behavior.

[0009] In another aspect, embodiments of this disclosure provide an electronic device, including: a processor and a memory for storing processor-executable instructions; enabling the electronic device to implement any of the window operation methods provided in embodiments of this disclosure.

[0010] In another aspect, embodiments of this disclosure provide a computer-readable storage medium storing computer program instructions, which, when executed on a computer, cause the computer to implement any of the window operation methods provided in embodiments of this disclosure.

[0011] In another aspect, embodiments of this disclosure provide a computer program product, which includes computer program instructions that, when executed on a computer, cause the computer to implement any of the window operation methods provided in embodiments of this disclosure. Attached Figure Description

[0012] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0013] Figure 1A is a block diagram of a terminal according to some embodiments of the present disclosure.

[0014] Figure 1B is another block diagram of a terminal according to some embodiments of the present disclosure.

[0015] Figure 1C is yet another block diagram of a terminal according to some embodiments of the present disclosure.

[0016] Figure 2 is a flowchart of a window operation method according to some embodiments of the present disclosure.

[0017] Figure 3 is a schematic diagram of the display interface of a terminal according to some embodiments of the present disclosure.

[0018] Figure 4 is yet another block diagram of a terminal according to some embodiments of the present disclosure.

[0019] Figure 5 is a block diagram of an electronic device according to some embodiments of the present disclosure. Detailed Implementation

[0020] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0021] Unless the context otherwise requires, throughout the specification and claims, the term "comprise" and its other forms, such as the third-person singular "comprises" and the present participle "comprising," are interpreted as open and encompassing, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples.

[0022] The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with terms such as "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0023] In this disclosure, the terms "exemplarily" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplarily" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplarily" or "for example" is intended to present the relevant concepts by way of example.

[0024] In addition, the use of “based on” implies openness and inclusivity, because processes, steps, calculations or other actions “based on” one or more of the stated conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated.

[0025] With the continuous development of the smart terminal industry, people's reliance on and demand for mobile phones and other terminal devices are constantly increasing. To avoid task conflicts, the user interface operating system of the terminal (such as Windows, macOS, Linux, and other desktop operating systems on mobile phones and computers) usually adopts a single-focus operation mode for task processing. That is, only one active window exists in a single interface operating system at any given time.

[0026] However, this operating method limits the number of tasks a terminal can handle, resulting in low operational performance and low task processing efficiency. Improving the operational performance and task processing efficiency of the terminal has become an urgent technical problem to be solved.

[0027] To address this issue, this disclosure provides a window operation method applied to a terminal. The method includes: displaying a first focus window and a second focus window; executing a first interaction event in response to a first interaction event triggered by a user on the first focus window; and executing a second interaction event in response to a second interaction event triggered by an intelligent agent on the second focus window. In some embodiments, the intelligent agent autonomously performs operations in the second focus window in response to user instructions or based on user behavior. This improves the terminal's operational performance and task processing efficiency.

[0028] For ease of understanding, one type of terminal involved in this disclosure will be introduced below.

[0029] As shown in Figures 1A to 1C, the terminal includes an artificial intelligence (AI) subsystem 110 and a multi-focus management subsystem 120. In some embodiments, the terminal further includes a user operation detection subsystem 130 and / or a control subsystem 140.

[0030] In some embodiments, the AI ​​subsystem 110 employs AI technology, possesses AI computing resources, or has a communication connection with the network side, enabling it to perform AI calculations based on the network-side AI computing resources. The AI ​​subsystem 110 includes an intelligent agent, which may also be referred to as an AI module, AI assistant, AI agent, AI unit, or AI intelligent agent, etc., and this disclosure does not limit this terminology. The intelligent agent must at least have interface operation capabilities, such as simulating a real person browsing the web.

[0031] In some embodiments, the intelligent agent has at least one of the following capabilities: predicting the user's operational needs for the second focus window based on user behavior; responding to user instructions or autonomously performing operations in the second focus window based on user behavior; simulating real-person operations to trigger interactive events (such as the second interactive events described below) in the second focus window; outputting the task processing result in the second focus window; and outputting prompt information to indicate that the task in the second focus window has been completed. In some embodiments, the meaning of the second focus window is detailed in the description of the method embodiments below.

[0032] The multi-focus management subsystem 120 is used to manage or maintain one or more focus windows in a terminal (e.g., a first focus window and at least one second focus window, or more specifically, at least one second focus window).

[0033] In some embodiments, the multi-focus management subsystem 120 has at least one of the following functions: separating the event source of an interactive event; distributing the interactive event to the corresponding focus window; and maintaining focus mapping information. In some embodiments, the focus mapping information includes at least one focus identifier and at least one window handle of a second focus window, with each focus identifier corresponding to a window handle of a second focus window.

[0034] The user operation detection subsystem 130 has at least one of the following functions: detecting user operations and / or detecting simulated operations of intelligent agents; acquiring interaction events (e.g., a first interaction event, a second interaction event, a third interaction event, or a fourth interaction event); and monitoring the status and / or position of each focus window (e.g., a second focus window).

[0035] The control subsystem 140 has at least one of the following functions: acquiring user instructions; forwarding user instructions to the AI ​​subsystem 110 (e.g., the agent in the AI ​​subsystem 110) or the multi-focus management subsystem 120; and issuing control instructions to at least one of the AI ​​subsystem 110 (e.g., the agent in the AI ​​subsystem 110), the multi-focus management subsystem 120, or the user operation detection subsystem 130.

[0036] In some embodiments, the terminal has at least AI computing capabilities to support the operation of intelligent agents. In some embodiments, the terminal has local AI computing capabilities; and / or, the terminal implements AI computing capabilities based on AI computing resources located at the network end.

[0037] In some embodiments, the terminal also has display capabilities. Or, in other words, the terminal has display space.

[0038] It should be noted that in the window operation method provided in this disclosure, the terminal can be a personal computer (PC) device, user equipment (UE), handheld device with various communication functions, vehicle device, wearable device, computer, smart home device or smart office device, etc., and this disclosure does not limit it.

[0039] It should be understood that the above terminal examples are merely for illustrating the technical solutions of this disclosure more clearly and do not constitute a limitation of this disclosure. Those skilled in the art will recognize that, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in this disclosure are equally applicable to similar technical problems.

[0040] To more clearly illustrate the window operation method provided in this disclosure, the window transmission method provided in this disclosure is described below with reference to the accompanying drawings. It should be noted that the various embodiments of this disclosure can be mutually referenced or consulted. For example, the same or similar steps, method embodiments, and device embodiments can be mutually referenced, and this disclosure does not limit them in this regard.

[0041] As shown in Figure 2, this embodiment of the present disclosure provides a window operation method applied to a terminal, the method including:

[0042] In S101, the first focus window and the second focus window are displayed.

[0043] In some embodiments, the first focus window receives at least real interaction events triggered by the user (such as the first interaction event described below). For example, the first focus window may receive interaction events from the user interacting with the system via a mouse, touchscreen, or keyboard. It should be understood that the first focus window corresponds to input from a real mouse and a real keyboard.

[0044] It should be noted that at least one primary focus window is present and displayed by default in the terminal to ensure that there is at least one focus window that the user can directly interact with.

[0045] The second focus window receives at least the virtual interaction events triggered by the agent (such as the second interaction event described below). The agent has interface operation capabilities and can trigger virtual mouse events, virtual touch events, or other virtual input events to replace or assist the user in performing operations. The capabilities of the agent can be referred to the relevant textual descriptions in Figures 1A to 1C above, or to the descriptions in other method embodiments below.

[0046] There can be one or more secondary focus windows. When there are multiple secondary focus windows, there are multiple focuses in the terminal for the agent to operate on. The agent can process tasks in multiple secondary focus windows simultaneously, thereby replacing or assisting the user in completing more tasks and improving task processing efficiency.

[0047] It should be noted that the number of second focus windows mentioned above is based on the premise that the second focus window has already been created. If the terminal does not actively create a second focus window, it may not exist on the terminal. The terminal only creates and displays the second focus window when the focus creation conditions are met. In some embodiments, the focus creation conditions and the method of creating the second focus window can be referred to the description in the following method embodiments.

[0048] To more clearly illustrate the window operation methods provided in this disclosure, the following is a brief introduction to the concepts of focus window (e.g., first focus window and / or second focus window) and focus.

[0049] The first and second focus windows mentioned above are both focus windows, which are application windows in an active state. This active state can also be understood as having focus or being in a focus state. In some embodiments, focus typically refers to the user's current activity center or focus point. When an application window has focus, it can be considered that the application window or the program corresponding to the application window is in a focus state and can be directly operated by the user or intelligent agent, for example, it can directly receive and execute interactive events triggered by the user or intelligent agent. It should be understood that when the terminal displays the first and second focus windows, at least two focus points exist simultaneously on the terminal, thereby enabling multi-focus operation.

[0050] It is understood that, in addition to the aforementioned focused window, there may be other application windows in the terminal that do not have focus, such as background applications that are inactive. This disclosure does not limit this.

[0051] In some embodiments, each focused window corresponds to a unique focus identifier and window handle. The terminal maintains the focus identifier and window handle of each focused window in real time.

[0052] In some embodiments, the focus identifier of the focus window may also be referred to as the window identifier or focus number, etc., to uniquely identify the focus corresponding to the focus window. The concept of focus can be referred to the description above.

[0053] In some embodiments, the window handle of the focused window is used to uniquely identify the program corresponding to the focused window.

[0054] To accurately distribute interactive events, the terminal can also maintain relevant information about each focused window (such as the window handle and / or focus marker of the focused window) to accurately distribute interactive events based on this information. For example, the terminal can monitor or track the state of each focused window in real time to update the window handle of each focused window. An example of how this update and maintenance is performed can be found in the description below.

[0055] In S102, in response to the first interaction event triggered by the user for the first focus window, the first interaction event is executed.

[0056] The first interaction event, also known as the first focus event, is the actual interaction event triggered by the user.

[0057] In some embodiments, the terminal sends a first interaction event to the program corresponding to the first focus window based on the window handle of the first focus window, and the program corresponding to the first focus window executes the first interaction event. It should be understood that the program corresponding to the first focus window is an active application that only receives interaction events triggered on that first focus window.

[0058] In some embodiments, the terminal can execute the first interaction event based on the type of the first interaction event. For example, if the type of the first interaction event is a scrolling event, the terminal controls the scrolling of the display page on the program corresponding to the first focus window. As another example, if the type of the first interaction event is a keyboard event, the terminal receives text input from the keyboard event and transmits the text input to the program corresponding to the first focus window.

[0059] Based on step S102, the first interactive event can be accurately sent to the corresponding program execution through the window handle.

[0060] In S103, in response to the second interaction event triggered by the agent for the second focus window, the second interaction event is executed; wherein, the agent responds to user instructions or autonomously performs operations in the second focus window according to user behavior.

[0061] Here, the intelligent agent uses AI technology and has AI computing capabilities. It can also be called an AI module, AI assistant, AI agent, AI unit, or AI intelligent agent, etc., and can autonomously perform operations in the second focus window.

[0062] In some embodiments, the intelligent agent achieves AI computing capabilities based on the terminal's local AI computing resources (e.g., AI resources in the terminal's AI subsystem) or based on network-side AI computing resources.

[0063] In some embodiments, the intelligent agent has interface operation capabilities. For example, the intelligent agent can trigger interactive events (such as a second interactive event) to perform actions such as swiping, selecting, and scrolling down on web pages, simulating human operation for web browsing or web browsing. As another example, the intelligent agent can use image recognition, text recognition, or similarity comparison to filter online resources such as items, services, or stores that meet certain requirements from a web page; in some embodiments, the intelligent agent can also determine the filtered results and output the results.

[0064] In some embodiments, the intelligent agent has at least one of the following capabilities: predicting the user's operational needs for the second focus window based on user behavior; responding to user instructions or autonomously performing operations in the second focus window based on user behavior; simulating real-person operations and triggering interactive events (e.g., second interactive events) in the second focus window; outputting the task processing results in the second focus window; and outputting prompt information to indicate that the task in the second focus window has been completed.

[0065] In some embodiments, the intelligent agent can also acquire local data from the terminal for task processing. For example, it can read and generate the data table required by the user in the background from the local computer file system of the terminal, based on the data storage location that has been informed in advance or automatically recorded before.

[0066] It should be understood that an intelligent agent can simulate a real person's actions in a focused window (e.g., a second focused window) without altering the third-party application. In some cases, because some software does not publicly expose its data query interface (e.g., real-time product prices in some software are not directly accessible for querying), an intelligent agent can simulate real-person actions to browse web pages and filter required data, thereby increasing the channels for obtaining data information and improving task processing efficiency. In some embodiments, the intelligent agent simulates real-person actions by triggering interaction events (e.g., a second interaction event).

[0067] Here, the second interaction event can also be called the second focus event, which is a virtual interaction event triggered by the agent.

[0068] In some embodiments, the second interaction event is triggered by the agent in response to a user command and / or based on user behavior. The triggering methods of the second interaction event are briefly described below.

[0069] (1) The second interaction event is triggered by the agent responding to the user's command.

[0070] In some embodiments, the user instruction includes a first operation instruction; the method further includes: the terminal acquiring the first operation instruction input by the user to the agent, the first operation instruction being used to instruct the agent to perform an operation on the second focus window; and in response to the first operation instruction, triggering a second interaction event through the agent on the second focus window.

[0071] In some embodiments, the first operation instruction may be input to the terminal via any of the following methods: keyboard, mouse, text, instruction button, or voice instruction.

[0072] In some embodiments, the terminal displays an application interface for controlling the intelligent agent, so that the user can select or input a first operation command based on the application interface.

[0073] In some embodiments, the application interface for controlling the agent described above may display tasks for orchestrating the agent based on user operations. For example, the application interface may display multiple draggable components and an orchestration area, each component corresponding to a functional node; tasks of the agent are orchestrated based on at least one component dragged by the user to the orchestration area; the tasks of the agent are issued to the agent through a first operation instruction, so that the agent can operate in a second focus window based on the tasks of the agent.

[0074] In some embodiments, the second focus window indicated by the first operation instruction already exists in the terminal.

[0075] In other embodiments, if the second focus window indicated by the first operation instruction does not exist on the terminal, the terminal generates a focus identifier and creates a second focus window corresponding to the focus identifier so that the agent can perform operations on the second focus window.

[0076] Based on this, the operation of the intelligent agent can be flexibly determined based on user instructions, so as to flexibly control the intelligent agent to complete the task on behalf of the user in the second focus window, thereby improving the efficiency of task processing.

[0077] (2) The second interaction event is triggered based on user behavior.

[0078] In some embodiments, the method further includes: the terminal determining the user's behavior in the first focus window; predicting the user's operational needs for the second focus window through an intelligent agent based on the user's behavior in the first focus window; and triggering a second interaction event for the second focus window through the intelligent agent based on the operational needs.

[0079] In some embodiments, the terminal determines user behavior based on at least one of the following: a first interaction event triggered by the user in the first focus window, or the display information of the first focus window.

[0080] In some embodiments, user behavior can also be understood as the user's behavioral intent or the user's behavioral purpose.

[0081] For example, user behavior could be: completing a presentation document for a quarterly report. In some embodiments, the operational demand predicted based on this user behavior could be: retrieving and analyzing market price information for the current and previous quarters, and / or generating data charts of market price information for the current and previous quarters. Thus, while the user fills in the content framework and writes the page in the first focus window, the terminal can automatically perform operations in the second focus window via an intelligent agent, thereby improving the efficiency of the user completing the presentation document for the quarterly report.

[0082] For example, user behavior could be: writing a paper manuscript. In some embodiments, the operational requirement predicted based on this user behavior could be: collecting reference papers.

[0083] For example, user behavior could be: developing a team-building plan. In some embodiments, the operational need predicted based on this user behavior could be: finding and filtering the most cost-effective merchants.

[0084] Based on this, the terminal can predict operational needs based on user behavior through an intelligent agent, thereby processing tasks synchronously through the intelligent agent in the second focus window, improving task processing efficiency and reducing the user's task processing pressure.

[0085] It should be noted that the above-mentioned triggering method for the second interactive event is only an example, and there may be other triggering methods, such as the agent triggering the second interactive event based on a preset target, etc. This disclosure does not limit this.

[0086] In some embodiments, step S103 includes: in response to a second interaction event triggered by the agent for the second focus window, determining the window handle of the second focus window; based on the window handle of the second focus window, sending the second interaction event to the program corresponding to the second focus window, and executing the second interaction event through the program corresponding to the second focus window. Based on this, the second interaction event can be accurately sent to the corresponding program for execution through the window handle.

[0087] It should be understood that the program corresponding to the second focus window is an active application that only receives interactive events triggered on that second focus window. In some embodiments, the terminal may also execute the second interactive event based on the event type of the second interactive event.

[0088] In some embodiments, the number of second focus windows is at least one. When there are multiple second focus windows, before executing the second interactive event, the terminal (e.g., a multi-focus management subsystem in the terminal) determines the target focus window corresponding to the second interactive event from among the multiple second focus windows and distributes the second interactive event to the target focus window so that the second interactive event can be executed through the program corresponding to the target focus window.

[0089] In some embodiments, when step S101 includes displaying a first focus window and at least one second focus window, step S103 includes: in response to a second interaction event triggered by the agent for at least one second focus window, determining the window handle of a target focus window based on the focus identifier of the second interaction event and preset focus mapping information; in some embodiments, the target focus window is the second focus window whose window handle corresponds to the focus identifier of the second interaction event; based on the window handle of the target focus window, sending the second interaction event to the program corresponding to the target focus window, and executing the second interaction event through the program corresponding to the target focus window; in some embodiments, the focus mapping information includes at least one focus identifier and at least one window handle of a second focus window, with each focus identifier corresponding to a window handle of a second focus window.

[0090] Based on this, the second interactive event can be accurately sent to the corresponding program in the second focus window for execution by using the focus identifier, thus improving the accuracy of task execution. This allows the terminal to handle tasks simultaneously in multiple focus windows without switching focus windows, improving task processing efficiency.

[0091] It should be noted that steps S102 and S103 above can be executed in parallel, allowing the user to perform tasks in the first focus window while the intelligent agent operates simultaneously with the user in the second focus window. This allows the intelligent agent to simulate a real person handling tasks on behalf of the user in the second focus window. Thus, without affecting the user's normal use of the terminal, the user can also observe the intelligent agent's task processing process through the second focus window, improving the user experience.

[0092] The window operation method provided in this disclosure, on the one hand, allows multiple focus windows (e.g., a first focus window and a second focus window) to exist on the terminal, enabling the terminal to process multiple tasks in parallel; on the other hand, it allows the user to process tasks in the first focus window while the intelligent agent can autonomously execute operations in the second focus window, realizing collaborative task processing between the user and the intelligent agent, saving user time and reducing the user's task processing burden; furthermore, considering that artificial intelligence has more efficient and accurate processing capabilities than users, it improves the accuracy and efficiency of task processing. In summary, the window operation method provided in this disclosure improves the terminal's operational performance and task processing efficiency.

[0093] In some embodiments, the first focus window can also receive virtual interaction events triggered by the agent, meaning the agent can autonomously perform operations within the first focus window. This allows the agent to perform operations in various focus windows (e.g., the first focus window and / or the second focus window) on behalf of or assist the user. Consequently, task processing can be completely or partially transferred to the agent, reducing the user's task processing burden and improving task processing efficiency.

[0094] For example, the above method further includes: obtaining a second operation instruction input by the user to the agent, the second operation instruction being used to instruct the agent to perform an operation on the first focus window; in response to the second operation instruction, triggering a fourth interaction event through the agent on the first focus window; and in response to the fourth interaction event triggered by the agent on the first focus window, executing the fourth interaction event.

[0095] In some embodiments, the terminal may also display an application interface for controlling the intelligent agent, allowing the user to select or input a second operation command based on the application interface. In some embodiments, the application interface for controlling the intelligent agent may also be used to orchestrate tasks for the intelligent agent based on user operations.

[0096] Based on this, an intelligent agent can also be implemented in the first focus window to replace or assist the user in performing operations, thereby reducing the user's task processing pressure and improving processing efficiency.

[0097] It should be understood that when a terminal's agent can autonomously perform operations in both the first and second focus windows, after receiving user commands input by the user, the agent can first determine whether to execute the operation directly in the first or second focus window to decide whether a new focus needs to be created. It should be noted that the focus here refers to a point of interest that can be manipulated by the user, created in the form of a focus window.

[0098] For example, if a user instruction (such as the second operation instruction mentioned above) instructs the agent to perform an operation on an existing first focus window, the terminal does not need to create a new focus window.

[0099] For example, if a user command instructs an agent to perform an operation on a second focus window, and that second focus window does not exist on the terminal, the terminal creates a new focus window.

[0100] This enables the agent to operate in both the first and second focus windows, allowing users to flexibly manage the agent and enriching the user experience.

[0101] Furthermore, considering that the agent operates in the second focus window, this window may contain content of interest to the user. Therefore, in some embodiments, the second focus window can also be configured to receive real interaction events triggered by the user, allowing the user to operate directly within the second focus window, thereby improving the user experience.

[0102] To avoid operational conflicts when the user and the agent simultaneously perform operations in the second focus window, this disclosure also provides a window operation method, including: acquiring a third interaction event triggered by the user in the second focus window during the execution of a second interaction event; stopping the execution of the second interaction event and executing the third interaction event in response to the third interaction event triggered by the user in the second focus window. Based on this, the user can interfere with the agent's operations in the second focus window, allowing them to intervene in the second focus window to process tasks when they discover content of interest there; and the user-triggered interaction events are executed first, avoiding operational conflicts and ensuring the stability of the terminal system.

[0103] In some embodiments, the method further includes: issuing a prompt message after the second interactive event is executed, the prompt message indicating that the second interactive event has been completed. For example, the prompt message may be in the form of text, voice, pop-up window, etc. In some embodiments, the prompt message may also include: the task execution result corresponding to the second interactive event. Based on this, without affecting the user's normal use of the terminal, the intelligent agent can simultaneously execute tasks in the second focus window and obtain task execution results, thereby reducing the user's task processing pressure and improving task processing efficiency.

[0104] As can be seen, the above mainly describes the window operation method provided in this disclosure from the perspectives of triggering, distributing, and executing interactive events. To more clearly illustrate the method provided in this disclosure, the creation of the focus window and the maintenance of focus mapping information are briefly introduced below.

[0105] In some embodiments, the method further includes: generating at least one focus identifier when the focus creation conditions are met, and creating at least one second focus window corresponding to the at least one focus identifier, each second focus window corresponding to a focus identifier; and maintaining focus mapping information based on the at least one focus identifier and the window handle of the at least one second focus window.

[0106] In some embodiments, the focus creation conditions include at least one of the following: obtaining a focus creation instruction from a user; obtaining a focus request instruction sent by an agent; or receiving a first operation instruction input by a user to an agent, and there is no second focus window on the terminal corresponding to the first operation instruction. In some embodiments, the first operation instruction is used to instruct the agent to perform an operation on the second focus window.

[0107] For details regarding the window handle, please refer to the description in step S101. The window handle of the second focus window can be determined in the following way: determine the program corresponding to the second focus window; and determine the window handle of the second focus window based on the program corresponding to the second focus window.

[0108] The focus mapping information includes at least one focus identifier and at least one window handle of a second focus window, with each focus identifier corresponding to a window handle of a second focus window. In some embodiments, the focus mapping information further includes a focus identifier of a first focus window and a window handle of the first focus window.

[0109] It should be understood that generating at least one focus identifier and creating at least one second focus window corresponding to at least one focus identifier can also be understood as creating one or more sets of focus. A focus usually refers to the central point or focus of a user's current activity. By generating a focus identifier and creating a second focus window corresponding to the focus identifier, at least one such focus can be formed.

[0110] It should be noted that creating a second focus window corresponding to the aforementioned focus identifier does not mean that the terminal must create a new application window.

[0111] In some embodiments, the terminal receives a user instruction input by the user to the agent, which instructs the agent to operate on a browser window. If a browser window already exists on the terminal and does not have focus, the terminal generates a focus identifier and sets the browser window to focus state. The browser window in focus state is the second focus window corresponding to the focus identifier, and the terminal can create the second focus window without creating a new application window. Alternatively, if no browser window exists on the terminal, the terminal generates a focus identifier, creates a browser window, and sets the browser window to focus state. The browser window in focus state is the second focus window corresponding to the focus identifier, and the terminal creates the second focus window by creating a new application window.

[0112] Based on this, when creating the second focus window, the window handle and focus identifier of the second focus window can be maintained in the focus mapping information. This allows for the accurate sending of each interactive event to the corresponding second focus window based on the focus mapping information, improving the accuracy of task processing. This enables the terminal to handle multiple tasks simultaneously without switching focus windows, thereby improving task processing efficiency.

[0113] In some embodiments, focus mapping information is stored in the terminal and maintained by the terminal's multifocus management subsystem.

[0114] Considering that the state of the focused window changes with the change of focus (e.g., focus creation, transfer, clearing, etc.), the window handles corresponding to each focus identifier may also change. Therefore, to ensure the accuracy of focus mapping information, this disclosure also provides a window operation method to maintain focus mapping information in real time.

[0115] In some embodiments, the method further includes: monitoring at least one second focus window corresponding to at least one focus identifier; obtaining the updated at least one second focus window when the focus state of the at least one second focus window corresponding to at least one focus identifier changes; and maintaining focus mapping information based on the window handle of the updated at least one second focus window.

[0116] The above monitoring of at least one second focus window corresponding to at least one focus identifier can also be understood as tracking the focus state of at least one second focus window corresponding to at least one focus identifier.

[0117] In some embodiments, the terminal monitors at least one second focus window corresponding to at least one focus identifier by listening to, detecting, or tracking interaction events.

[0118] For example, if a program is switched by clicking the mouse or switching the keyboard, the terminal updates at least one second focus window based on the switched application and determines the window handle of the updated at least one second focus window in order to maintain focus mapping information.

[0119] For example, for the target focus identifier and the second focus window corresponding to the target focus identifier in the focus mapping information, if the second focus window is the first application window with focus, and it is detected that the agent moves the focus from the first application window to the second application window by simulating a mouse click, then the terminal uses the second application window with focus as the second focus window corresponding to the target focus identifier, and updates the window handle of the second focus window with the window handle of the second application window.

[0120] For example, if the terminal detects that a task in a second focus window has been completed, it responds to the user's command or closes the second focus window by default, and clears the window handle and focus identifier of the second focus window in the focus mapping information.

[0121] Based on this, the focus state of the focus window can be monitored or tracked in real time, and the window handle can be updated in a timely manner to improve the accuracy of the terminal's execution of interactive events, thereby improving the accuracy of task execution.

[0122] To more clearly illustrate the window operation method provided in this disclosure, Figure 3 also shows a terminal display interface, which displays a first focus window 310 and a second focus window 320.

[0123] The first focus window 310 contains a first pointer 311, which moves based on user-triggered interaction events. Real mouse events triggered by the user are bound to the first pointer 311.

[0124] The second focus window 320 contains a second pointer 321. The second pointer 321 moves based on interaction events triggered by the agent, and virtual mouse events triggered by the agent are bound to the second pointer 321. It should be noted that there can be one or more second focus windows 320, and the number shown in Figure 3 does not constitute a limitation of this disclosure.

[0125] In some embodiments, when it is detected that a user switches applications via mouse clicks or keyboard input, the terminal updates the actually active application corresponding to the first focus window 310 and updates the window handle of the first focus window 310 based on the switched application. Similarly, when it is detected that an agent switches applications via simulated mouse clicks or keyboard input, the terminal updates the actually active application corresponding to the second focus window 320 and updates the window handle of the second focus window 320 based on the switched application.

[0126] In some embodiments, the terminal only allows the user to trigger interaction events in the first focus window 310, and the user-triggered interaction events are always sent to the first focus window 310; the terminal only allows the agent to trigger interaction events in the second focus window 320, and the agent-triggered interaction events are always sent to the second focus window 320.

[0127] Based on this, the operations of the user and the intelligent agent do not interfere with each other, reducing task conflicts. While the user is processing tasks, the intelligent agent is simultaneously performing operations in the second focus window 320, thereby improving task processing efficiency.

[0128] In other embodiments, the terminal allows the user to trigger an interaction event in either focus window, and always sends the interaction event triggered by the user in the first focus window 310 (e.g., the first interaction event) to the first focus window 310, and always sends the interaction event triggered by the user in the second focus window 320 (e.g., the third interaction event) to the second focus window 320; the terminal also allows the agent to trigger an interaction event in the second focus window 320, and always sends the interaction event triggered by the agent in the second focus window 320 (e.g., the second interaction event) to the second focus window 320. In the event of a conflict between the second and third interaction events, the execution of the second interaction event is stopped, and the execution of the third interaction event is performed.

[0129] Based on this, users can intervene in the operation of the agent in the second focus window, and the third interaction event triggered by the user has a higher priority than the second interaction event, so that when interesting content is found in the second focus window, the user can intervene in the second focus window to process the task, thereby improving the flexibility of task processing.

[0130] In some other embodiments, the terminal also allows the agent to trigger interactive events in the first focus window 310, and always sends the interactive events triggered by the agent in the first focus window 310 (such as the fourth interactive event mentioned above) to the first focus window 310. For example, upon receiving a second operation instruction input by the user to the agent, the agent is allowed to trigger interactive events in the first focus window 310, and the second operation instruction is used to instruct the agent to perform an operation on the first focus window. Based on this, the user does not need to operate directly in the first focus window, but instead instructs the agent to complete the task on their behalf, reducing the user's task processing burden and improving processing efficiency.

[0131] In some embodiments, as shown in FIG3, a first focus area 330 and a second focus area 340 may also be displayed on the terminal. The first focus area includes a first focus window 310, and the second focus area includes at least one second focus window 320, so as to arrange and display the focus windows in an orderly manner.

[0132] For ease of understanding, the following example illustrates the window operation method provided in this disclosure. The method includes steps one through four.

[0133] Step 1: Respond to the first interaction event triggered by the user on the first focused window and execute the first interaction event.

[0134] For example, a user triggers the first interaction event in the first focus window to complete the quarterly report presentation document.

[0135] Step 2: The terminal acquires the user's input command to the agent and determines whether the user command is a first operation command or a second operation command. In some embodiments, the first operation command instructs the agent to perform an operation on the second focus window, and the second operation command instructs the agent to perform an operation on the first focus window.

[0136] Step 3: If the user instruction is determined to be the first operation instruction and a second focus window corresponding to the first operation instruction exists in the terminal, the agent performs an operation on the second focus window based on the first operation instruction; or, if the user instruction is determined to be the first operation instruction and a second focus window corresponding to the first operation instruction does not exist in the terminal, the terminal obtains the focus request instruction sent by the agent and creates a second focus window for the agent to operate.

[0137] In some embodiments, the first operation instruction further instructs the agent to collect, retrieve, and analyze market price information for the current and previous quarters by operating on a second focus window, and / or generate market price data charts for the current and previous quarters.

[0138] In other embodiments, the agent can also determine the operation in the second focus window based on user behavior. For example, the agent can predict the user's operational needs based on user behavior in the first focus window, such as: collecting, retrieving, and analyzing market price information for the current and previous quarters, and / or generating data charts of market price information for the current and previous quarters.

[0139] Therefore, while the user fills in the content framework and writes the page in the first focus window, the terminal can automatically perform operations in the second focus window through the intelligent agent, thereby improving the efficiency of the user in completing the quarterly report presentation document.

[0140] Step 4: After the second interactive event is completed, a prompt message is issued to indicate that the second interactive event has been completed.

[0141] In some embodiments, the prompt message may further include: the task execution result corresponding to the second interactive event. For example, it may include market price data charts for the current quarter and the previous quarter.

[0142] Therefore, without affecting the user's normal use of the terminal, the intelligent agent can simultaneously execute tasks in the second focus window and obtain the task execution results, thereby reducing the user's task processing pressure and improving task processing efficiency.

[0143] It is understood that, in order to achieve the aforementioned functions, the device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0144] As shown in Figure 4, this embodiment of the present disclosure also provides a structural diagram of a terminal, which includes a window display module 410, a task processing module 420, and an intelligent agent 430. In some embodiments, the terminal further includes a focus maintenance module 440.

[0145] The window display module 410 is used to display the first focus window and the second focus window.

[0146] The task processing module 420 is used to respond to the first interaction event triggered by the user on the first focus window and execute the first interaction event.

[0147] The task processing module 420 is also used to execute the second interaction event in response to the second interaction event triggered by the agent 430 for the second focus window.

[0148] The intelligent agent 430 is used to respond to user commands or autonomously perform operations in the second focus window based on user behavior.

[0149] In some embodiments, the task processing module 420 is further configured to acquire a first operation instruction input by the user to the agent 430, the first operation instruction being used to instruct the agent 430 to perform an operation on the second focus window; in response to the first operation instruction, triggering a second interaction event for the second focus window through the agent 430.

[0150] In some embodiments, the task processing module 420 is further configured to determine the user's behavior in the first focus window; predict the user's operation needs for the second focus window through the intelligent agent 430 based on the user's behavior in the first focus window; and trigger a second interaction event for the second focus window through the intelligent agent 430 based on the operation needs.

[0151] In some embodiments, the task processing module 420 is further configured to, during the execution of the second interactive event, acquire a third interactive event triggered by the user in the second focus window; in response to the third interactive event triggered by the user in the second focus window, stop executing the second interactive event and execute the third interactive event.

[0152] In some embodiments, the task processing module 420 is further configured to issue a prompt message after the second interactive event is executed, the prompt message being used to indicate that the second interactive event has been completed.

[0153] In some embodiments, the task processing module 420 is configured to respond to a second interaction event triggered by the agent 430 for the second focus window, determine the window handle of the second focus window, and send the second interaction event to the program corresponding to the second focus window based on the window handle of the second focus window, and execute the second interaction event through the program corresponding to the second focus window.

[0154] In some embodiments, the window display module 410 is used to display a first focus window and at least one second focus window; the task processing module 420 is used to, in response to a second interaction event triggered by the agent 430 for at least one second focus window, determine the window handle of the target focus window based on the focus identifier of the second interaction event and preset focus mapping information; in some embodiments, the target focus window is the second focus window whose window handle corresponds to the focus identifier of the second interaction event; based on the window handle of the target focus window, the second interaction event is sent to the program corresponding to the target focus window, and the second interaction event is executed by the program corresponding to the target focus window; in some embodiments, the focus mapping information includes at least one focus identifier and at least one window handle of the second focus window, each focus identifier corresponding to a window handle of a second focus window.

[0155] In some embodiments, the focus maintenance module 440 is used to generate at least one focus identifier and create at least one second focus window corresponding to the at least one focus identifier when the focus creation conditions are met, with each second focus window corresponding to a focus identifier; and to maintain focus mapping information based on the at least one focus identifier and the window handle of the at least one second focus window; in some embodiments, the focus creation conditions can refer to the description above.

[0156] In some embodiments, the focus maintenance module 440 is further configured to monitor at least one second focus window corresponding to at least one focus identifier; when the focus state of at least one second focus window corresponding to at least one focus identifier changes, obtain the updated at least one second focus window; and maintain focus mapping information based on the window handle of the updated at least one second focus window.

[0157] In some embodiments, the task processing module 420 is further configured to acquire a second operation instruction input by the user to the agent 430, the second operation instruction being used to instruct the agent 430 to perform an operation on the first focus window; in response to the second operation instruction, trigger a fourth interaction event for the first focus window through the agent 430; and in response to the fourth interaction event triggered by the agent 430 for the first focus window, execute the fourth interaction event.

[0158] It should be noted that the module division in Figure 4 is illustrative and represents only one logical functional division. In actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single processing module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0159] In implementing the functions of the integrated modules described above using hardware, this disclosure also provides a possible structure for an electronic device used to execute the window operation method provided in this disclosure. Similarly, the electronic device and the window operation method described above can be referred to in correspondence with each other.

[0160] As shown in FIG5, the electronic device includes a processor 502 and a communication interface 503. In some embodiments, the electronic device may further include at least one of a bus 504 and a memory 501.

[0161] Processor 502 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 502 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 502 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP), and a microprocessor, etc.

[0162] Communication interface 503 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0163] The memory 501 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0164] In some embodiments, the memory 501 may exist independently of the processor 502. The memory 501 may be connected to the processor 502 via a bus 504 and is used to store instructions or program code executable by the processor 502, such as computer program instructions. When the processor 502 calls and executes the instructions or program code stored in the memory 501, it can implement the window operation method provided in the embodiments of this disclosure.

[0165] In other embodiments, memory 501 may also be integrated with processor 502.

[0166] Bus 504 can be an extended industry standard architecture (EISA) bus, etc. Bus 504 can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in Figure 5, but this does not mean that there is only one bus or one type of bus.

[0167] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform the window operation method as described in any of the embodiments above. It should be understood that this disclosure does not limit the form of the computer.

[0168] In some embodiments, the computer-readable storage media described above may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices for storing information and / or other machine-readable storage media. The term "machine-readable storage media" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0169] This disclosure provides a computer program product containing instructions, the computer product including a computer program, which, when run on a computer, causes the computer to perform the window operation method described in any of the above embodiments.

[0170] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A window operation method applied to a terminal, the method comprising: displaying a first focus window and a second focus window; in response to a first interaction event triggered by a user for the first focus window, executing the first interaction event; in response to a second interaction event triggered by an agent for the second focus window, executing the second interaction event; wherein the agent executes operations in the second focus window in response to a user instruction or autonomously according to a user behavior. 2.The method of claim 1, further comprising: obtaining a first operation instruction input by the user to the agent, the first operation instruction being used to instruct the agent to operate on the second focus window; in response to the first operation instruction, triggering a second interaction event by the agent for the second focus window. 3.The method of claim 1, further comprising: determining a user behavior of the user on the first focus window; predicting, by the agent, an operation demand of the user on the second focus window according to the user behavior of the user on the first focus window; based on the operation demand, triggering a second interaction event by the agent for the second focus window. 4.The method of claim 1, further comprising: during execution of the second interaction event, obtaining a third interaction event triggered by the user on the second focus window; in response to the third interaction event triggered by the user on the second focus window, stopping execution of the second interaction event and executing the third interaction event. 5.The method of claim 1, further comprising: after execution of the second interaction event is completed, issuing a prompt information, the prompt information being used to prompt that the second interaction event has been executed.

6. The method of claim 1, wherein, The response to the second interaction event triggered by the agent for the second focus window, executing the second interaction event, comprises: in response to the second interaction event triggered by the agent for the second focus window, determining a window handle of the second focus window; based on the window handle of the second focus window, sending the second interaction event to a program corresponding to the second focus window, and executing the second interaction event by the program corresponding to the second focus window.

7. The method of claim 1, wherein, The displaying a first focus window and a second focus window comprises: displaying a first focus window and at least one second focus window; The response to the second interaction event triggered by the agent for the second focus window, executing the second interaction event, comprises: in response to the second interaction event triggered by the agent for the at least one second focus window, based on a focus identifier of the second interaction event and preset focus mapping information, determining a window handle of a target focus window; wherein the target focus window is a second focus window in the at least one second focus window, whose window handle corresponds to the focus identifier of the second interaction event; based on the window handle of the target focus window, sending the second interaction event to a program corresponding to the target focus window, and executing the second interaction event by the program corresponding to the target focus window. The focus mapping information includes at least one focus identifier and a window handle of the at least one second focus window, and each focus identifier corresponds to a window handle of the second focus window.

8. The method of claim 7, further comprising: generating the at least one focus identifier and creating the at least one second focus window corresponding to the at least one focus identifier when a focus creation condition is met, each second focus window corresponding to a focus identifier; maintaining the focus mapping information based on the at least one focus identifier and the window handle of the at least one second focus window; wherein the focus creation condition includes at least one of: receiving a focus creation instruction from the user; receiving a focus application instruction from the agent; receiving a first operation instruction input by the user to the agent, and the terminal does not have a second focus window corresponding to the first operation instruction, the first operation instruction indicating that the agent operates on the second focus window.

9. The method of claim 8, further comprising: monitoring the at least one second focus window corresponding to the at least one focus identifier; obtaining updated at least one second focus window when the at least one second focus window corresponding to the at least one focus identifier changes; maintaining the focus mapping information based on the window handle of the updated at least one second focus window.

10. The method of claim 1, further comprising: obtaining a second operation instruction input by the user to the agent, the second operation instruction indicating that the agent operates on the first focus window; triggering a fourth interaction event for the first focus window by the agent in response to the second operation instruction; executing the fourth interaction event in response to the fourth interaction event triggered by the agent for the first focus window.

11. An electronic device comprising: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to cause the electronic device to perform the window operation method according to any one of claims 1-10.

12. A computer readable storage medium, wherein, The computer readable storage medium stores computer instructions, and when the computer instructions run on a computer, the computer executes the window operation method according to any one of claims 1-10.

13. A computer program product, wherein, The computer program product contains computer instructions, and when the computer instructions run on a computer, the computer executes the window operation method according to any one of claims 1-10. The computer program product contains computer instructions, and when the computer instructions run on a computer, the computer executes the window operation method according to any one of claims 1-10.

Images