Multi-window interaction method and device and storage medium

Abstract

The embodiment of the invention provides a multi-window interaction method and device and a storage medium, and belongs to the technical field of terminals. The method comprises the following steps: in response to starting a user-defined window control mode for the intelligent terminal equipment, updating first window information and associated window information of a focus window at the current moment into window management object information stored in a preset multi-window expansion interface object through a window manager on the intelligent terminal equipment; the focus window is any one of a free window and a split-screen window; in response to the received gesture event, acquiring a multi-window expansion interface object in the client application and acquiring target window information through the multi-window expansion interface object; and providing an operation interface corresponding to the gesture event and target window information for the client application through the multi-window expansion interface object, so that the client application operates the focus window according to the operation interface. The embodiment of the invention can improve the user operation experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method, device and storage medium for multi-window interaction. Background Technology

[0002] Smart terminal devices typically use the Android system for window management. However, existing window management only allows operation on the focused window, and is limited by the Android system to simple operations such as swiping. As the types and number of windows supported simultaneously on smart terminal devices increase, multiple free windows can appear on the same screen, or a combination of free windows and split-screen windows can be displayed. In such cases, existing technologies usually designate the split-screen window as the focused window, restricting operations on other windows. Therefore, existing window management methods struggle to meet users' multi-window operation needs, resulting in a poor user experience when multiple windows are open. Summary of the Invention

[0003] The main objective of this application is to propose a method, device, and storage medium for multi-window interaction, which can improve the user experience of interactive operations in the case of multiple windows on smart terminal devices.

[0004] To achieve the above objectives, a first aspect of this application proposes a multi-window interaction method applied to a smart terminal device, the multi-window interaction method comprising: In response to enabling a custom window control mode for the smart terminal device, the window manager on the smart terminal device updates the first window information and associated window information of the current focus window to the window management object information stored in the preset multi-window extension interface object; the focus window can be either a free window or a split-screen window, and the first window information includes the first window display area; In response to receiving a gesture event, the client application obtains the multi-window extension interface object and obtains target window information through the multi-window extension interface object, wherein the target window information is one of first window information and second window information. The multi-window extension interface object provides the client application with the operation interface corresponding to the gesture event and the target window information, enabling the client application to operate the focused window according to the operation interface.

[0005] To achieve the above objectives, a second aspect of the present application provides an intelligent terminal device, the intelligent terminal device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the multi-window interaction method described in any of the first aspects. To achieve the above objectives, a third aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multi-window interaction method described in any of the first aspects.

[0006] The multi-window interaction method, device, and storage medium proposed in this application update the first window information and associated window information of the focus window to the window management object information stored in the preset multi-window extension interface object after enabling the custom window control mode. This allows the client application to know the first window display area of ​​the focus window and the associated windows of the current focus window through the multi-window extension interface object based on the existing window application framework in the smart terminal device. Since the first window display area is known, when the operation object is the focus window, precise control of the focus window can be achieved through the custom operation interface in the multi-window extension interface object. At the same time, since both the focus window and the associated windows are known, operations on the associated windows can also be achieved by setting corresponding interfaces. Therefore, compared with related technologies, when there are multiple windows, this application embodiment can realize more complex operations on different window objects, thereby improving the user operation experience of multi-window interaction on smart terminal devices. Attached Figure Description

[0007] Figure 1 This is a flowchart illustrating an embodiment of the multi-window interaction method provided in this application; Figure 2 This is a schematic diagram of the layers of a free window and its border in one embodiment of the multi-window interaction method provided in this application; Figure 3 This is an application diagram illustrating an embodiment of the multi-window interaction method provided in this application; Figure 4 This is a schematic diagram illustrating the enabling of a custom window control mode in one embodiment of the multi-window interaction method provided in this application; Figure 5 This is a schematic diagram of the program framework in one embodiment of the multi-window interaction method provided in this application; Figure 6 This is a schematic diagram of data flow in one embodiment of the multi-window interaction method provided in this application; Figure 7 This is a schematic diagram of the hardware structure of the smart terminal device corresponding to the multi-window interaction method provided in this application. Detailed Implementation

[0008] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0009] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0010] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0011] The following is a description of the terminology used in the embodiments of this application: Free-form windows are a multi-window interaction mode provided by the operating system of smart terminal devices (such as mobile phones, tablets, and foldable screens). They refer to application window forms that can be freely resized, positioned, and layered, and can run in parallel with other windows and interact independently.

[0012] The focus window is the window that is currently interacting with the user and can uniquely receive keyboard and mouse input. It is the "identifier" of the user's current target in the operating system.

[0013] The window display area refers to the part of the window that is actually visible on the screen, that is, the area of ​​the window that can be directly seen by the user. It is determined by the window's position, size, stacking level (whether it is obscured by other windows), and screen boundaries. It is the "actual area presented to the user" of the window.

[0014] WMshell is an abbreviation for Window Manager Shell, which is essentially a toolset / interaction interface in the operating system used to manipulate windows via the command line.

[0015] Smart terminal devices typically use the Android system for window management. However, existing window management only allows operation on the focused window, and is limited by the Android system to simple operations such as swiping. As functional requirements for windows increase (such as closing, full-screen, and scaling), existing window management methods are insufficient to meet user needs. Furthermore, with the increasing number and types of windows supported simultaneously on smart terminal device screens, multiple free windows or combinations of free windows and split-screen windows can appear on the same screen. In such cases, existing technologies typically designate the split-screen window as the focused window, restricting operations on other windows. This makes it difficult to meet users' multi-window operation needs, resulting in a poor user experience in multi-window scenarios. Therefore, this application proposes a method, device, and storage medium for multi-window interaction, which can improve the user experience of free window interaction on smart terminal devices.

[0016] Understandably, referring to Figure 1 As shown, a multi-window interaction method provided in an embodiment of this application is applied to a smart terminal device, and the method includes: Step S100: In response to enabling the custom window control mode for the smart terminal device, the first window information and associated window information of the current focus window are updated to the window management object information stored in the preset multi-window extension interface object through the window manager on the smart terminal device; the focus window can be either a free window or a split-screen window, and the first window information includes the first window display area; Step S200: In response to receiving a gesture event, obtain a multi-window extension interface object in the client application and obtain target window information through the multi-window extension interface object. The target window information is one of the first window information and the second window information. Step S300: The multi-window extension interface object provides the client application with the operation interface corresponding to the gesture event and the target window information, so that the client application can operate the focused window according to the operation interface.

[0017] By enabling the custom window control mode and adding a custom interface to the window manager to update the first window information and associated window information of the focused window to the window management object information stored in the preset multi-window extension interface object, the client application can know the first window display area of ​​the focused window and the associated windows of the current focused window through the multi-window extension interface object based on the existing window application framework in the smart terminal device. At this time, since the first window display area is known, when the operation object is the focused window, precise control of the focused window can be achieved through the custom operation interface in the multi-window extension interface object. At the same time, since both the focused window and the associated windows are known, operations on the associated windows can also be achieved by setting corresponding interfaces. Therefore, when there are multiple windows, this embodiment of the application can realize more complex operations on different window objects, thereby improving the user operation experience of multiple window interaction operations on the smart terminal device.

[0018] The associated window information refers to the window information of the window adjacent to the currently focused window in the same layer. At least one associated window must be set. The first window information is used to identify the focused window and its operation area, while the associated window information is used to identify the associated window. Both the associated window and the focused window are windows that are currently allowed to be operated on. The second window information is the window information of the associated window of the focused window determined based on gesture events. This application does not restrict the location of the custom window control mode setting on the smart terminal device. For example, in some embodiments, related options can be set in the developer mode of the smart terminal device, such as... Figure 4 As shown, in developer mode, enabling control over free window behavior will allow you to enter custom window control mode.

[0019] Gesture events are generated based on the recognition of gesture images captured by an image device (such as a camera). The multi-window extension interface object is a custom interface object used to manage the operation interfaces of various operations supported by the currently active window, as well as the window information of windows that are allowed to perform operations, so as to execute specific operations on a specified window through the operation interfaces and corresponding window information. This application embodiment does not limit the operation interfaces supported by the multi-window extension interface object.

[0020] The first window display area is the portion of the first window that is actually visible on the screen. By default, the focus window is the outermost window at the current moment. In some embodiments, when there is a default window that is always on top, the focus window can be either the window that is always on top or a window specified by the user. This application does not limit this, and those skilled in the art can selectively set it according to the actual situation.

[0021] Understandably, by extending the interface object through multiple windows, the client application is provided with operation interfaces corresponding to gesture events and target window information, enabling the client application to perform operations on the focused window according to the operation interfaces, including: When the gesture type of the gesture event is a vertical swipe gesture, the swipe event is injected into the first window display area through the swipe interface provided to the client application by the multi-window extension interface object, so that the focus window follows the gesture event to swipe.

[0022] By injecting a swipe event into the first window display area, it's possible to simulate input gestures to swipe the focused window within that area. Because the first window display area is defined, more precise manipulation of the specified focused window is possible.

[0023] In some embodiments, input events can be injected using `InputManager.injectInputEvent` (a method that simulates input events). These input events are generated based on the first window's display area, allowing control over the content of the focused window. In some embodiments, an `IAwExtensionWindow.Stub` object can be bound during VMS initialization. When customizing the window control mode, `IAwExtensionWindow.Stub` can be instantiated as a multi-window extension interface object. This allows client applications to directly obtain the multi-window extension interface object using VMS, and subsequently, client applications can call multiple operation interfaces provided within the multi-window extension interface object to perform different operations on the window.

[0024] This application does not impose any restrictions on how the up and down swiping gestures are recognized. Those skilled in the art can selectively set the gesture recognition algorithm by referring to the prior art.

[0025] Understandably, by extending the interface object through multiple windows, the client application is provided with operation interfaces corresponding to gesture events and target window information, enabling the client application to perform operations on the focused window according to the operation interfaces, including: When the gesture type of the gesture event is a left or right swipe gesture, the second window information corresponding to the gesture event is provided to the client application through the multi-window extension interface object. The second window information is one of the parameters in the associated window information. Based on the second window information, the window manager on the smart terminal device is notified to update the current focus window to the second window corresponding to the second window information, and the window manager updates the window management object information stored in the multi-window extension interface object according to the updated focus window information and the associated window information corresponding to the focus window information; the second window corresponding to the second window information is the associated window of the current focus window in the gesture event movement direction.

[0026] This application does not limit how the gesture type is identified as a left or right swipe gesture. Those skilled in the art can selectively set it according to actual needs, which will not be elaborated here.

[0027] The second window information includes at least the window identifier of the window to which the gesture event is pointed. In some embodiments, the second window information may also include other parameters, which are not limited in this application. Since the second window information can identify a unique window, calling the focus switching interface in the window manager can switch the current focus window to the second window corresponding to the second window information, thereby realizing the switching of the focus window. For example, by setting the setXXX() interface in IAwExtensionWindow, when the client application calls setXXX(), the setXXX() method of IAwExtensionWindow in the window manager can be called through a callback to update the focus window.

[0028] This application does not limit how the second window associated with the currently focused window is determined based on a gesture event, specifying which direction of the adjacent layers it belongs to. For example, in some embodiments, when the windows associated with the currently focused window include windows from the previous and next layers, if the gesture type is a left swipe, the window from the previous layer can be designated as the associated second window, thus allowing the acquisition of the second window's window attributes. Similarly, if the gesture type is a right swipe, the window from the next layer can be designated as the associated second window. In other embodiments, a rightward gesture can determine the window from the previous layer as the associated second window, and a leftward gesture can determine the window from the next layer as the associated second window.

[0029] After setting the second window as the new focus window, if no top-view window exists, the second window is placed on top. If a top-view window exists (which can be a split-screen window or a free window), the second window is placed below it. At this point, when a new gesture occurs, the gesture operation will be performed on the second window based on its display area. This application does not limit the operations related to switching the first window from the focus window to the non-focus window. Those skilled in the art can selectively set these operations according to actual needs, such as placing the first window below the second window or placing it at the bottom of all windows. Further details are omitted here.

[0030] Understandably, the multi-window extension interface object provides the client application with operation interfaces corresponding to gesture events and target window information, enabling the client application to operate on the focused window according to the operation interfaces. This also includes: When the gesture type of the gesture event is a screenshot gesture, the screenshot interface for the first window display area is provided to the client application through the multi-window extension interface object. The image is captured in the first window display area and the target screenshot is output.

[0031] This application embodiment does not limit the shape of the first window display area, nor does it limit whether the first window display area is larger than the size of the focus window.

[0032] The coordinates of the focus window can be determined by the first window display area, and then the first window display area can be traversed pixel by pixel based on the area coordinates, thereby realizing image acquisition of the first window display area.

[0033] The target screenshot can be stored on a preset storage path. In some embodiments, the target screenshot will remain on the display screen of the smart terminal device, so as to facilitate checking whether the screenshot was successfully taken.

[0034] This application does not limit the specific gesture used for screenshotting; those skilled in the art can selectively set it according to actual needs. For example, in some embodiments, a clenched fist indicates taking a screenshot of the focused window. In other embodiments, other gestures may also be used. This application also does not limit how the gesture is recognized; those skilled in the art can selectively set it according to actual needs.

[0035] Understandably, by extending the interface object through multiple windows, the client application is provided with operation interfaces corresponding to gesture events and target window information, enabling the client application to perform operations on the focused window according to the operation interfaces, including: When the gesture event's gesture type is a close gesture, the close operation interface provided to the client application through the multi-window extension interface object notifies the window manager on the smart terminal device to close the focused window, re-determine the current focused window, and update the window management object information stored in the multi-window extension interface object; The multi-window extension interface object provides updated first-window information to client applications.

[0036] This application does not limit which gesture represents closing. For example, in some embodiments, a finger waving represents closing a window, while in other embodiments, a finger tapping represents closing a window. This application also does not limit how the closing gesture is recognized; those skilled in the art can choose existing model recognition methods, which will not be elaborated upon here.

[0037] This application does not limit whether the second window is a window on the layer above or below the focus window. Those skilled in the art can set the default settings according to actual needs. In some embodiments, configuration items can also be set to enable real-time input modification.

[0038] After the second window becomes the new focus window, various operations of the focus window can be implemented on the second window. In this embodiment, these operations will not be described in detail.

[0039] Understandably, the multi-window extension interface object provides the client application with operation interfaces corresponding to gesture events and target window information, enabling the client application to operate on the focused window according to the operation interfaces. This also includes: When the gesture type of the gesture event is a zoom gesture, the zoom parameters are determined based on the gesture event. The multi-window extension interface object provides a scaling interface to the client application for scaling parameters, which adjusts the size of the first window's display area and the size of the focused window according to the scaling parameters.

[0040] This application does not limit how the scaling parameters are determined. In some embodiments, a fixed scaling ratio can be used. For example, if a scaling gesture indicates magnification, a fixed magnification ratio is obtained as the scaling parameter. Similarly, if a scaling gesture indicates shrinkage, a fixed shrinkage ratio is obtained as the scaling parameter. In other embodiments, the scaling ratio can be determined in real-time based on the magnitude of the gesture change. For example, if a hand changes from open to clenched to indicate shrinkage, the shrinkage ratio is determined by the proportion of the shrinkage area caused by the final clenched fist state relative to the fully open hand. Similarly, if a hand changes from clenched to open to indicate magnification, the magnification ratio is dynamically determined by the proportion of the enlarged area of ​​the open hand. This application does not elaborate on these details; those skilled in the art can selectively set the scaling parameters according to actual conditions. The scaling parameters can be defined in the scaling interface or in the client application; this application does not limit this. Those skilled in the art can selectively set the scaling parameters according to actual conditions.

[0041] The methods for adjusting the size of the first window display area through the window manager, as described in this application embodiment, will not be elaborated on individually, nor will the methods for adjusting the window size of the focused window through the window manager be elaborated on individually.

[0042] In this embodiment, there is no limitation on whether to adjust the window size of the focus window or the area size of the first window display area first. They can be adjusted simultaneously, or the first window display area can be adjusted first, or the first window can be adjusted first. Those skilled in the art can selectively set this according to actual needs.

[0043] This application does not limit how to identify zoom gestures. Those skilled in the art can determine this through machine learning models, image recognition, and other methods. This application will not elaborate on these methods.

[0044] Understandably, the method also includes: In response to enabling a custom window control mode for smart terminal devices, the window manager on the smart terminal device is controlled to store the window attributes and display areas of each currently displayed window as a storage element in a preset container; each window includes at least one of free windows and split-screen windows; The storage elements in the container are sorted according to the focus window rules. The first window is determined as the focus window, and the first window information and associated window information of the first window are determined. The first window information and associated window information both include window identifiers. The first window information also includes the window display area. Update the window management object information based on the information of the first window and the associated windows.

[0045] This application does not limit the storage container; in some embodiments, the container can be set as a Map container.

[0046] The focus rule characterizes the relationship between the preceding and following windows of each window displayed on the current smart terminal. Based on the relationship between the preceding and following windows, the layer stacking of each window can be determined, thereby identifying the focus window. For example, in some embodiments, if the outermost layer is a split-screen window, then the free window in the next layer of the split-screen window is taken as the focus window. If the outermost layer is a free window, then that free window can be taken as the focus window.

[0047] The associated windows include at least the windows of the next layer, and in some embodiments, also the windows of the previous layer. For example, for any non-focused window, its associated windows include the windows of the previous layer and the windows of the next layer. For a focused window, if a top-mounted window exists, the associated windows of the focused window include the top-mounted window and the windows of the next layer; if no top-mounted window exists, the associated windows of the focused window only include the windows of the next layer. Sorting can be performed according to layer order, and top-mounted and focused windows can be identified, thereby allowing for quick determination of their associated windows based on the sorted windows.

[0048] Understandably, in some embodiments, the free windows stored on the storage path can be reordered each time the focus window is switched or closed, thereby improving the efficiency of subsequent operations on the focus free windows.

[0049] Understandably, the method also includes: The first border coloring interface provided to the client application through the multi-window extension interface object generates a border of the first color at the edge of the focused window. The second border coloring interface provided to the client application through the multi-window extension interface object generates a border of a second color on the edge of the window outside the focused window, with the saturation of the first color being greater than that of the second color.

[0050] This application does not limit how the border is generated. In some embodiments, a border layer can be added to the corresponding free window. The drawing order of the subviews on the free window can be changed by overriding the dispatchDraw method of a custom view inheriting from LinearLayout. The border properties of the border drawn in the border layer are defined using a drawing tool (such as Canvas / Paint), and then rendered onto the border layer based on the defined border properties. Simultaneously, the size, position, and Z-axis layer of the border are adjusted based on a layer interface class (such as SurfaceControl.Transaction) and attached to the multi-window layer of the free window. At this time, as shown... Figure 2 As shown, a border layer is set at the bottom of the multi-window layer where each free window is located.

[0051] This application does not limit what the first color and the second color are, and those skilled in the art can selectively set them according to actual needs. For example, in some embodiments, the first color is purple and the second color is gray. In other embodiments, the first color and the second color can also be dynamically configured, and the dynamic configuration of the first color and the second color can be achieved according to the dynamically input configuration items.

[0052] For example, see below. Figures 3 to 6 The method described in this application is applied to a client application, for example. Figure 3 As shown, the specific steps are as follows: S1. The client application detects that the smart terminal device has enabled the custom window control mode, and updates the first window information and associated window information of the currently focused window to the window management object information stored in the preset multi-window extension interface object through the window manager on the smart terminal device; the focused window can be either a free window or a split-screen window; as detailed below: like Figure 4 As shown, in the developer mode of the smart terminal device, the option to control free window behavior is checked. At this time, the client application can detect that the smart terminal device has enabled the custom window control mode. Figure 5As shown, the custom window control mode triggers a call to the collection interface in the WMShell module (i.e., the window manager) to collect various windows (including free windows and split-screen windows) on the smart terminal screen and store them in a Map container. Each element in the Map container corresponds to a window, and each element includes the window attributes (such as taskID) and display area of ​​the corresponding window. The Map containers are sorted according to the focus window rules. Then, based on the focus window rules (such as X-coordinate or time size, etc.), the window attributes of the focus window, as well as the window attributes and display area of ​​associated windows, can be quickly determined by traversing the window types corresponding to the taskIDs in each stored element of the sorted Map containers. Simultaneously, as... Figure 5 As shown, since the smart terminal initializes the IAwExtensionWindow.Stub object bound by the SetExtension method during VMS initialization, when the window manager can update the instantiated IAwExtensionWindow.Stub object (i.e., the multi-window extension interface object) based on the first window information and associated window information, the client application can obtain the multi-window extension interface object through the VMS object.

[0053] S2. The client application detects a gesture event and performs the corresponding operation on the first window through the multi-window extension interface object, as detailed below. Figure 6 As shown: S2.1 Camera shooting: The smart terminal device triggers the camera to shoot and obtain a gesture image; S2.2 Gesture processing module: The gesture processing module parses the gesture image, obtains the gesture features, and calls back the corresponding client application thread of the client application; S2.3, The client application thread generates gesture events based on gesture features and calls back the multi-window extension interface object, where gesture events include gesture type and gesture operation; for example, such as Figure 5 As shown, the client application obtains the VMS object, and through the VMS object, it obtains an instance of IAwExtensionWindow. The client application can call the corresponding methods of the IAwExtensionWindow instance to obtain the taskID of the focused window, the second window, and the display area of ​​the first window. Therefore, it can perform window operations based on the interfaces provided by the IAwExtensionWindow instance. For example, as shown... Figure 5As shown, the VMS object detects a window operation change by calling the operation interface through an instantiated object of IAwExtensionWindow, thereby triggering WMshhell to call the methods of the multi-window extension interface object to execute specific window operations and control the behavior of multiple windows. The methods of the instantiated object of IAwExtensionWindow include: swiping up / down on the focused window, changing the focused window, adjusting the size of the focused window, and closing the focused window.

[0054] Therefore, the method described in the above embodiments of this application can realize air interaction between multiple free windows. By combining gesture recognition with a custom window management mechanism, it breaks through the limitation of traditional air gestures being limited to a single window, enabling cross-window interaction on large-screen smart terminal devices and significantly improving multitasking efficiency. Furthermore, by providing a multi-window extension interface object, it can replace the traditional input simulation method, better solving cross-platform compatibility issues.

[0055] Please see Figure 7 , Figure 7 The hardware structure of another embodiment of a smart terminal device is illustrated. The smart terminal device includes: The processor 401 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 402 can be a NAND flash memory. The relevant program code is stored in the memory 402 and is called by the processor 401 to execute the methods described in the embodiments of this application. Input / output interface 403 is used to implement information input and output; The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404); The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0056] This application also provides a computer-readable storage medium that stores a computer program that, when executed by a processor, implements the above-described method.

[0057] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0058] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0059] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0060] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0061] The terms “comprising” and “having”, and any variations thereof, in the specification and accompanying drawings of this application are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or apparatus.

[0062] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0063] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0064] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0066] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for multi-window interaction, characterized in that, Applied to smart terminal devices, the method includes: In response to enabling a custom window control mode for the smart terminal device, the window manager on the smart terminal device updates the first window information and associated window information of the current focus window to the window management object information stored in the preset multi-window extension interface object; the focus window can be either a free window or a split-screen window, and the first window information includes the first window display area; In response to receiving a gesture event, the client application obtains the multi-window extension interface object and obtains target window information through the multi-window extension interface object, wherein the target window information is one of first window information and second window information. The multi-window extension interface object provides the client application with the operation interface corresponding to the gesture event and the target window information, enabling the client application to operate the focused window according to the operation interface.

2. The multi-window interaction method according to claim 1, characterized in that, The provision of the operation interface corresponding to the gesture event and the target window information to the client application through the multi-window extension interface object, enabling the client application to operate the focused window according to the operation interface, includes: When the gesture type of the gesture event is a vertical swipe gesture, the swipe event is injected into the first window display area through the swipe interface provided to the client application by the multi-window extension interface object, so that the focus window follows the swipe of the gesture event.

3. The multi-window interaction method according to claim 1, characterized in that, The provision of the operation interface corresponding to the gesture event and the target window information to the client application through the multi-window extension interface object, enabling the client application to operate the focused window according to the operation interface, includes: When the gesture type of the gesture event is a left or right swipe gesture, the multi-window extension interface object provides the client application with second window information corresponding to the gesture event, and the second window information is one of the parameters in the associated window information; Based on the second window information, the window manager on the smart terminal device is notified to update the current focus window to the second window corresponding to the second window information, and the window manager updates the window management object information stored in the multi-window extension interface object according to the updated focus window information and the associated window information corresponding to the focus window information; the second window corresponding to the second window information is the associated window of the current focus window in the direction of the gesture event movement.

4. The multi-window interaction method according to claim 1, characterized in that, The provision of the operation interface corresponding to the gesture event and the target window information to the client application through the multi-window extension interface object, enabling the client application to operate the focused window according to the operation interface, includes: When the gesture type of the gesture event is a close gesture, the window manager on the smart terminal device is notified to close the focus window, re-determine the current focus window, and update the window management object information stored in the multi-window extension interface object through the close operation interface provided to the client application by the multi-window extension interface object. The updated information of the first window is provided to the client application through the multi-window extension interface object.

5. The multi-window interaction method according to claim 1, characterized in that, The provision of the operation interface corresponding to the gesture event and the target window information to the client application through the multi-window extension interface object, enabling the client application to operate the focused window according to the operation interface, further includes: When the gesture type of the gesture event is a screenshot gesture, the screenshot interface for the first window display area is provided to the client application through the multi-window extension interface object, and the image is captured in the first window display area to output the target screenshot.

6. The multi-window interaction method according to claim 1, characterized in that, The provision of the operation interface corresponding to the gesture event and the target window information to the client application through the multi-window extension interface object, enabling the client application to operate the focused window according to the operation interface, further includes: When the gesture type of the gesture event is a zoom gesture, the zoom parameter is determined according to the gesture event; The scaling interface provided to the client application by the multi-window extension interface object is used to adjust the size of the display area of ​​the first window and the window size of the focused window according to the scaling parameters.

7. The multi-window interaction method according to claim 1, characterized in that, The window management object information is obtained through the following steps: In response to enabling a custom window control mode for the smart terminal device, the window manager on the smart terminal device is controlled to store the window attributes and display areas of each currently displayed window as a storage element in a preset container; each window includes at least one of free windows and split-screen windows; The storage elements in the container are sorted according to the focus window rules. The first window is determined as the focus window, and the first window information and associated window information of the first window are determined. The first window information and the associated window information both include a window identifier. The first window information also includes a window display area. Update the window management object information based on the first window information and the associated window information.

8. The multi-window interaction method according to claim 1, characterized in that, The method further includes: The first border coloring interface provided to the client application through the multi-window extension interface object generates a border of the first color at the edge of the focused window. The second border coloring interface provided to the client application through the multi-window extension interface object generates a border of a second color at the edge of the window outside the focus window, wherein the saturation of the first color is greater than the saturation of the second color.

9. A smart terminal device, characterized in that, The intelligent terminal device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the multi-window interaction method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the multi-window interaction method as described in any one of claims 1 to 8.

Images