Interaction method, electronic device, readable medium and program product

Abstract

The present application relates to the technical field of computers, and in particular to an interaction method, an electronic device, a readable medium and a program product. According to the method, thumbnail windows corresponding to one or more task windows displayed on a screen are displayed in one area in a centralized mode, which can be referred to as a task control area hereinafter. On this basis, on the basis of an operation of a user on the thumbnail windows in the task control area for instructing to move a window display position, a foldable device can recognize a window operated by a user and a target area or target position to which the user intends to drag the window. Therefore, the interaction solution provided by the present application avoids damage to a screen caused by sliding operations of a user on the screen, and can improve the efficiency of adjusting the position of each window displayed by the screen by the user, thereby improving the use experience of the user.

Description

Interaction methods, electronic devices, readable media and program products

[0001] This application claims priority to Chinese Patent Application No. 202410176385.0, filed on February 7, 2024, entitled “Interactive Method, Electronic Device, Readable Medium and Program Product”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of computer technology, and more specifically to an interactive method, electronic device, readable medium, and program product. Background Technology

[0003] With the development of smart terminal technology, the emergence of foldable devices has met users' needs for different screen sizes. A foldable device can include one or more display screens capable of displaying application interfaces. When a foldable device includes a single screen, that screen can be a flexible folding screen, such as a flexible folding screen. When a foldable device includes multiple screens, each screen can be a screen that folds using flexible folding technology or folding technology such as hinges.

[0004] While foldable devices provide users with more screen space, they also lead to more frequent switching between multiple screens. Referring to Figure 1(a), when the foldable device 10 is unfolded, screens B and C can form a larger screen A, which displays one or more windows. Continuing to refer to Figure 1(b), after the foldable device 10 is folded into a semi-unfolded state, or after it is unfolded into a semi-unfolded state, screens B and C can each display one or more task windows. For example, a user can edit a document in a window displayed on screen B and open multiple reference documents for viewing on screen C. When the user needs to move the window displaying reference documents from screen C to screen B, or when the user needs to move the window being edited from screen B to screen C, the user needs to drag the window across screens.

[0005] However, the operation path for dragging windows across screens is relatively long. During the dragging process, users may cause wear and tear on the screen of the foldable device 10. Furthermore, if the foldable device 10 is placed on an object, such as a desktop, the user may apply pressure to the screen B while dragging the window, which could cause the foldable device 10 to tip over, resulting in a poor user experience.

[0006] Summary of the Invention

[0007] This application provides an interaction method, electronic device, readable medium, and program product that allows users to quickly adjust window positions on foldable devices or various collaborative devices that require dragging windows across screens. The operation path for users to adjust window positions across screens is shortened; window positions can be adjusted simply by dragging a mouse on the foldable device screen or sliding a touch object a short distance. This avoids screen damage caused by swiping operations and improves the efficiency of adjusting the positions of displayed windows, thereby enhancing the user experience.

[0008] In a first aspect, this application provides an interaction method applied to an electronic device, the method comprising: displaying a first interface, wherein the first interface includes a first window and a first display area, the first display area including a first identifier image of the first window; detecting a first drag operation by a user on the first identifier image; and moving the first window in response to the first drag operation.

[0009] For example, the aforementioned electronic device can be a foldable laptop, and the aforementioned first interface can be the interface or window displayed by the electronic device running various applications, such as the desktop. The aforementioned first window can be a task window displaying the corresponding application interface, such as a window displaying the homepage interface of a video application. Alternatively, the aforementioned first window can also be a sub-page window on the corresponding application interface, such as a video playback window of a video application. The aforementioned first display area can be, for example, a task control area including thumbnail windows or application icons corresponding to one or more task windows displayed by the electronic device. The aforementioned thumbnail window can include the first identifier image of the aforementioned first window, i.e., the thumbnail window or application icon of the first window. The aforementioned first drag operation can be an operation in which the user drags the first identifier image in the task control area. Correspondingly, the first window can be adjusted to be displayed in the area specified by the user according to the direction and distance of the drag operation.

[0010] In some implementations, the logo image is also called a preview image, or in other implementations, the logo image can be carried or implemented in the form of controls, buttons or other forms.

[0011] In this way, users can adjust the screen position of the actual window by performing small swipes on the thumbnail window in the task control area. For example, without using a mouse or other connected device, a user only needs to slide their finger up or down a short distance on the thumbnail window in the task control area to move the corresponding window from one screen to another. This avoids damaging the screen with swiping operations and improves the efficiency of adjusting the position of windows on the screen, thereby enhancing the user experience.

[0012] In one possible implementation of the first aspect described above, the electronic device includes a foldable device, wherein the foldable device includes a first screen portion and a second screen portion located on both sides of a folding axis.

[0013] For example, the aforementioned electronic device can be a foldable laptop, which can be a bi-fold, tri-fold, or multi-fold device. Taking a bi-fold device as an example, the first screen portion of the foldable laptop can be a screen on one side of the folding axis, and the second screen portion can be a screen on the other side of the folding axis. When the foldable laptop is unfolded, the two screens can form a large-sized screen. The screen portions can be components of the same screen or two relatively independent screens; there are no restrictions on this.

[0014] In one possible implementation of the first aspect described above, the first screen portion is located in a first direction of the second screen portion, and the second screen portion is located in a second direction of the first screen portion, wherein the first direction is opposite to the second direction.

[0015] For example, one screen portion of a foldable laptop can be located on one side of the folding hinge, and the other screen portion can be located on the other side of the folding hinge. Therefore, one screen portion can be positioned relative to the other screen portion in a certain direction, such as the first direction or the second direction mentioned above. The first direction could be, for example, an upward direction, and the second direction could be, for example, a downward direction. The first screen portion could be screen A in Figure 2a below, and the second screen portion could be screen B in Figure 2b below. The first screen portion, i.e., screen A, could be suspended in mid-air, and the second screen portion, i.e., screen B, could be placed on a desktop or other supporting surface.

[0016] In one possible implementation of the first aspect described above, the first display area is located in the first screen portion.

[0017] In one possible implementation of the first aspect above, detecting a user's first drag operation on the first identifier image includes: detecting an operation in which the user drags the first identifier image to move in a second direction.

[0018] The aforementioned first display area, such as the task control area, can be displayed within the aforementioned first screen portion, for example, in an area near the edge of the second screen portion within the first screen portion, as shown in Figures 2a to 2c below for the task control area 210. In another embodiment described below, the aforementioned task control area can also be displayed within the second screen portion, as detailed in the relevant description below, without limitation.

[0019] In one possible implementation of the first aspect described above, the second screen portion includes a displayed second interface, the second interface includes a second window, and the first display area includes a second identifier image corresponding to the second window.

[0020] In one possible implementation of the first aspect described above, the method further includes: detecting a second drag operation by the user dragging the second icon image in a first direction; and corresponding to the second drag operation, moving the second window to the first interface displayed on the first screen portion.

[0021] In one possible implementation of the first aspect above, detecting a user's first drag operation on the first icon image includes: detecting that the user drags the first icon image a first distance in a second direction to satisfy a first threshold condition; and / or, detecting a user's second drag operation on the second icon image in a first direction includes: detecting that the user drags the second icon image a second distance in a first direction to satisfy a first threshold condition.

[0022] For example, if the first window is originally displayed on the first screen, the user can drag the first icon in the second direction to instruct the first window corresponding to the first icon to be displayed on the second screen.

[0023] Similarly, if the second window is originally displayed on the second screen, the user can drag the second icon in the first direction to instruct the second window corresponding to the second icon to be displayed on the first screen.

[0024] In one possible implementation of the first aspect described above, detecting a user's first drag operation on the first identifier image includes:

[0025] The system detects a first combined operation by the user on a first icon, wherein the first combined operation includes a first operation of dragging the first icon in a second direction and hovering it, and detects an operation by the user selecting a first area in a second display area triggered by the first operation, wherein the second display area includes at least one display area recommended for the first window, the at least one display area includes the first area, and the at least one display area is located within the display area of ​​a second screen portion.

[0026] In one possible implementation of the first aspect above, moving the first window in response to the first drag operation includes: displaying the first window in the corresponding area of ​​the first region in the second screen portion in response to the first combined operation.

[0027] For example, the first combination of operations mentioned above may include operation ② shown in Figure 2b below. The user can drag the window down and hover or press and hover the finger on the window. At this time, the user can trigger the display of the second display area, such as clicking the selected layout position in the layout menu shown in Figure 2b.

[0028] In one possible implementation of the first aspect described above, the method includes: detecting a second combined operation by a user on a second identifier image, wherein the second combined operation includes a second operation of dragging and hovering the second identifier image in a first direction; and detecting an operation by the user selecting a second region in a third display area triggered by the second operation, wherein the third display area includes at least one display area recommended for a second window, the at least one display area including the second region, and the at least one display area being located within the display area of ​​a first screen portion; and corresponding to the second combined operation, displaying the second window in the corresponding area of ​​the second region in the first screen portion.

[0029] For example, the second combination of operations mentioned above may include operation ① shown in Figure 2b below. The user can drag the window upward and hover it or press and hover his finger on the window. At this time, the user can trigger the display of the second display area, such as clicking the selected layout position in the layout menu shown in Figure 2b.

[0030] In one possible implementation of the first aspect above, a third distance by which the first identifier is dragged to move in the second direction satisfies the second threshold condition; and / or, a fourth distance by which the second identifier is dragged to move in the first direction satisfies the second threshold condition.

[0031] In one possible implementation of the first aspect above, the duration of the hovering operation satisfies the third threshold condition.

[0032] In one possible implementation of the first aspect described above, detecting a user's first drag operation on the first icon image includes: detecting a third combined operation by the user acting on the first icon image with a mouse, the third combined operation including a third operation of placing the mouse on and hovering over the first icon image; and detecting an operation by the user clicking the third operation with the mouse to trigger the operation of a first control among a plurality of controls displayed, wherein the first control is used to indicate a portion of the screen where the first window is displayed.

[0033] For example, the third combination of operations described above may include operation ③ or operation ④ shown in Figure 2c below. The user can hover the mouse over window 211 to cause the electronic device to display layout controls, that is, the third operation described above triggers the display of multiple controls. Correspondingly, the first control described above can be any of the layout controls described above.

[0034] In one possible implementation of the first aspect described above, the first control indicates that the screen portion to be displayed is the second screen portion.

[0035] In one possible implementation of the first aspect above, moving the first window in response to the first drag operation includes: displaying the first window in the second screen portion indicated by the first control in response to the third combined operation.

[0036] In one possible implementation of the first aspect described above, an icon indicating a second direction is displayed on the first control.

[0037] In one possible implementation of the first aspect above, the duration of the hovering operation satisfies the fourth threshold condition.

[0038] In one possible implementation of the first aspect described above, the first display area is located in the second screen portion.

[0039] In one possible implementation of the first aspect above, detecting a user's first drag operation on the first identifier image includes: detecting an operation in which the user drags the first identifier image to move in a first direction.

[0040] For example, the first display area mentioned above is a task control area, which can also be displayed on a second screen portion placed on the desktop.

[0041] In one possible implementation of the first aspect described above, detecting a user's first drag operation on the first icon image includes: detecting a fourth combined operation by the user on the first icon image, wherein the fourth combined operation includes a fourth operation of dragging the first icon image in a first direction and hovering it, and detecting an operation by the user selecting a third area in a fourth display area triggered by the fourth operation, wherein the fourth display area includes at least one display area recommended for the first window, the at least one display area includes the third area, and the at least one display area is located within the display area of ​​the first screen portion.

[0042] In one possible implementation of the first aspect above, detecting a user's first drag operation on the first icon image includes: detecting a fifth combined operation by the user acting on the first icon image with the mouse, the fifth combined operation including a fifth operation of placing the mouse on the first icon image and hovering it; and detecting an operation by the user clicking the fifth operation with the mouse to trigger a second control among a plurality of displayed controls, wherein the second control is used to indicate a screen portion of the first window to be displayed, the screen portion including a first screen portion or a second screen portion.

[0043] In one possible implementation of the first aspect described above, the first identifier image includes: a thumbnail window corresponding to the first window; or, an application icon corresponding to the application interface displayed in the first window.

[0044] In a second aspect, this application provides an electronic device, including: one or more processors; one or more memories; the one or more memories storing one or more programs, which, when executed by one or more processors, cause the electronic device to perform the interaction methods provided in the first aspect and various possible implementations of the first aspect.

[0045] Thirdly, this application provides a computer-readable medium storing instructions that, when executed on a computer, cause the computer to perform the interaction methods provided in the first aspect and various possible implementations of the first aspect.

[0046] Fourthly, this application provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the interaction methods provided in the first aspect and various possible implementations of the first aspect.

[0047] The beneficial effects of the second to fourth aspects mentioned above can be referred to the relevant descriptions in the first aspect and various possible implementations of the first aspect, which will not be repeated here. Attached Figure Description

[0048] Figure 1 shows a schematic diagram of an application scenario of an interaction method provided in an embodiment of this application.

[0049] Figure 2a shows a schematic diagram of an interface corresponding to a touch-based interaction method provided in an embodiment of this application.

[0050] Figure 2b shows a schematic diagram of an interface corresponding to a touch-based interaction method provided in an embodiment of this application.

[0051] Figure 2c shows a schematic diagram of an interface corresponding to a mouse-based interaction method provided in an embodiment of this application.

[0052] Figure 2d shows a schematic diagram of the display position and style of the task control area in an interactive system consisting of a foldable device and an external display screen, according to an embodiment of this application.

[0053] Figure 3a shows a schematic diagram of the state changes of a two-fold folding device provided in an embodiment of this application during unfolding operation.

[0054] Figure 3b shows a schematic diagram of the state changes of another two-fold folding device provided in the embodiment of this application during unfolding operation.

[0055] Figure 3c shows a schematic diagram of the state changes of a multi-folding device provided in an embodiment of this application during unfolding operation.

[0056] Figure 4 is a schematic diagram of the implementation process of an interaction method provided in an embodiment of this application.

[0057] Figure 5 shows a schematic diagram of an interface for enabling / disabling the super window function according to an embodiment of this application.

[0058] Figure 6a shows a schematic diagram of the user interface for the third user operation provided in this application embodiment, which includes the user dragging the target window downwards.

[0059] Figure 6b shows a schematic diagram of the response interface for a user dragging a target window downwards, which is included in the third user operation provided in this application embodiment.

[0060] Figure 6c shows a schematic diagram of the user interface for the third user operation provided in this application embodiment, which includes the user dragging the target window upwards.

[0061] Figure 6d shows a schematic diagram of the response interface for the third user operation provided in this application embodiment, which includes the user dragging the target window upwards.

[0062] Figure 7a shows a schematic diagram of the user interface for the third user operation provided in this application embodiment, which includes the user dragging the target window downwards and hovering it.

[0063] Figure 7b shows a schematic diagram of the response interface for a third user operation provided in an embodiment of this application, which includes a user dragging a target window downwards and hovering it.

[0064] Figure 7c shows a schematic diagram of the response interface after a user selects a target area in a layout menu triggered by a downward hover, which is part of the third user operation provided in this application embodiment.

[0065] Figure 8a shows a schematic diagram of the user interface for the third user operation provided in this application embodiment, which includes the user dragging the target window upwards and hovering it.

[0066] Figure 8b shows a schematic diagram of the response interface for the third user operation provided in this application embodiment, which includes the user dragging the target window upwards and hovering it.

[0067] Figure 8c shows a schematic diagram of the response interface after a user selects a target area in a layout menu triggered by an upward hover, which is included in the third user operation provided in this application embodiment.

[0068] Figure 9a shows a schematic diagram of an operation interface provided in an embodiment of this application, which includes a user hovering the mouse over a target window.

[0069] Figure 9b shows a schematic diagram of the operation interface corresponding to the user clicking the downward-pointing control in the layout control that is triggered by hovering in the third user operation provided in the embodiment of this application.

[0070] Figure 9c shows a schematic diagram of the response interface corresponding to the user clicking the downward-pointing control in the layout control that is triggered by hovering in an embodiment of this application for the third user operation.

[0071] Figure 9d shows a schematic diagram of another operation interface provided in the embodiment of this application, which includes a user hovering the mouse over the target window.

[0072] Figure 9e shows a schematic diagram of the operation interface corresponding to the user clicking the upward-pointing control in the layout control that is triggered by hovering in the third user operation provided in the embodiment of this application.

[0073] Figure 9f shows a schematic diagram of the response interface corresponding to the user clicking the upward-pointing control in the layout control that is triggered by hovering in the third user operation provided in the embodiment of this application.

[0074] Figure 10 shows a schematic diagram of an application scenario for another interaction method provided in an embodiment of this application.

[0075] Figure 11 shows a schematic diagram of the implementation process of another interaction method provided in the embodiments of this application.

[0076] Figure 12 shows a schematic diagram of an operating system software structure provided in an embodiment of this application.

[0077] Figure 13 shows a schematic diagram of another operating system software structure provided in an embodiment of this application.

[0078] Figure 14 shows a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0080] It is understood that the electronic devices in the embodiments of this application may also be referred to as terminals, terminal equipment, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal equipment may be foldable laptops, foldable mobile phones, foldable tablets, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in smart cities, wireless terminals in smart homes, etc.

[0081] As mentioned earlier, in the semi-open state of a foldable device as shown in Figure 1 above, the path for users to drag windows across screens from screen B to screen C or from screen C to screen B is relatively long, which can cause wear and tear on the screen of the foldable device and may also cause the device to tip over, resulting in a poor user experience.

[0082] To address the aforementioned issues, this application provides an interaction method. Specifically, this method centrally displays the thumbnail windows corresponding to one or more task windows on the screen in a single area, hereinafter referred to as the task control area. Based on this, the foldable device can identify the window being operated on and the target area or position to which the user intends to drag the window, based on the user's operation of instructing the thumbnail windows to move their display positions within the task control area. For example, in the task control area 210 shown in Figure 2a, windows 211, 212, etc., are thumbnail windows corresponding to the display windows of various application interfaces on the screen of the foldable device 10. The user can drag window 211 or window 212 upwards within the task control area 210 to control the actual task window corresponding to window 211 to be displayed on screen B or display area B of the foldable device 10. The user can also drag window 211 or window 212 downwards within the task control area 210 to control the actual task window corresponding to window 211 to be displayed on screen C or display area C of the foldable device 10.

[0083] This allows users to quickly adjust window positions on foldable devices or various collaborative devices that require dragging windows across screens. The operation path for adjusting window positions across screens is shortened; users only need to drag the mouse on the foldable device screen or slide a short distance using a touch object to complete the adjustment. For example, without using a mouse or other access device, users only need to slide their finger up or down a short distance on the corresponding window in the task control area to move the window from one screen to another. The interaction solution provided in this application avoids screen damage caused by swiping operations and improves the efficiency of adjusting the position of windows displayed on the screen, thereby enhancing the user experience.

[0084] It is understood that the aforementioned task control area may also be referred to as the task hub in some embodiments. The electronic device supports the function of displaying the aforementioned task control area, which may be referred to as the super window function in some embodiments, etc., without limitation.

[0085] User operations on the aforementioned task control area are not limited to dragging windows up or down, but may also include dragging the corresponding window up and hovering it, or hovering it over the corresponding window.

[0086] As an example, referring to operation ① shown in Figure 2b, the user can drag window 211 upwards and hover, or press and hover their finger on window 211. At this time, the foldable device 10 can display the layout menu 221 shown in Figure 2b in or above the task control area. The user clicks on the selected layout position in the layout menu 221, and the actual task window corresponding to window 211 can be adjusted to the corresponding layout position in screen B or display area B. Continuing to refer to operation ② shown in Figure 2b, the user can drag window 212 downwards and hover, or press and hover their finger on window 212. At this time, the foldable device 10 can display the layout menu 222 shown in Figure 2C in or below the task control area. The user clicks on the selected layout position in the layout menu 222, and the actual task window corresponding to window 212 can be adjusted to the corresponding layout position in screen C or display area C.

[0087] User operations on the aforementioned task control area can also include hovering the mouse over the corresponding window in the task control area and clicking on the brought-up control, etc., without limitation. As an example, referring to operation ③ shown in Figure 2c, the user can hover the mouse over window 211. At this time, the foldable device 10 can display the layout control shown in Figure 2c on window 211. The user moves the mouse to the upward control 231 in the layout control and clicks the control, which will adjust the actual task window corresponding to window 211 to screen B or display area B. Continuing to refer to operation ④ shown in Figure 2c, the user can hover the mouse over window 212. At this time, the foldable device 10 can display the layout control shown in Figure 2c on window 212. The user moves the mouse to the downward control 232 in the layout control and clicks the control, which will adjust the actual task window corresponding to window 212 to screen C or display area C.

[0088] It is understood that the electronic devices to which the interaction method provided in this application is applicable are not limited to foldable devices, but may also include two or more cross-screen collaborative devices that require dragging windows across screens. In this case, the display position and style of the task control area may not be limited to the positions and styles exemplified in Figures 2a to 2c. Referring to Figure 2d, for the foldable device 10 and the external display screen 20, the task control area 240 may also be displayed on the right edge of screen B. When the user moves the mouse and hovers over the corresponding window in the task control area 240, they can click the control indicating right in the corresponding brought-up layout controls to control the corresponding actual task window to be displayed on screen A of the external display screen 20. In some embodiments, the placement orientation of the foldable device 10 and the external display screen 20 may be opposite or different from that shown in Figure 2b. The foldable device 10 may also be other forms of electronic devices such as laptops, desktop computers, or tablets. The display position and style of the task control area on the screen of the foldable device 10 may also be other positions and styles, which are not limited here.

[0089] It is understood that the foldable device to which the interaction method provided in this application is applicable can be a bi-fold foldable device, a tri-fold foldable device, or a multi-fold foldable device, and no limitation is made here. In an example scenario, referring to Figure 3a, the foldable screen 10 is a bi-fold foldable device 100, which may include a main screen 100a, a secondary screen 100b, a folding axis 103, a folding part 104, and a folding part 105. The main screen 100a is located on the folding part 104, and the secondary screen 100b is located on the folding part 105. The folding part 104 and the folding part 105 are connected by the folding axis 103. Thus, the bi-fold foldable device 100 can support the unfolding operation ④ of unfolding the folding part 104 and the folding part 105 to the same plane. After the bi-fold foldable device 100 is unfolded, the main screen 100a and the secondary screen 100b can form a larger display screen.

[0090] It is understood that when the angle α between the folding parts 104 and 105 unfolding based on the folding axis 103 is less than the preset angle for triggering entry into the folded state, the two-fold folding device 100 is in the folded state. In this folded state, the main screen 100a can be in an active state, which can be used to display the application interface or respond to user operations, while the secondary screen 100b can be in an inactive state, such as a state where the screen is off and no interface is displayed. When the angle α between the folding parts 104 and 105 unfolding based on the folding axis 103 is greater than the aforementioned preset angle, the two-fold folding device 100 can switch to the unfolded state. In this unfolded state, both the main screen 100a and the secondary screen 100b can be in an active state, that is, both the main screen 100a and the secondary screen 100b display the display interface. At this time, based on the interaction method provided in this application, the user can quickly adjust the display position of the task window on the two-fold folding device 100 to be displayed on the main screen 100a or on the secondary screen 100b.

[0091] In some embodiments, referring to FIG3b, the bi-fold folding device 100 may further include a secondary screen 100c. For example, the main screen 100a is located on the front of the folding part 104, the secondary screen 100c is located on the back of the folding part 104, and the secondary screen 100b is located on the folding part 105. The user may also use other methods to unfold the bi-fold folding device 100, such as unfolding operation ⑤ shown in FIG3b, in which the user unfolds the folded secondary screens 100b and 100c of the bi-fold folding device 100 to the same plane. After the bi-fold folding device 100 is unfolded, the secondary screens 100c and 100b can form a larger display screen. In this folded state, the main screen 100a may be in an active state, while the secondary screens 100c and 100b may both be in an inactive state. When the included angle α between the folding part 104 and the folding part 105 unfolded based on the folding axis 103 is greater than the aforementioned preset angle, the bi-fold folding device 100 may switch to the unfolded state. In this unfolded state, both secondary screens 100c and 100b can be active, while the main screen 100a can be inactive. At this time, based on the interaction method provided in this application, the user can quickly adjust the display position of the task window on the bi-fold device 100 to be displayed on either secondary screen 100b or secondary screen 100c.

[0092] In another example scenario, referring to Figure 3c, the foldable device can be a tri-fold device 200. This tri-fold device 200 may include a main screen 200a, secondary screens 200b, 200c, and 200d, a folding axis 204, a folding axis 205, a folding section 206, a folding section 207, and a folding section 208. The main screen 200a is located on the front of the folding section 206, the secondary screen 200b is located on the back of the folding section 206, the secondary screen 200c is located on the folding section 207, and the secondary screen 200d is located on the folding section 208. The folding sections 206 and 207 are folded together via the folding axis 204, and the folding sections 207 and 208 are folded together via the folding axis 205. Based on the folding axis 204, the tri-fold device 200 can support an unfolding operation that allows the folding sections 206 and 207 to unfold to the same plane. It can be understood that the unfolding operation ⑥ involves unfolding the secondary screen 200c, which is folded behind the main screen 200a in the tri-fold folding device 200, until it is on the same plane as the secondary screen 200b. After the tri-fold folding device 200 is unfolded based on the unfolding operation ⑥, the secondary screens 200b and 200c can form a larger display screen.

[0093] When the angle α1 between the folding parts 206 and 207 unfolded based on the folding axis 204 is less than the aforementioned preset angle, and the angle α2 between the folding parts 207 and 208 unfolded based on the folding axis 205 is also less than the aforementioned preset angle, the two-fold folding device 100 is in a folded state. In this folded state, the main screen 200a can be in an active state, while the secondary screens 200b, 200c, and 200d can all be in an inactive state. When the angle α1 between the folding parts 206 and 207 unfolded based on the folding axis 204 is greater than the aforementioned preset angle, the two-fold folding device 100 can assume that the user has performed an unfolding operation. At this time, the state where the angle α1 of the two-fold folding device 100 is greater than the aforementioned preset angle and the angle α2 is less than the aforementioned preset angle can be referred to as the semi-unfolded state. In this semi-unfolded state, the secondary screens 200b and 200c can both be in an active state, while the main screen 200a and 200d can both be in an inactive state. At this time, based on the interaction method provided in this application, the user can quickly adjust the display position of the task window to be displayed on the secondary screen 200b or the secondary screen 200c on the two-fold folding device 100.

[0094] Furthermore, the user can also unfold the folded secondary screen 200d of the tri-fold device 200 to the same plane as the secondary screens 200b and 200c through the unfolding operation ⑦. After the secondary screen 200d is unfolded, the secondary screens 200b, 200c, and 200d can form a larger display screen. At this time, the state in which the tri-fold device 200 unfolds the secondary screens 200b, 200c, and 200d to the same plane based on the unfolding operations ⑥ and ⑦ is referred to as the unfolded state. In this unfolded state, the secondary screens 200b, 200c, and 200d can all be active, while the main screen 200a can be inactive. At this time, the tri-fold device 200 can display one or more task windows on the secondary screens 200c and 200d, and one or more task windows such as the input method soft keyboard can be displayed on the secondary screen 200b. Among them, the secondary screens 200c and 200d form a larger display screen based on the aforementioned unfolding operation ⑦. At this time, based on the interaction method provided in this application, the user can quickly adjust the display position of the task window on the tri-fold device 200 to be displayed on the secondary screen 200b, or to be displayed on the larger display screen formed by the secondary screens 200c and 200d.

[0095] This application does not impose any limiting description on the location of the folding axis, the folding method, or the folding direction of the foldable device.

[0096] Based on the interaction methods provided by the task control area in the examples of Figures 2a to 2d above, as well as the display position and style of the task control area, the implementation process of the interaction method provided in this application will be described in detail below with specific embodiments for different application scenarios.

[0097] The following section will describe in detail the specific implementation process of the interaction method provided in this application in a foldable device, with reference to Embodiment 1.

[0098] Example 1

[0099] This application provides a specific implementation process of an interaction method applied to a foldable device. The foldable device can refer to the foldable device 10 in the scenario shown in Figure 1 above, and can be an electronic device such as a foldable screen laptop, foldable screen phone, foldable screen tablet, or foldable large-screen device. The following continues to use the foldable device 10 in the scenario shown in Figure 1 as an example to describe the specific implementation process of the interaction method provided in this application.

[0100] Figure 4 illustrates a schematic diagram of the implementation process of an interaction method according to an embodiment of this application. It can be understood that the executing entity for each step of the process shown in Figure 4 can be a foldable device 10. As mentioned earlier, the foldable device may include electronic devices such as foldable laptops or foldable phones, foldable tablets, and large-screen foldable devices.

[0101] Specifically, as shown in Figure 4, the implementation process may include the following steps:

[0102] 401: An unfolding operation within the preset angle range was detected.

[0103] For example, the foldable device 10 may include the two-fold folding device supporting different folding methods illustrated in Figures 3a and 3b, the three-fold folding device illustrated in Figure 3c, and other multi-fold folding devices, as well as other foldable devices supporting different folding methods, etc., without limitation. As an example, referring to Figures 3a to 3c, the unfolding operation detected by the foldable device may include unfolding operation ④ shown in Figure 3a, unfolding operation ⑤ shown in Figure 3b, unfolding operations ⑥ and ⑦ shown in Figure 3c, etc.

[0104] It is understood that during the process of the foldable device 10 transitioning from a folded state to an unfolded state, when the included angle between adjacent screens connected by the folding axis is greater than a small preset angle (which can be denoted as the first preset angle), the two adjacent screens are activated, and the foldable device 10 can first enter the aforementioned semi-unfolded state. At this time, the foldable device 10 can continue to implement the following steps 402 to 408, responding to user operations to display the task control area and adjusting the position of the relevant task windows in the display area of ​​each screen based on the task control area, that is, continuing to implement the interaction method provided in the embodiments of this application.

[0105] As the foldable device 10 continues to unfold, when the angle between two adjacent screens connected by the folding axis reaches a relatively large preset angle (which can be denoted as the second preset angle), for example, 180 degrees, the two screens can combine into a larger screen. In other words, the two adjacent screens, as different display areas of the same screen, combine to form the same larger display area. At this time, the foldable device 10 is in the unfolded state and can display the interface through the aforementioned larger screen or larger display area. At this point, the foldable device 20 does not require cross-screen operation and therefore does not need to perform the following steps. It can open the window controls for the Super Window function without displaying the task control area.

[0106] 402: In response to the first user action, display a window control that enables the SuperWindow feature.

[0107] For example, the first user operation includes opening the control center, or opening the settings interface for the Super Window function, etc.

[0108] As an example, referring to Figure 5, when the foldable device 10 is in the aforementioned semi-open state, the user can open the control center 510 from the taskbar displayed on the screen C of the foldable device 10. As shown in Figure 5, the control center 510 may include a window control 511 that can quickly open the super window function.

[0109] It is understood that in this embodiment, the first user operation described above is, for example, the user operation of opening the control center 510 as shown in Figure 5. In other embodiments, the first user operation described above may also be a voice command operation, etc. No limitation is made here.

[0110] 403: A second user action was detected on the window control. The task control area corresponding to the Super Window function is displayed.

[0111] For example, the second user operation mentioned above may include a user clicking on a window control, and a user instructing the foldable device 10 to activate the super window function controlled by the window control via voice command, etc., without limitation.

[0112] As an example, continuing to refer to Figure 5, when a user clicks the window control 511 in the control center 510, the foldable device 10 can display the task control area 520. This task control area 520 can be used to control the currently displayed individual task windows, and each thumbnail window displayed in the task control area 520 corresponds to one of the currently open individual task windows on the foldable device 10. It can be understood that these individual task windows can be the interface display windows of applications running on the foldable device 10.

[0113] It is understood that the task control area 520 shown in Figure 5 can be displayed on screen B, and can be displayed near the junction of screen B and screen C. In some other embodiments, the display position and style of the task control area may differ from the display position and style exemplified in Figure 5. For example, the task control area may also be displayed on screen C, and this is not a limitation.

[0114] In other embodiments, the task control area 520 may not display a thumbnail window, but instead display icons of relevant applications or other identifiers that can indicate the actual task window of the corresponding application, etc., without limitation.

[0115] 404: A third-user action was detected acting on the target window within the task control area.

[0116] For example, the third user operation includes at least one of the following: dragging the target window up or down (refer to Figure 2a above), dragging the target window up and then hovering it or hovering it over the corresponding window (refer to Figure 2b above), and using the mouse to hover over the target window in the task control area and click on the brought-up control (refer to Figure 2c above). These are not limited here. It is understood that the target window can be any window among one or more task windows displayed in the task control area, and the third user operation can control the target window to be displayed in any display area of ​​screen B or screen C of the foldable device 10. These are not limited here.

[0117] It is understood that the foldable device 10 may preset relevant thresholds corresponding to the aforementioned third user operation as recognition conditions for identifying the third user operation and triggering the foldable device 10 to continue executing the following steps. For example, referring to FIG2a above, the foldable device 10 may preset a first distance threshold corresponding to the user's upward or downward movement as a recognition condition for detecting whether the user's operation of dragging the target window upward or downward can trigger the implementation of step 405 below. As another example, referring to FIG2b above, the foldable device 10 may preset a second distance threshold corresponding to the user's upward or downward movement before hovering as a recognition condition for detecting whether the user wants to bring up the layout menu, and this recognition condition can be used to identify the user's intent. It is understood that in some embodiments, the aforementioned first distance threshold may be greater than the aforementioned second distance threshold.

[0118] In other embodiments, referring to Figure 2c above, the foldable device 10 may also preset a time threshold corresponding to the duration of mouse hover as a recognition condition for detecting whether the user wants to call up the up or down control, etc., without limitation.

[0119] Specifically, the process of a user performing the aforementioned third user operation on the screen of the foldable device 10, and the corresponding interface changes displayed by the foldable device 10 in response to the aforementioned third user operation, will be described in detail below with reference to relevant interface diagrams, and will not be repeated here.

[0120] 405: Determine the coordinates of the display area of ​​the target window for the third user operation instruction display.

[0121] For example, the foldable device 10 can determine the coordinates of the display area indicated by the operation based on the drag direction of the third user operation or the position information of the display area selected by the third user operation. In some embodiments, the coordinates of the display area can be determined based on the position information of the layout area selected by the user in the layout menu. For example, if the user drags the target window up and hovers it, and selects a layout area under one of the layout styles in the layout menu that triggers the display, the foldable device 10 can display the actual task window corresponding to the target window in that layout area.

[0122] It is understood that the aforementioned display area coordinates may include a set of coordinate information corresponding to the boundary position of the display area. This set of coordinate information may include the coordinate values ​​of each point in the display area that can be determined in the display screen of the foldable device, which will not be elaborated here.

[0123] 406: Display the actual task window corresponding to the target window in the target display area indicated by the determined display area coordinates.

[0124] For example, the foldable device 10 determines the corresponding target display area based on the display area coordinates determined above. Referring to Figures 2a to 2c above, this display area can be located in screen B of the foldable device 10 or in screen C of the foldable device 10, without limitation. Furthermore, the foldable device 10 can control the display of the actual application window corresponding to the target window in the target display area. It can be understood that the actual application window can be the interactive interface display window of an application such as an office application or a browser application.

[0125] The following section, in conjunction with Figures 6a to 6d, 7a to 7c, 8a to 8c, and 9a to 9f, describes the relevant interface changes involved in performing steps 404 to 406 of the foldable device 10. Specifically:

[0126] Figures 6a to 6d illustrate a schematic diagram of an interface change process triggered by an operation mode of a third user operation according to an embodiment of this application.

[0127] Referring to Figure 6a, after the foldable device 10 detects the second user operation in step 403, it can display the task control area 520 shown in Figure 6a, which is the same as the task control area 520 shown in Figure 5. As shown in Figure 6a, the user can drag the window 521 downwards in the task control area 520 displayed by the foldable device 10. This window 521 can be a thumbnail window corresponding to the actual task window 610 currently displayed on screen B. Therefore, the operation of dragging the window 521 downwards as shown in Figure 6a can instruct the foldable device 10 to display the actual task window 610 on screen C, or in other words, adjust the actual task window 610 to be displayed on screen C. The window 521 in the task control area 520 can be a thumbnail window of the actual task window 610.

[0128] It is understood that when the foldable device 10 detects that the user has dragged the window 521 downwards by a distance exceeding a first preset distance, it determines to display the actual task window 610 corresponding to the window 521 on the screen C. At this time, the foldable device 10 can continue to display the interface shown in Figure 6b.

[0129] Referring to Figure 6b, in response to a third user operation where the user drags down a thumbnail window within the task control area, the foldable device 10 can display the actual task window 610 corresponding to the thumbnail window on screen C. At this time, the user can continue to perform other operations within the actual task window 610 displayed on screen C to achieve interaction with the application corresponding to the actual task window 610.

[0130] Referring to Figure 6c, after the foldable device 10 detects the second user operation in step 403, it can display the task control area 520 shown in Figure 6c, which is the same as the task control area 520 shown in Figure 5. The user can drag window 522 upwards within the task control area 520 displayed on the foldable device 10. This window 522 can be a thumbnail corresponding to the actual task window 620 currently displayed on screen C. Therefore, the operation of dragging window 522 upwards as shown in Figure 6c can instruct the foldable device 10 to display the actual task window 620 on screen B, or to adjust the actual task window 620 for display on screen B. Window 522 in the task control area 520 can be a thumbnail of the actual task window 620. Window 522 and window 521 can be the same thumbnail or different thumbnail windows. Correspondingly, the actual task window 620 can be the same task window or a different task window; this is not limited here.

[0131] It is understood that when the foldable device 10 detects that the user has dragged the window 522 upwards by a distance exceeding a first preset distance, it determines to display the actual task window 620 corresponding to the window 522 on the screen B. At this time, the foldable device 10 can continue to display the interface shown in Figure 6d.

[0132] Referring to Figure 6d, in response to a third user operation where the user drags a thumbnail window down within the task control area, the foldable device 10 can display the actual task window 620 corresponding to the thumbnail window on screen B. At this time, the user can continue to perform other operations within the actual task window 620 displayed on screen B to achieve interaction with the application corresponding to the actual task window 620.

[0133] Figures 7a to 7c illustrate a schematic diagram of the interface change process triggered by another operation mode of a third user operation according to an embodiment of this application.

[0134] Referring to Figure 7a, after the foldable device 10 detects the second user operation in step 403, it can display the task control area 520 shown in Figure 7a, which is the same as the task control area 520 shown in Figure 5 above. Referring to operation ⑧ shown in Figure 7a, the user can drag the window 521 down a certain distance and then hover it in the task control area 520 displayed on the foldable device 10. This operation ⑧ can trigger the foldable device 10 to display a layout menu, the form of which can be referred to as the layout menu 710 shown in Figure 7b below.

[0135] Referring to Figure 7b, in response to the aforementioned operation ⑧, the foldable device 10 displays a layout menu 710 above the layer containing the task control area 520. As shown in Figure 7b, the layout menu 710 can be displayed below the dragged window 521, corresponding to the downward movement operation in operation ⑧. This layout menu 710 can display various recommended layouts, each controlling the display area of ​​the actual task window 720 corresponding to window 521 on the lower screen C. For example, the user can click on the recommended window 711 in the layout menu 710 shown in Figure 7b. In response to the user's selection of the recommended window 711, the foldable device 10 can display the actual task window 720 in the lower area of ​​screen C, as shown in Figure 7c, where the actual task window 720 is displayed on screen C. Window 521 can be a thumbnail of the actual task window 720 displayed on screen B as shown in Figures 7a and 7b. This window 720 can be the same as or different from the aforementioned actual task window 610; no restrictions are placed here.

[0136] Referring again to Figure 7c, the area 730 above the screen C of the foldable device 10 can display task windows corresponding to other applications that were originally displayed, such as the input method soft keyboard window, or it can choose not to display other task windows.

[0137] Figures 8a to 8c illustrate the interface change process triggered by another operation mode of a third user operation according to an embodiment of this application.

[0138] Referring to Figure 8a, after the foldable device 10 detects the second user operation in step 403, it can display the task control area 520 shown in Figure 8a, which is the same as the task control area 520 shown in Figure 5 above. Referring to the operation ⑨ shown in Figure 8a, the user can drag the window 521 upwards a certain distance and then hover it in the task control area 520 displayed on the foldable device 10. This operation ⑨ can trigger the foldable device 10 to display a layout menu, the form of which can be referred to as the layout menu 810 shown in Figure 8b below.

[0139] Referring to Figure 8b, in response to the aforementioned operation ⑨, the foldable device 10 displays a layout menu 810 above the layer containing the task control area 520. As shown in Figure 8b, the layout menu 810 can be displayed below the dragged window 521, corresponding to the upward movement operation in operation ⑨. This layout menu 810 can display various recommended layouts, each controlling the display area of ​​the actual task window 820 corresponding to window 521 on the upper screen B. For example, the user can click on the recommended window 811 in the layout menu 810 shown in Figure 8b. In response to the user's selection of the recommended window 811, the foldable device 10 can display the actual task window 820 in the upper left area of ​​screen B, as shown in Figure 8c. Window 521 can be a thumbnail of the actual task window 820 displayed on screen C as shown in Figures 8a and 8b. This window 820 can be the same as or different from the aforementioned actual task window 610; no restrictions are placed here.

[0140] Referring again to Figure 8c, the upper right and lower display areas 830 of the screen B of the foldable device 10 can display task windows corresponding to other applications that were originally displayed, such as the input method soft keyboard window, or they can choose not to display other task windows.

[0141] Figures 9a to 9f illustrate the interface change process triggered by another operation mode of a third user operation according to an embodiment of this application.

[0142] Referring to Figure 9a, after the foldable device 10 detects the second user operation in step 403, it can display the task control area 520 shown in Figure 9a, which is the same as the task control area 520 shown in Figure 5 above. As shown in Figure 9a, the user can hover the mouse 910 over the window 521 in the task control area 520. At this time, the foldable device 10 can respond to the mouse hover operation and display layout controls 911 and 912 on the window 521. Layout control 911 can be used to adjust the actual task window corresponding to window 521 to be displayed in the lower screen C, or it can be described as a downward pointing control; layout control 912 can be used to adjust the actual task window corresponding to window 521 to be displayed in the upper screen B, or it can be described as an upward pointing control.

[0143] Referring again to Figure 9b, the user can use mouse 910 to click on the layout control 911 displayed on the foldable device 10. Correspondingly, in response to the mouse click, the foldable device 10 can control the actual task window 920 corresponding to window 521 to be displayed on screen C indicated by the layout control 911, as shown in Figure 9c. It can be understood that this actual task window 920 could originally be a window displayed on screen B, and in some other embodiments, this actual task window 920 could originally be a window displayed on screen C.

[0144] Referring again to Figure 9d, the user can hover the mouse over window 522 in the task control area 520. In response to this mouse hover, the foldable device 10 can display layout controls 911 and 912 on window 522. Referring to Figure 9e, the user can then click the layout control 912 displayed on the foldable device 10 using the mouse 910. Correspondingly, referring to Figure 9f, in response to the mouse click, the foldable device 10 can control the actual task window 930 corresponding to window 522 to be displayed on screen B as indicated by the layout control 912. The actual task window 930 displayed on screen B is shown in Figure 9f. It can be understood that this actual task window 930 could originally be a window displayed on screen C; in other embodiments, the task window 930 could also originally be a window displayed on screen B.

[0145] It is understood that in some embodiments, the foldable device 10 may not display other windows before adjusting the actual task window corresponding to the thumbnail window of the user-operated task control area to be displayed on the screen C in response to the aforementioned third user operation. In this case, the foldable device 10 can display the relevant actual task window in the central area of ​​the screen C in response to the third user operation, and the actual task window can be enlarged to fit the size of the screen C.

[0146] In some embodiments, the screen C of the foldable device 10 may display the task window of an input method soft keyboard or other applications before displaying the actual task window of the third user operation instruction. In this case, in response to the third user operation, the foldable device 10 can reduce the size of the task window of the input method soft keyboard or other applications originally displayed on the screen C, and display the actual task window of the third user operation instruction in an area of ​​the screen C where no other windows are currently displayed. In some embodiments, in response to the third user operation, the foldable device 10 may also display the actual task window of the third user operation instruction above the task window of the input method soft keyboard or other applications originally displayed on the screen C, or control the display area of ​​the actual task window to partially overlap with the display area of ​​the task window of the input method soft keyboard or other applications, etc., without limitation. The aforementioned actual task window may include the aforementioned actual task window 610, actual task window 720, and actual task window 920, etc., without limitation.

[0147] Similarly, in some embodiments, the screen B of the foldable device 10 may not display other windows before displaying the actual task window of the third user operation instruction. In this case, in response to the third user operation, the foldable device 10 can control the actual task window of the third user operation instruction to be displayed in the central area of ​​the screen B, and the actual task window can be enlarged to fit the size of the screen B.

[0148] In some embodiments, the screen B of the foldable device 10 may display task windows of other applications before displaying the actual task window of the third user operation instruction. In this case, in response to the third user operation, the foldable device 10 can reduce the size of the task windows of other applications originally displayed on the screen B and display the actual task window of the third user operation instruction in an area of ​​the screen B where no other windows are currently displayed. In some embodiments, in response to the third user operation, the foldable device 10 may also display the actual task window of the third user operation instruction above the task windows of other applications originally displayed on the screen B, or control the display area of ​​the actual task window to partially overlap with the display areas of the task windows of other applications, etc., without limitation. The actual task window may include the actual task window 610, actual task window 720, and actual task window 920, etc., without limitation.

[0149] It is understood that, based on the implementation process of steps 401 to 406 shown in Figure 4 above, and the operation mode and related interface change process of the third user operation illustrated in Figures 6a to 6d, 7a to 7c, 8a to 8c, and 9a to 9f above, when the interaction method provided in this application embodiment is applied to the foldable device 10, it can provide users with more convenient cross-screen operation services such as adjusting the display position of the window across screens. Furthermore, the task control area displayed near the screen folding axis of the foldable device 10 or on the lower screen C in the semi-open state can effectively avoid the problem that the device may tip over when the user applies pressure while sliding on the screen of the foldable device 10.

[0150] The following section will describe in detail the specific implementation process of the interaction method provided in this application in the scenario of an external extended display screen, with reference to Embodiment 2.

[0151] Example 2

[0152] This application provides a specific implementation process of an interaction method applied to an external extended display screen of an electronic device. The electronic device may include mobile phones, tablets, laptops, desktop computers, large-screen devices, etc. The electronic device may be a foldable device or an electronic device with a non-foldable display screen; no limitation is made herein.

[0153] Figure 10 illustrates a scenario of an external extended display screen for an electronic device according to an embodiment of this application.

[0154] As shown in Figure 10, the scenario includes an electronic device 30 and an external extended display screen 20. The extended display screen 20 can be connected to the electronic device 30 and provide an extended screen display area for the electronic device 30 to facilitate user use.

[0155] Figure 11 illustrates a schematic diagram of the implementation flow of an interaction method according to an embodiment of this application. It can be understood that the executing entity for each step of the process shown in Figure 11 can be the aforementioned electronic device 30. As mentioned above, the electronic device 30 can be the aforementioned tablet, laptop, desktop computer, large-screen device, etc., or it can be the aforementioned foldable device 10; no limitation is made here.

[0156] Specifically, as shown in Figure 11, the implementation process may include the following steps:

[0157] 1101: A connection to the extended display has been detected.

[0158] For example, the electronic device 30 can establish a connection with the extended display screen 20 via a data cable. The interface type of the data cable may include High Definition Multimedia Interface (HDMI), mini HDMI, micro HDMI, or Universal Serial Bus (USB), etc. In other embodiments, the electronic device 30 and the extended display screen 20 may also establish a data connection via wireless communication, which is not limited here.

[0159] It is understandable that after the electronic device 30 and the extended display screen 20 establish a connection, they can transmit display data related to windows or interfaces and display the task windows of some applications running on the electronic device 30.

[0160] 1102: In response to the first user action, display window controls that enable the SuperWindow function.

[0161] The first user operation can refer to the operation of opening the control center 510 in the taskbar as illustrated in Figure 5 of Embodiment 1 above. The detailed process of the specific electronic device 30 performing this step 1102 can be referred to the relevant description of step 402 in Embodiment 1 above, and will not be repeated here.

[0162] 1103: A second user action on the window control was detected, and the task control area corresponding to the Super Window function was displayed.

[0163] The first user operation can refer to the operation of clicking the window control 511 as illustrated in Figure 5 of Embodiment 1 above. The detailed process of the specific electronic device 30 performing this step 1103 can be referred to the relevant description of step 403 in Embodiment 1 above, and will not be repeated here.

[0164] 1104: A third-user action was detected acting on a target window within the task control area.

[0165] For example, the display position and style of the task control area in this application embodiment can refer to the display position and style exemplified in Figure 2d above. The specific operation method of the third user operation can refer to the relevant description of the control that indicates right or left click in Figure 2d above, which will not be repeated here.

[0166] For a detailed description of the specific electronic device 30 performing step 1104, please refer to the relevant description of step 404 in the above embodiment 1, which will not be repeated here.

[0167] 1105: Determine the coordinates of the display area of ​​the target window for the third user operation instruction display.

[0168] For example, in the embodiments of this application, the display area coordinates of the target window for the third user operation instruction can be located in screen A of the extended display screen 20 in the scenario shown in FIG10, or in screen B of the electronic device 30, or in screen C of the foldable device 10, without limitation.

[0169] For a detailed description of the specific electronic device 30 performing step 1105, please refer to the relevant description of step 405 in the above embodiment 1, which will not be repeated here.

[0170] 1106: Display the actual task window corresponding to the target window in the target display area indicated by the determined display area coordinates.

[0171] For a detailed description of the specific electronic device 30 performing step 1106, please refer to the relevant description of step 406 in the above embodiment 1, which will not be repeated here.

[0172] It is understood that, based on the implementation flow of steps 1101 to 1106 shown in Figure 11 above, when the interaction method provided in this application embodiment is applied to other electronic devices different from the foldable device 10, it can provide users with more convenient cross-screen operation services such as adjusting the display position of windows across screens, making it convenient for users to switch between the local display screen of the electronic device 30 and the extended display screen 20 to display the actual task window. Furthermore, when the electronic device 30 is the foldable device 10, it also allows users to easily transfer the actual task window displayed on the foldable device 10 to the extended display screen 20. In this scenario, the interaction method provided in this application can effectively avoid the problem of the device tipping over when the user applies pressure while sliding on the screen of the foldable device 10.

[0173] The operating system software structure and hardware structure of the electronic device to which the interaction method provided in the embodiments of this application is applied will be described exemplarily below with reference to the accompanying drawings.

[0174] Figure 12 is a schematic diagram of the software structure of an operating system according to an embodiment of this application.

[0175] It is understood that the operating system of electronic device 30 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered architecture of Windows... TM Taking the system as an example, the system software structure of electronic device 30 is illustrated.

[0176] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces.

[0177] Windows TMOperating systems can generally be divided into kernel mode and user mode.

[0178] User mode can include applications such as system applications, office applications, conferencing applications, video applications, and creative applications, as shown in Figure 12. Applications in user mode can call the core capability set and drivers in kernel mode based on the Windows subsystem and the native application programming interface (native API).

[0179] Referring again to Figure 12, the kernel mode can provide a super window component, a control center, and a driver management module. The super window component provides the electronic device 30 with the ability to manage window layouts. This component can integrate window layout menus, task hub windows, and floating ball capabilities.

[0180] The task hub window corresponds to the window displayed in the task control area. Users can manipulate the thumbnail window within the task hub window to adjust the display position of the corresponding actual task window. The task hub window is the core window of the super window component, providing the ability to display thumbnail windows corresponding to the actual task window and offering convenient layout capabilities such as adjusting the layout of the actual task window by sliding the thumbnail window up and down. In other embodiments, the task hub window, or task control area, may display icons or window icons of relevant applications; this is not a limitation.

[0181] The window layout menu can be used to display the layout menu for the target window when the trigger conditions are met. For example, when a user moves and hovers the target window in the task hub window, a quick layout menu pops up to facilitate the user to quickly preview the layout effect. Real-time layouts include layout menu 221 or layout menu 222 shown in Figure 2b above, or layout menu 710 shown in Figure 7b above, or layout menu 810 shown in Figure 8b above, etc.

[0182] The hover ball can be used to provide a convenient entry point for collapsing and expanding the task hub window; the hover ball can be closed in the control center.

[0183] The control center can be used to provide a switch entry for the super window function, such as by displaying the aforementioned window control 511, etc.

[0184] The driver management module can be used to manage various types of drivers, including display drivers and integrated sensor drivers, and provides driver version installation and upgrade capabilities, for example.

[0185] The SuperView component can also include Design for X (DFX) plugins for each stage of the product lifecycle, providing SuperView with design capabilities for each stage of the product lifecycle (DFX capabilities for short). Electronic devices can provide DFX capabilities to the SuperView component through the HiViewDFX plugin (a type of DFX plugin).

[0186] Continuing to refer to Figure 12, Windows TM The kernel mode of the operating system can also include integrated sensor drivers (ISH drivers) and display drivers. The ISH driver can provide device form factor switching notifications. For example, when electronic device 30 is a foldable device, the ISH driver can provide notifications related to form factor switching, such as when the foldable device transitions from a folded state to a semi-open state, and vice versa.

[0187] Continuing to refer to Figure 12, Windows TM The kernel mode of an operating system can also provide basic capability components and backend business components.

[0188] The basic capability components include gesture recognition, upgrade functionality, shape change recognition, and the ability to listen for keyboard and mouse events and window events. These basic capability components can provide foundational capabilities for backend business logic, window layout menus, task hub windows, floating balls, and other capability components.

[0189] Backend business components can provide backend characteristic business logic processing, such as icon management, drop-down algorithms, waterfall layout, window memory, and window movement animation effects.

[0190] Continuing to refer to Figure 12, Windows TM The kernel mode of the operating system may also include a basic input / output system (BIOS), which is used to receive input and output signals and calculation and perception results provided by embedded controllers (EC, also known as EC chips) and integrated sensor frameworks (ISH FW), including keyboard magnetic attraction signals from electronic devices 30, opening and closing signals from foldable devices, and calculation and perception results determined by ISH FW based on sensor information such as gyroscope sensors and accelerometers for angle calculation and pattern recognition, etc., which will not be elaborated here.

[0191] Continuing with the layered architecture of Harmony TMTaking the system as an example, the system software structure of electronic device 30 is illustrated.

[0192] Figure 13 provides a schematic diagram of the software structure of another operating system according to an embodiment of this application.

[0193] As shown in Figure 13, the electronic device 30 can be equipped with a distributed system, such as HarmonyOS. TM Operating System (Harmony) TM This distributed system can adopt a layered architecture, which may include an application layer, an application framework and system service layer, a driver layer, and a hardware abstraction layer (HAL).

[0194] The application layer can include a series of application packages. These application packages can be system applications, office applications, conferencing applications, video applications, and authoring applications, such as application packages that include input methods, themes, video, and browsers; there are no restrictions on this.

[0195] The application framework and system service layer provides distributed framework services to the application layer. System services, the core of the distributed operating system, provide services to applications and SDKs within the application layer through the application framework. These distributed framework services may include desktop management services, window management services, graphics services, the Ark development framework, and a multimodal input framework. Specifically, the desktop management service can determine whether a cross-screen window repositioning gesture has been detected based on the foldable device's posture recognition and finger touch events, and then notify the window management service.

[0196] The window management service can adjust the window display position based on detected cross-screen gestures, manage the actual task window corresponding to the target window to be displayed across the screen portion indicated by the aforementioned gestures, and process cross-screen layout windows.

[0197] The application framework and system service layer may also include system services such as folding gesture feedback, layer control and compositing, display memory management, and touchscreen management. Touchscreen management, in particular, can be used to report touch events generated by users' fingers on the touchscreen, for example, by reporting them to the desktop management service.

[0198] The driver layer, also known as the kernel layer, can include various driver programs used to control the hardware devices controlled by the hardware abstraction layer to perform related functions. In the embodiments of this application, the driver layer may include sensor drivers, frame buffers (a type of display device driver), and touch drivers, etc.

[0199] The Hardware Abstraction Layer (HAL) is an interface layer located between the operating system kernel and hardware circuitry, used to abstract hardware devices. In the embodiments of this application, the hardware devices abstracted in the HAL may include sensors or sensor modules, touch screens, keyboards, mice, etc., and are not limited thereto.

[0200] Figure 14 shows a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application.

[0201] As shown in Figure 14, the electronic device 30 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identity module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0202] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 30. In other embodiments of this application, the electronic device 30 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0203] Processor 110 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0204] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0205] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the aforementioned memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0206] In this embodiment, the processor 110 can control the fetching and execution of instructions from the memory through the controller, so that the electronic device 30 can implement the steps of the implementation process shown in FIG4 of Embodiment 1 or the steps of the implementation process shown in FIG11 of Embodiment 2, so as to realize the interactive method provided in this application.

[0207] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a universal serial bus (USB) interface, etc.

[0208] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 30. In other embodiments of this application, the electronic device 30 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0209] The wireless communication function of electronic device 30 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0210] Electronic device 30 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0211] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In this embodiment, the display screen 194 of the electronic device 30 may include screens B and C of the foldable device 10 in its semi-open state. In some embodiments, the electronic device 30 may include one or N display screens 194, where N is a positive integer greater than 1.

[0212] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 30. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0213] Internal memory 121 can be used to store computer executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc. The data storage area may store data created during the use of electronic device 30 (such as audio data, phone book, etc.). In addition, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 30 by running instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0214] Electronic device 30 can implement audio functions such as music playback and recording through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0215] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 30 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 30 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 30 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands.

[0216] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 30. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 30 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 30, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 30 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0217] The 180E accelerometer sensor can detect the magnitude of acceleration in various directions (typically three axes) of an electronic device 30. When the electronic device 30 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of the electronic device and is applied to applications such as screen orientation switching and pedometers.

[0218] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 30, in a different position than display screen 194.

[0219] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 30 can receive button input and generate key signal inputs related to user settings and function control of electronic device 30.

[0220] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can be corresponding to touch operations applied to different applications (such as taking photos, playing audio, etc.). Furthermore, motor 191 can also correspond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects.

[0221] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0222] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with or separate from the electronic device 30.

[0223] This application also provides a computer program product for implementing the interactive methods provided in the above embodiments.

[0224] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer program modules or module code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0225] Computer program modules or module code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0226] Module code can be implemented using a high-level modular language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used to implement module code when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0227] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, optical discs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0228] In this specification, the reference to "an embodiment" or "an embodiment" means that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technology disclosed according to an embodiment of this application. The appearance of the phrase "in an embodiment" in various places in the specification does not necessarily refer to the same embodiment.

[0229] The disclosure of embodiments of this application also relates to means for performing operations in text. This means may be specifically constructed for the claimed purpose or may include a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored on a computer-readable medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, application-specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, and each may be coupled to a computer system bus. Furthermore, the computer mentioned in the specification may include a single processor or may employ an architecture involving multiple processors for increased computing power.

[0230] Furthermore, the language used in this specification has been primarily chosen for readability and instructional purposes and may not have been chosen to depict or limit the disclosed subject matter. Therefore, the embodiments disclosed herein are intended to illustrate, and not limit, the scope of the concepts discussed herein.

Claims

1. An interaction method applied to an electronic device, characterized in that, The method includes: Displaying a first interface, wherein the first interface includes a first window and a first display area, and the first display area includes a first identifier image of the first window; A user's first drag operation on the first identifier image was detected; In response to the first drag operation, move the first window.

2. The method according to claim 1, characterized in that, The electronic device includes a foldable device, wherein the foldable device includes a first screen portion and a second screen portion located on both sides of a folding axis.

3. The method according to claim 2, characterized in that, The first screen portion is located in a first direction of the second screen portion, and the second screen portion is located in a second direction of the first screen portion, wherein the first direction is opposite to the second direction.

4. The method according to claim 3, characterized in that, The first display area is located in the first screen portion.

5. The method according to claim 4, characterized in that, The detection of a user's first drag operation on the first identifier image includes: An operation was detected in which the user dragged the first icon image to move in the second direction.

6. The method according to claim 4, characterized in that, The second screen portion includes a second interface, the second interface includes a second window, and the first display area includes a second identifier image corresponding to the second window.

7. The method according to claim 6, characterized in that, The method includes: A second drag operation was detected, in which the user dragged the second icon image toward the first direction; Corresponding to the second drag operation, the second window is moved to the first interface displayed on the first screen portion.

8. The method according to claim 7, characterized in that, The detection of a user's first drag operation on the first identifier image includes: The detection indicates that the first distance the user drags the first icon image in the second direction meets a first threshold condition; and / or, The second drag operation, which detects that the user is dragging the second identifier image in the first direction, includes: The detection indicates that the second distance the user drags the second identifier image in the first direction satisfies the first threshold condition.

9. The method according to claim 4, characterized in that, The detection of a user's first drag operation on the first identifier image includes: A first combination of user actions on the first icon image is detected, wherein the first combination of actions includes a first action of dragging the first icon image in the second direction and hovering it, and... The system detects that the user has selected a first area within the second display area triggered by the first operation. The second display area includes at least one display area recommended for the first window, the at least one display area including the first area, and the at least one display area is located within the display area of ​​the second screen portion.

10. The method according to claim 9, characterized in that, Moving the first window in accordance with the first drag operation includes: Corresponding to the first combined operation, the first window is displayed in the corresponding area of ​​the first area in the second screen portion.

11. The method according to claim 6, characterized in that, The method includes: A second combined operation by the user on the second icon image is detected, wherein the second combined operation includes a second operation of dragging the second icon image in the first direction and hovering it, and... The system detects that a user selects a second area in a third display area triggered by the second operation, wherein the third display area includes at least one display area recommended for the second window, the at least one display area includes the second area, and the at least one display area is located within the display area of ​​the first screen portion; Corresponding to the second combined operation, the second window is displayed in the corresponding area of ​​the second region within the first screen portion.

12. The method according to claim 11, characterized in that, The third distance by which the first identifier is dragged in the second direction satisfies the second threshold condition; and / or, The fourth distance by which the second identifier is dragged and moved in the first direction satisfies the second threshold condition.

13. The method according to any one of claims 9 to 12, characterized in that, The duration corresponding to the hovering operation meets the third threshold condition.

14. The method according to claim 4, characterized in that, The detection of a user's first drag operation on the first identifier image includes: A third combination of user actions on the first icon image is detected, the third combination of actions including placing the mouse on and hovering the mouse over the first icon image, and... The system detects that a user's click on the third operation triggers an operation on the first control among a plurality of displayed controls, wherein the first control is used to indicate the portion of the screen where the first window is displayed.

15. The method according to claim 14, characterized in that, The first control indicates that the screen portion to be displayed is the second screen portion.

16. The method according to claim 15, characterized in that, Moving the first window in accordance with the first drag operation includes: Corresponding to the third combined operation, the first window is displayed in the second screen portion indicated by the first control.

17. The method according to claim 16, characterized in that, An icon indicating the second direction is displayed on the first control.

18. The method according to any one of claims 14 to 17, characterized in that, The duration corresponding to the hovering operation meets the fourth threshold condition.

19. The method according to claim 3, characterized in that, The first display area is located in the second screen portion.

20. The method according to claim 19, characterized in that, The detection of a user's first drag operation on the first identifier image includes: An operation was detected in which the user dragged the first icon image to move in the first direction.

21. The method according to claim 19, characterized in that, The detection of a user's first drag operation on the first identifier image includes: A fourth combined operation by the user on the first icon image is detected, wherein the fourth combined operation includes a fourth operation of dragging the first icon image in the first direction and hovering it, and... The system detects that the user selected a third area within the fourth display area triggered by the fourth operation. The fourth display area includes at least one display area recommended for the first window, the at least one display area including the third area, and the at least one display area is located within the display area of ​​the first screen portion.

22. The method according to claim 19, characterized in that, The detection of a user's first drag operation on the first identifier image includes: A fifth combination of user actions on the first icon image was detected, the fifth combination of actions including placing the mouse on and hovering the mouse over the first icon image, and... The system detects that a user's click on the fifth operation triggers the operation of a second control among a plurality of displayed controls, wherein the second control is used to indicate a screen portion that displays the first window, the screen portion including either the first screen portion or the second screen portion.

23. The method according to any one of claims 1 to 19, characterized in that, The first identification image includes: The thumbnail window corresponding to the first window; or, The application icon corresponding to the application interface displayed in the first window.

24. An electronic device, characterized in that, include: One or more processors; One or more memories; the one or more memories storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the interaction method of any one of claims 1 to 23.

25. A computer-readable medium, characterized in that, The readable medium stores instructions that, when executed on a computer, cause the computer to perform the interactive method of any one of claims 1 to 23.

26. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the interactive method of any one of claims 1 to 23.

Images