Collapsible device hinge interaction

Abstract

Systems and methods for triggering a first action based on user interaction with a folding area of a foldable touch screen device are disclosed. A foldable touch screen device includes a flexible touch screen element having a folding region separating a first partition and a second partition of the flexible touch screen element. The foldable touch screen device includes a processor and a memory. The foldable touch screen device may be configured to: display a first object in the first partition; detecting initiation of a drag gesture at a location of the first partition corresponding to the first object; detecting a path of the drag gesture approaching or passing through a first shortcut overlay of the one or more shortcut overlays displayed in the folding area; a first action corresponding to the first one of the one or more shortcut overlays is triggered.

Description

Technical Field

[0001] The present invention generally relates to foldable touchscreen devices, and more specifically, to systems and methods for providing functionality in the folded area of ​​a foldable capacitive touchscreen device. Background Technology

[0002] There are currently many foldable touchscreen devices on the market, including smartphones, tablets, and laptops. As these devices become more widely used, the demand for more functional space on these devices is also increasing.

[0003] Although most devices now feature continuous flexible capacitive touchscreens, the potential capabilities of this feature have not been fully explored. For example, touch gestures on current devices are typically limited to touch gestures on a single planar portion of the screen. Touch gestures utilizing folded areas (also known as hinges) are not yet widespread, and related research is scarce.

[0004] Foldable touchscreen devices are complex for multitasking, requiring multiple controls and gestures to switch between applications and windows. Many foldable touchscreen devices perform poorly in window management, making it difficult to switch between screens or display full-screen windows.

[0005] In particular, touchscreen interaction for folded areas (also known as hinges) has not yet been implemented in smartphone devices.

[0006] This area urgently needs improvement. Summary of the Invention

[0007] Foldable touchscreen devices, such as laptops, touchpads, e-readers, or foldable smartphones, can include flexible displays. A flexible display is a type of display that can be rolled up without distorting the image or text it shows. Foldable touchscreen devices can include a flexible display spanning the folding area, or "hinge," and touchscreen sections on adjacent sides of the folding area. Therefore, the folding area can receive touch input, allowing the touchscreen device to adapt to different usage scenarios. Manufacturers often prioritize other features, such as the size of the touchscreen sections and / or the overall design of the device, by minimizing the hinge radius. Although often overlooked by manufacturers, the hinge is an important element that can enhance the user experience.

[0008] One potential use for hinges is as a natural separator between touchscreen sections on foldable devices. In this case, hinges can be used for multitasking, as adjacent touchscreen sections (on either side of the hinge) can be used to display different applications or tasks side-by-side. Hinges can also act as a bridge between two touchscreen sections, enabling a seamless transition between different applications or tasks.

[0009] In addition to the practical use, the hinge has unique characteristics that can be used for accessibility purposes. For example, since the hinge is a physical feature that can be easily achieved through touch, it can be used to provide a more intuitive way for visually impaired users to interact with a foldable touch screen device.

[0010] Despite the potential advantages of the hinge, this area is often overlooked in the design and interaction of foldable touch screen devices. Due to the possibility of distortion, it is generally avoided to place user interface (UI) elements in or near the hinge area.

[0011] Embodiments of the present invention describe intuitive and novel interaction techniques for foldable touch screen devices. In some examples, these techniques provide a shortcut dynamic portal that can adapt to user input and system / application events, enabling the execution of commonly used tasks and commands without the need to open multiple menus step by step.

[0012] Embodiments of the present invention can be used for various purposes, including window management, shortcuts, and multitasking. In some examples, these functions can be more efficient and effective compared to current methods.

[0013] According to an aspect of the present invention, there is provided a foldable touch screen device comprising a flexible touch screen element having a first partition, a second partition, and a folding area separating the first partition and the second partition. The touch screen element is foldable at the folding area. The foldable touch screen device further comprises a processor and a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to perform the following operations: display a first object in the first partition; detect initiation of a drag gesture at a location of the first partition corresponding to the first object; detect a path of the drag gesture proximate to or across a first shortcut overlay layer of one or more shortcut overlay layers displayed in the folding area; trigger a first action corresponding to the first shortcut overlay layer of the one or more shortcut overlay layers.

[0014] In some implementations, the foldable touch screen device is further configured to perform the following operations before detecting the path of the drag gesture proximate to or across the first shortcut overlay layer of the one or more shortcut overlay layers: detect the drag gesture at a location of the first partition proximate to the folding area; display the one or more shortcut overlay layers in the folding area.

[0015] In some implementations, the one or more shortcut overlay layers are window management overlay layers.

[0016] In some implementations, the foldable touch screen device is further to: detect a stop of the drag gesture; in response to detecting the stop of the drag gesture, remove the display of the one or more shortcut overlays from the fold region.

[0017] In some implementations, the one or more shortcut overlays are parallel horizon overlays.

[0018] In some implementations, the foldable touch screen device is further to: detect the drag gesture across the fold region at a second one of the one or more shortcut overlays; trigger a second action corresponding to the second one of the one or more shortcut overlays.

[0019] According to another aspect of the present application, a foldable touch screen device is provided, comprising a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section. The touch screen element is foldable at the fold region. The foldable touch screen element further comprises a processor and a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to, during execution of an electronic reader application, detect a drag gesture in a first direction along the fold region, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; and insert a virtual bookmark at a current display location of an electronic document.

[0020] In some implementations, the drag gesture originates at or near a first edge of the fold region.

[0021] In some implementations, the foldable touch screen device is further to: detect a drag gesture in a second direction along the fold region substantially opposite the first direction; and remove the virtual bookmark.

[0022] According to yet another aspect of the present application, a foldable touch screen device is provided, comprising a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section. The touch screen element is foldable at the fold region. The foldable touch screen element further comprises a processor and a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to: display content using a full screen mode; detect a drag gesture in a first direction along the fold region, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; and display the content using a dual screen mode.

[0023] According to yet another aspect of the present application, there is provided a foldable touch screen device comprising a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section. The touch screen element is foldable at the fold region. The foldable touch screen element further comprises a processor and a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to perform the following operations: detect a first touch gesture at the fold region; display a defined region containing an application shortcut.

[0024] According to yet another aspect of the present application, there is provided a foldable touch screen device comprising a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section. The touch screen element is foldable at the fold region. The foldable touch screen element further comprises a processor and a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to perform the following operations during execution of a particular application: display one or more application control overlays defined by the particular application at the fold region; detect a touch gesture at a location of the fold region corresponding to one of the one or more application control overlays; trigger a first action corresponding to the one of the one or more application control overlays.

[0025] According to yet another aspect of the present application, there is provided a foldable touch screen device comprising a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section. The touch screen element is foldable at the fold region. The foldable touch screen element further comprises a processor and a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to perform the following operations: detect a grip posture of the foldable touch screen device; in response to detecting the grip posture, display a thumb input overlay at the fold region; detect a touch gesture at a location of the fold region corresponding to the thumb input overlay; trigger a first action corresponding to the touch gesture and the location corresponding to the thumb input overlay.

[0026] In some implementations, the foldable touch screen device is executing a video game application, and the first action is a video game control action.

[0027] In some implementations, the foldable touch screen device is executing a camera application, and the first action is a camera control action.

[0028] According to yet another aspect of the present application, there is provided a foldable touch screen device comprising a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section. The touch screen element is foldable at the fold region. The foldable touch screen element further comprises a processor and a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to perform the following operations: display one or more user-modifiable objects; receive a selection of one of the one or more user-modifiable objects; display, at the fold region, one or more shortcut overlay layers corresponding to actions that modify the one of the one or more user-modifiable objects; in response to a selection of one of the one or more shortcut overlay layers, modify the one of the one or more user-modifiable objects according to the action corresponding to the selected one of the one or more shortcut overlay layers.

[0029] According to yet another aspect of the present application, there is provided a computer-implemented method comprising: displaying a first object in a first section of a flexible touch screen element; detecting initiation of a drag gesture at a location of the first section corresponding to the first object; detecting a path of the drag gesture proximate to or across a first one of one or more shortcut overlay layers displayed in a fold region; triggering a first action corresponding to the first one of the one or more shortcut overlay layers.

[0030] In some implementations, the method further comprises: prior to detecting the path of the drag gesture proximate to or across the first one of the one or more shortcut overlay layers, performing the following operations: detecting the drag gesture at a location of the first section proximate to the fold region; displaying the one or more shortcut overlay layers at the fold region.

[0031] In some implementations, the one or more shortcut overlay layers are window management overlay layers.

[0032] In some implementations, the method further comprises: detecting cessation of the drag gesture; in response to detecting the cessation of the drag gesture, removing the display of the one or more shortcut overlay layers from the fold region.

[0033] In some implementations, the one or more shortcut overlay layers are parallel universe overlay layers.

[0034] In some implementations, the method further includes detecting the drag gesture across the fold region at a second one of the one or more shortcut overlay layers; triggering a second action corresponding to the second one of the one or more shortcut overlay layers.

[0035] According to yet another aspect of the present application, a computer-implemented method is provided, comprising: detecting a drag gesture along a first direction of a fold region of a flexible touch screen element, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; inserting a virtual bookmark at a current display location of an electronic document.

[0036] In some implementations, the drag gesture originates at or near a first edge of the fold region.

[0037] In some implementations, the method further includes detecting a drag gesture along a second direction of the fold region substantially opposite the first direction; removing the virtual bookmark.

[0038] According to yet another aspect of the present application, a computer-implemented method is provided, comprising: displaying content on a flexible touch screen element using a full screen mode; detecting a drag gesture along a first direction of a fold region of the flexible touch screen element, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; displaying the content using a dual screen mode.

[0039] According to yet another aspect of the present application, a computer-implemented method is provided, comprising: detecting a first touch gesture at a fold region of a flexible touch screen element; displaying a defined region containing application shortcuts.

[0040] According to yet another aspect of the present application, a computer-implemented method is provided, comprising: displaying one or more application control overlays defined by an executing application at a fold region of a flexible touch screen element; detecting a touch gesture at a location of the fold region corresponding to one of the one or more application control overlays; triggering a first action corresponding to the one of the one or more application control overlays.

[0041] According to yet another aspect of the present application, a computer-implemented method is provided, comprising: detecting a holding posture of a foldable touch screen device, wherein the foldable touch screen device has a flexible touch screen element; in response to detecting the holding posture, displaying a thumb input overlay at a fold region of the flexible touch screen element; detecting a touch gesture at a location of the fold region corresponding to the thumb input overlay; triggering a first action corresponding to the touch gesture and the location corresponding to the thumb input overlay.

[0042] In some implementations, the foldable touch screen device is executing a video game application, and the first action is a video game control action.

[0043] In some implementations, the foldable touch screen device is executing a camera application, and the first action is a camera control action.

[0044] According to yet another aspect of the application, there is provided a computer- implemented method comprising: displaying one or more user-modifiable objects on a flexible touch screen element; receiving a selection of one of the one or more user-modifiable objects; displaying one or more shortcut overlay layers corresponding to actions that modify the one of the one or more user-modifiable objects in a folded region of the flexible touch screen element; in response to a selection of one of the one or more shortcut overlay layers, modifying the one of the one or more user-modifiable objects according to the action corresponding to the selected one of the one or more shortcut overlay layers. BRIEF DESCRIPTION OF DRAWINGS

[0045] Reference will now be made, by way of example, to the accompanying drawings, which show example embodiments of the application, in which:

[0046] Figure 1 shows a front view of a first example foldable touch screen device according to an example of the application;

[0047] Figure 2 is a high level operational diagram of an example computing system according to an example of the application;

[0048] Figure 3 depicts a simplified architecture of software components according to an example of the application, which can be stored in the memory of the example computing system of Figure 2 ;

[0049] Figure 4 is a flow diagram of an example method for triggering a first action according to an example of the application;

[0050] Figure 5 shows three instances related to an example method for triggering a first action according to an example of the application;

[0051] Figure 6 is a flow diagram of an example method for triggering a second action according to an example of the application;

[0052] Figure 7 shows the first four instances related to an example method for triggering a second action according to an example of the application;

[0053] Figure 8 The last four instances related to the exemplary method for triggering a first action are shown according to an example of the application;

[0054] Figures 9A-9E The first five instances related to the exemplary method for triggering a first action are shown according to an example of the application;

[0055] The last five instances related to the exemplary method for triggering a first action are shown according to an example of the application;

[0056] Figure 11 is a flowchart of an exemplary method for inserting and removing a virtual bookmark at a current display position of an electronic document according to an example of the application;

[0057] Figure 12 A foldable touch screen device providing a first display of an electronic document is shown according to an example of the application;

[0058] Figure 13 A foldable touch screen device providing a second display of an electronic document is shown according to an example of the application;

[0059] Figure 14 is a flowchart of an exemplary method for displaying content using a dual screen mode according to an example of the application;

[0060] Figure 15 A foldable touch screen device displaying content using a full screen mode is shown according to an example of the application;

[0061] Figure 16 A foldable touch screen device displaying content using a dual screen mode is shown according to an example of the application;

[0062] Figure 17 is a flowchart of an exemplary method for displaying a defined area containing application shortcuts according to an example of the application;

[0063] Figure 18 is a flowchart of an exemplary method for triggering a first action corresponding to one or more shortcut overlays according to an example of the application;

[0064] Figure 19 A foldable touch screen device providing a first display of a video application is shown according to an example of the application;

[0065] Figure 20 A foldable touch screen device providing a second display of a video application is shown according to an example of the application;

[0066] Figure 21is a flowchart of an exemplary method for triggering a first action corresponding to a touch gesture and a location corresponding to a thumb input overlay according to examples of the present invention;

[0067] Figure 22 shows a foldable touch screen device displaying a video game application according to examples of the present invention;

[0068] Figure 23 shows a foldable touch screen device displaying a camera application according to examples of the present invention;

[0069] Figure 24 is a flowchart of an exemplary method for modifying one of one or more user modifiable objects according to examples of the present invention;

[0070] Figure 25 shows a foldable touch screen device providing a first display of a word processing application according to examples of the present invention;

[0071] Figure 26 shows a foldable touch screen device providing a second display of a word processing application according to examples of the present invention;

[0072] Figure 27 shows a foldable touch screen device providing a first display of a design application according to examples of the present invention;

[0073] Figure 28 shows a foldable touch screen device providing a second display of a design application according to examples of the present invention;

[0074] Similar reference characters can be used to represent similar elements in different figures. DETAILED DESCRIPTION

[0075] Embodiments described herein can operate on various foldable touch screen devices, such as dual screen notebooks, foldable notebooks, standard notebooks, tablets, smartphones, etc.

[0076] In the present invention, the term "computing system" refers to an electronic device having computing capability. Examples of computing systems include, but are not limited to, personal computers, laptop computers, tablet computers ("tablets"), smartphones, surface computers, augmented reality devices, etc.

[0077] In the present invention, the terms "touch screen element" and "touch screen" refer to a combination of a display and a touch sensing system capable of functioning as an input device by receiving touch input. Non-limiting examples of touch screen displays are surface capacitive touch screens and projected capacitive touch screens.

[0078] In the present disclosure, the term "touchscreen device" refers to a computer system having a touchscreen element.

[0079] In the present disclosure, the term "application" refers to a software program comprising a set of instructions executable by a processing device of an electronic device.

[0080] Figure 1 An example foldable touchscreen device 100 is shown, which is an example operating environment for example embodiments. As shown, the example foldable touchscreen device 100 includes a flexible touchscreen element 140. The flexible touchscreen element 140 can be a foldable touchscreen element that is foldable at a fold region 135. The touchscreen element 140 can be used to present content and sense touches thereon. As noted above, the touchscreen element 140 can also be described as a touchscreen 140. The touchscreen element 140 can implement one or more touchscreen technologies. For example, the touchscreen element 140 can be a capacitive touchscreen (e.g., surface capacitive, projected capacitive, mutual capacitive, self-capacitive), a resistive touchscreen, a surface acoustic wave (SAW) touchscreen, etc.

[0081] The example foldable touchscreen device 100 includes a body 105, which can be made of plastic, glass, fiber, stainless steel, aluminum, other suitable materials, or any combination of these materials. The body 105 encloses a plurality of components of the example foldable touchscreen device 100, including a processor and a memory. The body 105 is used to house the flexible touchscreen element 140. The body can be described as including a first portion 110 and a second portion 115 that are connected by a fold edge 120.

[0082] The flexible touchscreen element 140 is used to fold the example foldable touchscreen device 100 into various physical configurations, including a flat configuration. The flexible touchscreen element 140 can be folded about the fold edge 120 such that the first portion 110 and the second portion 115 of the example foldable touchscreen device 100 are moved toward and / or away from each other. The fold edge 120 can be made of a flexible material that is bendable, such as a flexible polymer.

[0083] The flexible touch screen element 140 of the example foldable touch screen device 100 has three touch screen partitions: a first partition 125, a second partition 130, and a fold region 135 separating the first partition 125 and the second partition 130. The fold region 135 can also be referred to as a hinge. As shown, the fold region 135 can be defined by a first border 150 along an edge of the first partition 125 and a second border 155 along an edge of the second partition 130. The first border 150 and the second border 155 can not be physical borders, but can be defined logically and can be dynamically shifted (or ignored). It should be understood that the terms "first" and "second" are not intended to be limiting. Furthermore, the "first" or "second" touch screen partition can be arbitrarily designated; a touch screen partition designated as "first" in one instance can be designated as "second" in another instance, and vice versa. Similarly, the "first" and "second" borders can be arbitrarily designated.

[0084] It can be appreciated that when the example foldable touch screen device 100 is in an unfolded state (i.e., the example foldable touch screen device 100 is a flat device with a fold angle 145 of 180 degrees), the first partition 125, the second partition 130, and the fold region 135 are indistinguishable, forming a continuous display.

[0085] Although Figure 1 A laptop computer is shown, but the example foldable touch screen device 100 can be a smartphone, a tablet, and / or other similar electronic device. The example foldable touch screen device 100 can be a type of computing system within the scope of the present disclosure.

[0086] Figure 2 is a high-level operational diagram of an example computing system 200 in accordance with an example of the present disclosure. In at least some embodiments, the example computing system 200 can be an example of the example touch screen device 100 Figure 1 ) and is not intended to be limiting.

[0087] The example computing system 200 includes various components. For example, as shown, the example computing system 200 can include a processor 302, an input / output (I / O) interface 304, a network interface 306, a storage unit 378, and a memory 380. As shown, the above-described example components of the computing system 200 communicate through a bus 308. The bus 308 provides communication between the components of the computing system 200. The bus 308 can be any suitable bus architecture, including, for example, a memory bus, a peripheral bus, or a video bus.

[0088] The processor 302 can include one or more processors, such as a microprocessor, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a dedicated logic circuit, or a combination thereof.

[0089] The network interface 306 can include one or more network interfaces that enable wired or wireless communication with a network (e.g., an intranet, the Internet, a peer-to-peer (P2P) network, a wide area network (WAN), and / or a local area network (LAN)) or other nodes. The network interface 306 can include wired links (e.g., an Ethernet cable) and / or wireless links (e.g., one or more antennas) for communicating within and / or between networks.

[0090] The storage unit 378 can be one or more storage units and can include mass storage units, such as solid state drives, hard disk drives, disk drives, and / or optical drives.

[0091] The I / O interface 304 can be one or more I / O interfaces and can enable connection with one or more suitable input devices (e.g., the touch panel 344) and / or one or more suitable output devices (e.g., the touch screen display 342). The touch panel 344 and the touch screen display 342 form part of the touch screen element 140.

[0092] The touch panel 344 can include various touch sensors for sensing touch input, which can depend on the touch sensing mode (e.g., a capacitive sensor) used by the touch screen element 140.

[0093] The example computing system 200 can include one or more memories 380, which can include volatile (e.g., random access memory (RAM)) and non-volatile or non-transitory memory (e.g., flash memory, magnetic storage, and / or read-only memory (ROM)). One or more non-transitory memories of the memory 380 store programs including software instructions for execution by the processor 302 to perform examples described in this disclosure and the like. In example embodiments, these programs include software instructions for implementing an operating system (OS) and software applications.

[0094] In some examples, the memory 380 can include software instructions for the computing system 200 for execution by the processor 302 to perform the operations described in this disclosure. In some other examples, one or more data sets and / or modules can be provided by an external memory (e.g., an external drive in wired or wireless communication with the computing system 200) or by a transitory or non-transitory computer readable medium. Examples of non-transitory computer readable media include RAM, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, a CD-ROM, or other portable memory.

[0095] Figure 3 A simplified architecture of software components that can be stored in the memory 380 of the example computing system 200 ( Figure 2 ) is depicted. As shown, the software components include application software 350 and an operating system (OS) 310.

[0096] The OS 310 is software. The OS 310 allows the application software 350 to access the processor 302, the memory 380, the I / O interface 304, the network interface 306, and the storage unit 378 ( Figure 2 ). The OS 310 can be Apple™ iOS™, Android™, Microsoft™ Windows™, or Google™ Chrome OS™, among others.

[0097] The application software 350 adapts the example computing system 200 ( Figure 2 ) and, in combination with the OS 310, operates as a device that performs a particular function. For example, the application software 350 can adapt the example computing system 200 ( Figure 2 ) to perform determination of a folding angle.

[0098] Figure 4 is a flowchart of a method 400 for triggering a first action according to an example of the disclosure. The method 400 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 ) operating as the foldable touch screen device 100 ( Figure 1 ). In particular, the operation 410 and subsequent operations can be performed by one or more processors (e.g., the processor 302 of Figure 2 ) of the computing system.

[0099] Operation 410: The computing system displays a first object in a first partition of the flexible touch screen element.

[0100] Referring again to Figure 4 , following operation 410, operation 420 follows.

[0101] Operation 420: The system detects initiation of a drag gesture at a location of the first partition corresponding to the first object.

[0102] Operation 430: The system detects a path of the drag gesture proximate to or across a fold region at a first one of the one or more shortcut overlay layers.

[0103] Operation 440: The system triggers a first action corresponding to the first one of the one or more shortcut overlay layers.

[0104] Referring now to Figure 5 , three instances 502, 504, 506 of a method for triggering a first action according to examples of the present invention are shown. For example, the three instances 502, 504, 506 can illustrate Figure 4 sequences of operations of the foldable touch screen device 100 in example implementations of the method 400 of FIG. 4. Each of the three instances 502, 504, and 506 illustrates a foldable touch screen device 100 having a flexible touch screen element 140 that includes a first partition 125, a second partition 130, and a fold region 135. Four shortcut overlay layers 508 are displayed at the fold region 135 of each of the foldable touch screen devices 100 illustrated in instances 502, 504, and 506. It will be appreciated that Figure 5 the first and second partitions 125, 130 shown can correspond to Figure 1 the second and first partitions 130, 125 shown; as previously mentioned, "first" and "second" can be arbitrarily assigned and are not intended to be limiting.

[0105] As illustrated in instance 502, a first circle 510 is displayed in the first partition 125 of the foldable touch screen device 100. The first circle 510 is a first object.

[0106] As illustrated in instance 504, the first circle 510 has been selected and dragged along the first partition 125 to proximate the fold region 135. In particular, the first circle 510 is proximate to a first one of the four shortcut overlay layers 508 at the fold region 135. The four shortcut overlay layers 508 can be described as dynamic hinge shortcut overlay layers. The first one of the dynamic hinge shortcut overlay layers displays the word "copy" indicating a copy function shortcut.

[0107] The fold region 135 (or "hinge") can serve as a natural bridging area between the first and second partitions 125, 130 of the flexible touch screen element 140 of the foldable touch screen device 100. Providing contextual shortcut overlays such as a dynamic hinge shortcut overlay as a portal at the hinge can enable the user to transition an object (e.g., the first circle 510) between the first and second partitions 125, 130.

[0108] As the first circle 510 is dragged toward the fold region 135, the foldable touch screen device 100 can detect initiation of the drag gesture at a location corresponding to the first partition 125 of the first circle 510.

[0109] As shown in instance 506, the first circle 510 has been dragged across the fold region 135 at the first shortcut overlay and is now displayed at the second partition 130. The first shortcut overlay displays the word "copy" indicating a copy function shortcut.

[0110] As the first circle 510 is dragged, the system detects the path of the drag gesture proximate to or across the first of the one or more shortcut overlays. Accordingly, the system triggers the first action corresponding to the first of the one or more shortcut overlays (in this example, the "copy" function corresponding to the first shortcut overlay).

[0111] The second circle 512 now appears at the first partition 125. The second circle 512 is a second object. The second circle 512 is a copy of the first circle 510.

[0112] Thus, by dragging an object toward the fold region 135 (hinge) to access a dynamic hinge shortcut overlay, a "copy" function can be accomplished in a single step. It should be appreciated that although this example shows a path of a drag gesture across the fold region 135 from the first partition 125 to the second partition 130, this is not intended to be limiting. For example, a path of a drag gesture beginning in the first partition 125, proximate to or touching the desired shortcut overlay of the fold region 135, and returning to the first partition 125 (i.e., not crossing the second partition 130) can suffice. The "copy" function can be accomplished using Microsoft™ Office applications or whiteboard applications, among others. This improves current methods that require multiple steps, such as selecting a component and selecting the appropriate command from a pop-up menu. When the user is working in an immersive or dual screen view that simultaneously utilizes the first and second partitions 125, 130 and needs to frequently copy and paste content between the first and second partitions 125, 130, Figure 5Examples can be particularly useful. Depending on which shortcut overlay layer the path of the drag gesture has passed through or is proximate to, an action corresponding to a different shortcut overlay layer of the one or more shortcut overlay layers can be performed. Other actions can include various actions that can be performed on a selectable object, such as a resize action, a delete action, etc., and can depend on the software application being executed.

[0113] Referring again to Figure 4 In some embodiments, one or more shortcut overlay layers can be displayed in the fold region prior to detecting initiation of the drag gesture at operation 420. In some embodiments, the one or more shortcut overlay layers can be displayed when the system detects initiation of the drag gesture, or when the system detects the drag gesture at a location of the first partition proximate to the fold region.

[0114] After triggering the first action at operation 440, the system can detect cessation of the drag gesture. In some embodiments, in response to detecting cessation of the drag gesture, the system can remove display of the one or more shortcut overlay layers from the fold region.

[0115] Figure 6 is a flowchart of an example method 600 for triggering a second action according to examples of the disclosure. The method 600 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 ) running as a foldable touch screen device 100 (e.g., the foldable touch screen device 100 of Figure 1 ). Specifically, the operations 410 and onward can be performed by one or more processors of the computing system (e.g., the processor 302 of Figure 2 ).

[0116] As shown, Figure 6 The method 600 of includes six operations 410, 420, 430, 440, 610, 620. Figure 6 The first four operations 410, 420, 430, 440 of the method 600 are the same as the operations 410, 420, 430, 440 of the method 400 of Figure 4 .

[0117] Operation 410: The computing system displays a first object in a first partition of a flexible touch screen element.

[0118] Operation 420: The system detects initiation of a drag gesture at a location of the first partition corresponding to the first object.

[0119] Operation 430: The system detects a path of the drag gesture proximate to or through a first shortcut overlay layer of the one or more shortcut overlay layers.

[0120] Operation 440: The system triggers the first action corresponding to a first one of the one or more shortcut overlay layers.

[0121] Operation 610: The system detects a drag gesture across a fold region at a second one of the one or more shortcut overlay layers.

[0122] Operation 620: The system triggers the first action corresponding to a first one of the one or more shortcut overlay layers.

[0123] Reference is now made to Figure 7 which shows the first four instances 502, 504, 506, 702 related to the method for triggering a second action, according to examples of the present application. For example, the four instances 502, 504, 506, 702 can show Figure 6 the sequence of operations of the foldable touch screen device 100 in an example implementation of the method 600. In this example, Figure 7 the first three instances 502, 504, 506 are the same as the three instances 502, 504, 506 of Figure 5 It is understood that Figure 7 the first and second partitions 125, 130 shown can correspond to Figure 1 the second and first partitions 130, 125 shown; as previously mentioned, "first" and "second" can be arbitrarily assigned and are not intended to be limiting.

[0124] As shown in instance 502, a first circle 510 is displayed at the first partition 125 of the foldable touch screen device 100. The first circle 510 is a first object.

[0125] As shown in instance 504, the first circle 510 has been selected and dragged along the first partition 125 to approach the fold region 135. Specifically, the first circle 510 is approaching a first one of four shortcut overlay layers 508 at the fold region 135. The four shortcut overlay layers 508 can be described as dynamic hinge shortcut overlay layers. The first one of the dynamic hinge shortcut overlay layers displays the word "copy" indicating a copy function shortcut.

[0126] The fold region 135 (or "hinge") can serve as a natural bridging area between the first and second partitions 125, 130 of the flexible touch screen element 140 of the foldable touch screen device 100. Providing contextual shortcut overlay layers such as the dynamic hinge shortcut overlay layers as a portal at the hinge can enable a user to transition an object (e.g., the first circle 510) between the first and second partitions 125, 130.

[0127] When the first circle 510 is dragged towards the fold region 135, the foldable touch screen device 100 can detect initiation of the drag gesture at the location of the first partition 125 corresponding to the first circle 510.

[0128] As shown in instance 506, the first circle 510 that has been dragged crosses the fold region 135 at the first shortcut overlay and is now displayed in the second partition 130. The first shortcut overlay displays the word "copy" representing the copy function shortcut.

[0129] As the first circle 510 is dragged, the system detects the path of the drag gesture in proximity to or crossing the first of the one or more shortcut overlays. Accordingly, the system triggers the first action corresponding to the first of the one or more shortcut overlays (in this example, the "copy" function corresponding to the first shortcut overlay)

[0130] The second circle 512 now appears at the first partition 125. The second circle 512 is a second object. The second circle 512 is a copy of the first circle 510.

[0131] Further, as shown in instance 702, the first circle 510 has crossed the fold region 135 from the second partition 130 back to the first partition 125 and is now displayed in the first partition 125. The second circle 512 continues to be displayed in the second partition 125. The first circle 510 has crossed the fold region 135 at the second of the four shortcut overlays 508. The second shortcut overlay corresponds to the change color action. As a result of being in proximity to or crossing the fold region 135 at the second of the four shortcut overlays 508, the second action has been triggered. In this example, the color of the first circle 510 has changed.

[0132] It should also be appreciated that the path of the drag gesture can cross or be in proximity to any number (e.g., more than two) of shortcut overlays displayed in the fold region 135. For example, Figure 6 The operations 610 and 620 of the method 600 can be repeated multiple times. As such, the dragging of the object (e.g., the first circle 510 and / or the second circle 512) across or in proximity to the fold region 135 can execute various commands. By dragging the object across or in proximity to the fold region 135 multiple times (corresponding to multiple shortcut overlays) in the same drag gesture, various commands can be used in combination. Repeated hinge spanning gestures can execute a set of commands to trigger an action. Examples of commands that can be executed in this manner can include "save" and "full screen."

[0133] The repeat hinge spanning gesture can be beneficial to a user working in a Microsoft Office application, a design application, a whiteboard application, or other such application on a foldable touch screen device. For example, when creating an object with repeating elements (e.g., a chart), a user can want to quickly duplicate content and assign new properties, such as color and position. For another example, a user can drag a desired object across or near the fold area and perform a specific gesture to duplicate the object and assign new properties in a single step, thus simplifying the creation process of an object with repeating elements (e.g., a chart) and improving the efficiency and convenience of the task.

[0134] Figure 8 The last four instances 802, 804, 806, 808 related to methods for triggering a first action are shown in accordance with examples of the present invention. For example, the four instances 802, 804, 806, 808 can show different operations of the foldable touch screen device 100 in some example implementations of the method 400 of Figure 4 or the method 600 of Figure 6 The four instances 802, 804, 806, 808 each show a foldable touch screen device 100 having a flexible touch screen element 140 that includes a first partition 125, a second partition 130, and a fold area 135. In this example, the triggered actions can be related to window management. A window management overlay 810 is displayed in the fold area 135 of each foldable touch screen device 100 shown in the four instances 802, 804, 806, 808. The window management overlay 810 includes four shortcut overlays.

[0135] Figure 8 The four instances 802, 804, 806, 808 of the method 400 involve using the fold area 135 as a portal for system window management assistance on the foldable touch screen device 100.

[0136] As shown in instance 802, a first window 812 (i.e., a first object) is displayed in the first partition 125 of the flexible touch screen element 140 of the foldable touch screen device 100.

[0137] As shown in instance 804, the first window 812 has been dragged along the first partition 125 and through a first one of the window management shortcut overlays in the window management overlay 810 in the fold area 135. The window management overlay 810 can be described as a dynamic hinge shortcut overlay. Each of the four shortcut overlays of the window management overlay 810 can correspond to a different window layout action. The first one of the dynamic hinge shortcut overlays represents an "expand" function shortcut. The first window 812 expands to occupy the entire second partition 130.

[0138] Thus, by dragging an object to the fold area 135 (hinge) to access the window management shortcut overlay, the expand action can be accomplished in a single step.

[0139] The third example 806 illustrates the first window 812 after it has been dragged along the first partition 125 and past the second shortcut overlay in the window management overlay 810 in the fold area 135. The second shortcut overlay in the dynamic hinge shortcut overlay represents a “half-expand” action shortcut. The first window 812 is half-expanded to occupy the left half of the second partition 130.

[0140] Thus, by dragging an object (e.g., the first window 812) to the fold area 135 (hinge) to access the window management overlay 810, the half-expand action can be accomplished in a single step.

[0141] The fourth example 808 illustrates the first window 812 after it has been dragged along the first partition 125 and past the third shortcut overlay in the window management overlay 810 in the fold area 135. The third shortcut overlay in the dynamic hinge shortcut overlay represents a “close” action shortcut. The first window 812 has been closed.

[0142] Thus, by dragging an object to the fold area (hinge) to access the respective window management shortcut overlay, the “close” action can be accomplished in a single step.

[0143] As illustrated, the fold area 135 can serve as a portal for system window management assistance on the foldable touchscreen device 100. By dragging a window to the hinge, a user can access a window management overlay. The window management overlay can enable the user to easily rearrange the window layout on the device by dragging one or more windows over the various layout commands of the window management overlay.

[0144] The hinge portal provides a natural transition for managing windows across the screen, addressing a common problem with foldable personal computers (PCs) that can have difficulty moving windows across the hinge. In combination with the Figure 8 The described interactions can simplify the process of organizing windows and improve the overall efficiency of window management on foldable touchscreen devices. Further, it should be appreciated that although Figure 8 While examples are illustrated of triggering a first action by dragging a window, in some examples, the drag gesture can also pass through or near one or more additional shortcut overlays in the fold area to trigger one or more additional actions to be performed.

[0145] Figures 9A-9EThe first five instances 902, 904, 906, 908, 910 related to an example method for triggering a first action are shown in accordance with an example of the present invention. For example, the five instances 902, 904, 906, 908, 910 can show Figure 4 the operation of the foldable touch screen device 100 in some example implementations of the method 400 or Figure 6 the method 600. Each of the five instances 902, 904, 906, 908, 910 shows a foldable touch screen device 100 having a flexible touch screen element 140 that includes a first partition 125, a second partition 130, and a fold region 135. The foldable touch screen device 100 is in a "book mode" orientation in which the fold region 135 is oriented vertically rather than horizontally. The flexible touch screen element 140 displays online content through a web browser application that is executing on the foldable touch screen device 100. In this example, the triggered action can relate to browsing an online website.

[0146] Figure 9A Instance 902 is shown in which a list view of objects at the first partition 125 and the second partition 130 is displayed. Each of the objects is represented by a displayed image and can represent a hyperlink.

[0147] Figure 9B Instance 904 is shown in which a list view of objects at the first partition 125 and the second partition 130 is displayed. Instance 904 additionally shows a finger of a user's hand 912 initiating a drag operation that starts with a first one of the objects at the first partition 125. Three shortcut overlays 914 are displayed at the fold region 135. The three shortcut overlays 914 can correspond to shortcuts that correspond to an "app launcher" feature and / or a "split view" feature. The "app launcher" feature and / or the "split view" feature can enable a user to browse the list view of objects at the second partition 130 while opening a detailed view of one or more objects at the first partition 125 and other functionality. The three shortcut overlays 914 can correspond to actions for opening a detailed view of one or more selected objects. The three shortcut overlays 914 can be displayed upon detecting the drag gesture at a location of the first partition 125 that is proximate to the fold region 135. In other examples, the shortcut overlays 914 can be displayed prior to detecting the drag gesture, or can be displayed after the drag gesture has been initiated but prior to the drag gesture being proximate to the fold region 135. The path of the drag operation extends from the first one of the objects at the first partition 125 to the first one of the three shortcut overlays 914 at the fold region 135.

[0148] Figure 9CAn example 906 is shown in which the foldable touch screen device 100 is shown after the action corresponding to the first shortcut overlay is performed. The second zone 130 displays a list view of objects. Each of the objects is represented by an image that is displayed and can represent a hyperlink. The first zone 125 displays a detailed view corresponding to a first one of the objects.

[0149] Figure 9D An example 908 is shown in which the list view of objects at the second zone 130 is displayed and a detailed view of three objects at the first zone 125 is displayed. The example 908 additionally shows a finger of the user's hand 912 initiating a drag operation that starts with a first one of the objects at the first zone 125. The fold area 135 displays four shortcut overlays 916. The four shortcut overlays 916 can correspond to the navigation actions of "unlink," "close," "flip," and / or "back," among others. The four shortcut overlays 916 can be displayed when the drag gesture is detected at a location of the first zone 125 that is proximate to the fold area 135. In other examples, the shortcut overlays 916 can be displayed before the drag gesture is detected, or can be displayed after the drag gesture has been initiated but before the drag gesture is proximate to the fold area 135. The path of the drag operation extends from the first one of the objects at the first zone 125 to a fourth one of the four shortcut overlays 916 at the fold area 135.

[0150] The fourth one of the four shortcut overlays 916 corresponds to the "close" action.

[0151] Figure 9E An example 910 is shown in which the foldable touch screen device 100 is shown after the "close" action is performed. The first zone 125 displays a detailed view of two objects, namely the detailed view of the first one of the objects that was removed as a result of the "close" action. The second zone 130 displays a list view of objects. The list view of objects includes the first one of the objects. For example, the four shortcut overlays 916 have been removed from the fold area 135 in response to detecting the stop of the drag gesture. In other examples, the shortcut overlays 916 can be displayed at the fold area 135 even if the drag gesture is not detected.

[0152] As Figures 9A-9E shown, the hinge can be used as a portal to optimize the "parallel view" feature and / or the "app launcher" feature of a browser application. According to examples of the present application, the "parallel view" feature and / or the "app launcher" feature can be extended to a foldable touch screen device to allow a user to easily manage multiple sub-pages at the same time.

[0153] Figures 10A-10EFive subsequent examples 1002, 1004, 1006, 1008, and 1010 related to a method for triggering a first action, according to an example of the invention, are shown. For example, these five examples 1002, 1004, 1006, 1008, and 1010 can be illustrated... Figure 4 Method 400 or Figure 6 The exemplary implementation of method 600 includes an operational sequence of a foldable touchscreen device 100. Each of the five examples 1002, 1004, 1006, 1008, and 1010 illustrates a foldable touchscreen device 100 having a flexible touchscreen element 140, which includes a first partition 125, a second partition 130, and a folding region 135. In this example, the triggered action may be related to layer management in applications such as design applications.

[0154] like Figure 10A As shown in Example 1002, the first sequence of layers 1012 is displayed in the first partition 125 of the flexible touchscreen element 140 of the foldable touchscreen device 100. The user's hand 912 is touching the first layer in the sequence of layers 1012. The first layer is the first layer in the sequence of layers 1012. The first layer is an object. As shown, the user's hand 912 is initiating a drag gesture toward the first layer in the folding area 135.

[0155] like Figure 10B As shown in the second example 1004, when a drag gesture is detected at a location near the folded area 135 in the first partition 125, one or more shortcut overlays 1014 are displayed in the folded area 135. The user's hand 912 is touching a dragged copy of the first layer at the location of one of the one or more shortcut overlays 1014. One of the one or more shortcut overlays 1014 corresponds to a "layer" action shortcut.

[0156] like Figure 10C As shown in the third example 1006, the user's hand 912 continues to perform a drag gesture along the folded area 135 at the "Layer" action shortcut to select and adjust the order of layers 1012.

[0157] like Figure 10D As shown in example 1008, the first layer is displayed in the folded area 135 at the position of the fingers of the user's hand 912 performing the drag gesture. The first layer is being dragged to the remaining layers in the first sequence of layers 1012.

[0158] like Figure 10EThe first layer in the first order of the layers 1012 being dragged to the last position of the remaining layers in the first order of the layers 1012 is shown by the illustrated example 1010. The user's finger of the hand 912 is positioned at the location of the first layer.

[0159] As Figures 10A-10E examples of the application can provide the ability to adjust the position / order of layers and other objects in a design application (e.g., a Powerpoint™ application). Examples of the application can avoid the need to use a combination of keyboard shortcuts, right-click menus, and / or ribbon menus to perform such actions, which can be time consuming and inconvenient. Examples of the application can enable a user to simply drag an object to the layer command of the fold area, then swipe to adjust the layer order, and then reposition the object.

[0160] Figure 11 is a flowchart of an example method 1100 for inserting and removing a virtual bookmark at a current display position of an electronic document, in accordance with examples of the application. The method 1100 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 ) running as a foldable touch screen device 100 (e.g., the foldable touch screen device 100 of Figure 1 ). In particular, the operation 1110 and subsequent operations can be performed by one or more processors of the computing system (e.g., the processor 302 of Figure 2 ).

[0161] Operation 1110: The system detects a drag gesture along a first direction of the fold area. The first direction can be substantially parallel to a longitudinal axis of the fold area.

[0162] Operation 1120: The system inserts a virtual bookmark at a current display position of the electronic document.

[0163] Operation 1130: The system removes a virtual bookmark from a current display position of the electronic document.

[0164] Reference is made to Figure 12 wherein a foldable touch screen device 100 providing a first display of an electronic document 1204 is shown, in accordance with examples of the application. Figure 12 may be shown Figure 11an exemplary partial implementation of the method 1100. The foldable touchscreen device has a flexible touchscreen element 140 with a first section 125, a second section 130, and a fold region 135. The foldable touchscreen device 100 is in a "book mode" orientation. A longitudinal axis 1202 of the fold region 135 extends through the illustration of the foldable touchscreen device 100. The flexible touchscreen element 140 is displaying an electronic document 1204 (e.g., an electronic book). A first page of the electronic document 1204 is displayed in the first section 125, and a second page of the electronic document 1204 is displayed in the second section 130. A finger of a user's hand 912 is initiating a drag gesture at the fold region 135. The finger of the user's hand 912 is touching the fold region 135 near the top of the flexible touchscreen element 140, and a downward hollow arrow 1206 is displayed to indicate a downward direction of the drag gesture.

[0165] Thus, the drag gesture is being initiated in a first direction along the fold region 135. As shown, the first direction is substantially parallel to the longitudinal axis 1202 of the fold region 135. As a result of detecting the drag gesture substantially parallel to the longitudinal axis 1202 of the fold region 135, a virtual bookmark can be inserted at a current display location of the electronic document 1204 during display of the electronic document 1204.

[0166] Inserting a bookmark in an electronic document using a drag gesture is intuitive and natural to a user, and can provide a seamless experience.

[0167] In a similar manner, a bookmark can be removed from an electronic document. The system can remove the virtual bookmark in response to detecting a drag gesture in a second direction along the fold region substantially opposite the first direction.

[0168] Reference is now made to the following drawings in which: Figure 13 wherein a foldable touchscreen device 100 providing a second display of an electronic document 1204 is shown in accordance with an example of the invention. Figure 13 may be shown Figure 11 an exemplary partial implementation of the method 1100. The foldable touchscreen device has a flexible touchscreen element 140 with a first section 125, a second section 130, and a fold region 135. The foldable touchscreen device 100 is in a "book mode" orientation. A longitudinal axis 1202 of the fold region 135 extends through the illustration of the foldable touchscreen device 100. The flexible touchscreen element 140 is displaying an electronic document 1204. A first page of the electronic document 1204 is displayed in the first section 125, and a second page of the electronic document 1204 is displayed in the second section 130. A virtual bookmark 1302 is displayed along an upper portion of the fold region 135. For example, as a result of performing Figure 12The illustrated operation, a virtual bookmark 1302 can have been inserted. A finger of the user's hand 912 is initiating a drag gesture at the fold region 135. The finger of the user's hand 912 is touching the fold region 135 near the bottom of the virtual bookmark 1302, and an upward hollow arrow 1304 is displayed to indicate the upward direction of the drag gesture.

[0169] Thus, the drag gesture is being initiated in a second direction of the fold region 135 that is substantially opposite the first direction. As illustrated, the second direction is substantially parallel to the longitudinal axis 1202 of the fold region 135.

[0170] After the drag gesture is performed, the system can remove the virtual bookmark 1302.

[0171] Figure 14 is a flow diagram of an example method 1400 of displaying content using a dual-screen mode in accordance with examples of the present invention. The method 1400 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 running as a foldable touchscreen device 100 (e.g., the foldable touchscreen device 100 of Figure 1 ). In particular, the operations 1410 and thereafter can be performed by one or more processors of the computing system (e.g., the processors 302 of Figure 2 ).

[0172] Operation 1410: The system displays content using a full-screen mode.

[0173] Operation 1420: The system detects a drag gesture along a first direction of the fold region.

[0174] Operation 1430: The system displays content using a dual-screen mode.

[0175] Reference is now made to Figure 15 and Figure 16 wherein example implementations of the method 1400 of Figure 14 are shown. Figure 15 A foldable touchscreen device 100 displaying content using an immersive full-screen mode in accordance with examples of the present invention is shown. The foldable touchscreen device 100 has a flexible touchscreen element 140 having a first partition 125, a second partition 130, and a fold region 135. The flexible touchscreen element 140 is displaying content in a full-screen mode. Two fingers of the user's hand 912 are initiating a drag gesture at the fold region 135. An arrowed hollow line 1502 is displayed at the fold region 135 indicating the direction of the drag gesture.

[0176] Figure 16 It is shown that, as a result of detecting the drag gesture performed in Figure 15 , content is being displayed on the flexible touchscreen element 140 in a dual-screen mode.

[0177] Figure 17 is a flowchart of an example method 1700 for displaying a defined area containing application shortcuts, according to an example of the disclosure. The method 1700 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 running as a foldable touch screen device 100 (e.g., the foldable touch screen device 100 of Figure 1 ). Specifically, the operations 1710 and thereafter can be performed by one or more processors of the computing system (e.g., the processors 302 of Figure 2 ).

[0178] Operation 1710: The system detects a first touch gesture at the fold area. The first touch gesture can be a multi-finger gesture, such as a two-finger tap.

[0179] Operation 1720: The system displays a defined area containing application shortcuts. For example, in response to detecting the first touch gesture at the fold area, a defined area containing application shortcuts (e.g., an “app dock”) can be triggered to appear at the fold area, thereby allowing a user to easily open an application on either screen.

[0180] Thus, the touch gesture can provide a convenient way for a user to access and launch an application without having to open multiple menus or screens in a hierarchical manner. This example can be particularly useful in a foldable touch screen device (e.g., a foldable laptop or tablet) that provides multi-tasking functionality. By providing a simple and intuitive gesture to activate an app dock, a user can easily and quickly access a desired application, thereby improving productivity and efficiency.

[0181] Figure 18 is a flowchart of an example method 1800 for triggering a first action corresponding to one or more application shortcuts. The method 1800 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 running as a foldable touch screen device 100 (e.g., the foldable touch screen device 100 of Figure 1 ). Specifically, the operations 1710 and thereafter can be performed by one or more processors of the computing system (e.g., the processors 302 of Figure 2 ).

[0182] Operation 1810: The system displays one or more application control overlays at the fold area. The one or more application control overlays can control currently executing applications.

[0183] Operation 1820: The system detects a touch gesture at the fold area at a location corresponding to the one or more application control overlays. The touch gesture can be a multi-finger gesture.

[0184] Operation 1830: The system triggers a first action corresponding to one or more application control overlays.

[0185] Referring now to the drawings, in which is shown an example of a foldable touch screen device providing a first display of a video application in accordance with the present application. Figure 19 wherein an example of a foldable touch screen device providing a first display of a video application in accordance with the present application is shown. Figure 19 An example implementation of the method 1800 is shown. Figure 19 The foldable touch screen device 100 has a flexible touch screen element 140 having a first section 125, a second section 130, and a fold region 135. The flexible touch screen element 140 is displaying content in a dual screen mode, with the first section 125 displaying a currently executing video player application. A first application control overlay 1902 and a second application control overlay 1904 are displayed in the fold region 135. The first application control overlay 1902 is a brightness control overlay, and the second application control overlay 1904 is a volume control overlay. A finger of the user's hand 912 is touching the fold region 135 at a display location of the second application control overlay 1904.

[0186] For example, as a touch gesture is detected at a location of the fold region 135 corresponding to the second application control overlay 1904, the application can increase or decrease the volume of the video player application. Similarly, as a touch gesture (which can be a multi-finger gesture) is detected at a location of the fold region 135 corresponding to the first application control overlay 1902, the application can increase or decrease the brightness of the video player application.

[0187] The method 1800 can be performed when the foldable touch screen device is in a "tent mode." Figure 18 The method 1800. Figure 20A foldable touch screen device 100 providing a second display of a video application is shown in accordance with an example of the application. The foldable touch screen device 100 has a flexible touch screen element 140 with a first section 125, a second section 130, and a fold region 135. The foldable touch screen device 100 is in a "tent mode" orientation. The first section 125 and the fold region 135 of the flexible touch screen element are visible, and the foldable touch screen device 100 is executing a video player application. A first application control overlay 2002 and a second application control overlay 2004 are displayed in the fold region 135. Because of the "tent mode" orientation, the fold region 135 is at the top of the foldable touch screen device 100. The first application control overlay 2002 is a brightness control overlay, and the second application control overlay 2004 is a volume control overlay. A finger of a user's hand 912 is touching the fold region 135 at a display location of the second application control overlay 2004. A touch gesture can be detected in the fold region 135 to trigger an application control action corresponding to the application control overlays 2002, 2004 in a manner similar to that described above in connection with Figure 19 .

[0188] Figure 21 is a flowchart of an example method 2100 for triggering a first action corresponding to a touch gesture and a location corresponding to a thumb input overlay in accordance with an example of the application. The method 2100 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 ) operating as a foldable touch screen device 100 (e.g., the foldable touch screen device 100 of Figure 1 ). In particular, the operations 1710 and thereafter can be performed by one or more processors (e.g., the processor 302 of Figure 2 ) of the computing system.

[0189] Operation 2110: The system detects a holding posture of the foldable touch screen device.

[0190] The holding posture can refer to the position of a user's hand relative to the partially folded touch screen device 100 such that the user's thumb is located near the fold region 135 and the remaining one or more fingers are located under the device 100, for example, as shown in Figure 22 and Figure 23 .

[0191] The grip pose can be detected using one or more of a variety of methods, including capacitive touch screen sensing with or without internal inertial measurement unit (IMU) sensors and / or camera. For example, capacitive touch screen sensing can detect a user's thumb near the fold region 135, and / or the palms of both hands at respective locations of the first section 125 of the flexible touch screen element 140. Further or alternatively, internal IMUs can detect the partially folded pose of the touch screen device 100, and the initiation and maintenance of the second section 130 of the flexible touch screen element 140 in the raised position.

[0192] Operation 2120: The system displays a thumb input overlay at the fold region.

[0193] Operation 2130: The system detects a touch gesture at a location of the fold region corresponding to the thumb input overlay.

[0194] Operation 2140: The system triggers a first action corresponding to the touch gesture and the location corresponding to the thumb input overlay.

[0195] Reference is now made to Figure 22 wherein a foldable touch screen device 100 displaying a video game application is shown according to an example of the present application. Figure 22 An example implementation of the method 2100 is shown Figure 21 An example implementation of the method 2100 is shown The foldable touch screen device has a flexible touch screen element 140 having a first section 125, a second section 130, and a fold region 135. The foldable touch screen device 100 is in a laptop orientation, wherein the fold region 135 is oriented horizontally, and the foldable touch screen device 100 is half-folded. The foldable touch screen device 100 is displaying a currently executing video game application in the second section 130. A thumb input overlay 2202 including virtual keys is displayed near the bottom of the flexible touch screen element 140 in the first section 125. A user's left hand 2204 and a user's right hand 2206 are gripping the bottom of the foldable touch screen device 100 such that a user's left thumb 2208 and a user's right thumb 2210 are positioned near the virtual keys of the thumb input overlay 2202 displayed in the first section 125.

[0196] When a foldable touchscreen device (e.g., a smartphone) is in laptop mode, the user's thumb can easily reach the folded area, thus providing natural input space for tapping and / or swiping. By using dynamic hinge shortcuts in the folded area, an additional dimension can be intuitively added to game controls. For example, in some implementations, swipe gestures in the folded area can be used to zoom in, zoom out, or switch weapons. Using dynamic hinge shortcuts in the folded area can provide a more immersive and intuitive gaming experience on foldable touchscreen devices (e.g., foldable smartphones).

[0197] For example, it can be performed while a foldable touchscreen device is running a camera application. Figure 21 Method 2100. Now for reference. Figure 23 The illustration shows a foldable touchscreen device 100 for displaying a camera application according to an example of the present invention. The foldable touchscreen device 100 has a flexible touchscreen element 140 having a first partition 125, a second partition 130, and a folding region 135. The foldable touchscreen device 100 is oriented towards a laptop and is displaying the viewport of the currently running camera application in the first partition 125. A thumb input overlay 2302, including a first application control overlay 2304 and a second application control overlay 2306, is displayed in the folding region 135 of the flexible touchscreen element 140. In this example, the first application control overlay 2304 is an exposure control overlay, and the second application control overlay 2306 is a zoom control overlay. Other user interface elements of the camera application can be displayed in the second partition 130. The user's left hand 2204 and right hand 2206 are gripping the bottom of the foldable touchscreen device 100, such that the user's left thumb 2208 and right thumb 2210 are located near the first and second application control overlays, respectively.

[0198] This allows for dynamic sliding control within the folded area 135, for example, camera application actions such as "zoom" and "exposure".

[0199] Figure 22 and Figure 23 An example of how to display an application-specific control overlay in the folded area 135 is shown so that the user can intuitively and conveniently access the application-specific control overlay when holding the foldable touchscreen device 100 in a semi-folded position (e.g., with the laptop facing outwards). It should be understood that other implementations of method 2100 may exist when performing other applications, and are not limited to game applications or camera applications.

[0200] Figure 24is a flowchart of an example method 2400 for modifying one of one or more user-modifiable objects according to an example of the disclosure. The method 2400 can be performed by one or more processors of a computing system (e.g., the computing system 200 of Figure 2 FIG. 1) running as a foldable touch screen device 100 Figure 1 . In particular, the operations 2410 and thereafter can be performed by one or more processors of the computing system (e.g., the processors 302 of Figure 2 FIG. 1).

[0201] Operation 2410: The system displays one or more user-modifiable objects.

[0202] Operation 2420: The system receives a selection of one of the one or more user-modifiable objects.

[0203] Operation 2430: The system displays one or more shortcut overlays corresponding to the one of the one or more user-modifiable objects at the fold region.

[0204] Operation 2440: The system modifies the one of the one or more user-modifiable objects according to an action corresponding to a selected one of the one or more shortcut overlays.

[0205] Figure 25 and Figure 26 shows an example implementation of the method 2400 of Figure 24 FIG. 1). Figure 25 shows a foldable touch screen device 100 providing a first display of a word processing application according to an example of the disclosure. The foldable touch screen device 100 has a flexible touch screen element 140 with a first section 125, a second section 130, and a fold region 135. The foldable touch screen device 100 is in a full screen mode and a laptop orientation. The foldable touch screen device 100 is executing a word processing application and is displaying text. The text can include a user-modifiable object, e.g., a selected portion of the text. A first portion of the text 2502 is selected text, indicated by highlighting on the first portion of the text 2502. A finger of a user's hand 912 is touching the flexible touch screen element 140 at a location of the first portion of the text 2502.

[0206] Figure 26This illustrates that after the first portion of text 2502 is selected, one or more shortcut overlays 2602 are displayed in the folded area 135. The one or more shortcut overlays 2602 correspond to actions that can be applied to selecting text 2502. The user's hand 912 is touching the flexible touchscreen element 140 at the location of one or more shortcut overlays 2602.

[0207] Providing one or more shortcut overlays in the collapsed area allows users to easily access common text editing actions such as copy, paste, and formatting options (e.g., bold, italics, and underline) by simply selecting a portion of text to activate one or more shortcut overlays in the collapsed area. Once an overlay is activated, users can simply select one of the one or more shortcut overlays representing the desired action to adjust the text as needed. This allows users to edit text quickly and efficiently without having to navigate through multiple menus or use keyboard shortcuts. One or more shortcut overlays can include the most relevant and useful shortcuts, making the editing process more efficient and intuitive.

[0208] Figure 27 and Figure 28 It shows Figure 24 Another exemplary implementation of method 2400. Figure 27 A foldable touchscreen device 100 providing a first display for a design application according to an example of the present invention is shown. The foldable touchscreen device 100 has a flexible touchscreen element 140 having a first partition 125, a second partition 130, and a folding region 135. The foldable touchscreen device 100 is executing a design application. A value input box 2702 is displayed in the upper right portion of the flexible touchscreen element 140. A user's hand 912 is touching an element of the value input box 2702. In some embodiments, after activating the value input box 2702 (e.g., by touch input selection), a dynamic slider can be displayed in the folding region 135, as will be referenced. Figure 28 Described.

[0209] Figure 28 The image shows a dynamic slider 2802 displayed in the collapsed area 135 after the activation value input box 2702. The user's hand 912 is touching the dynamic slider 2802.

[0210] In some embodiments, the dynamic slider 2802 at the fold area 135 can be activated only after the value entry box 2702 is selected, thereby ensuring that the dynamic slider 2802 is displayed at the fold area 135 only when the user needs it. Other context-specific action shortcuts can be displayed in the fold area 135 in a similar manner, depending on the current user selections and / or currently active functions of the application. Thus, the user interaction with the design application can be efficient and intuitive.

[0211] As such, the user can easily adjust values in the design application by simply selecting the value entry box 2702 and using the dynamic slider 2802 at the fold area 135. Manual entry of values can be avoided, and quick adjustments can be made without having to open multiple menus and / or use keyboard shortcuts step-by-step.

[0212] As described herein, additional user utility and enjoyment can be provided with the fold area or "hinge" of a foldable touch screen device. The embodiments disclosed herein describe a dynamic portal that provides shortcuts at the hinge, which can adapt to user input as well as system events and / or application events. These shortcuts can reduce or eliminate the need to click through multiple steps in multiple menus to perform commonly performed tasks. The embodiments described herein facilitate Windows TM management and multitasking, and can be used in a variety of application scenarios, including presentation applications (e.g., Powerpoint TM ), e-reading applications, video applications, camera applications, video game applications, and text editing applications.

[0213] The embodiments described herein can be implemented in any combination. A single device can be used to implement any one or more or all of the embodiments in any combination.

[0214] Although the present application describes methods and processes by steps performed in a certain order, one or more steps of the methods and processes can be omitted or changed as appropriate. In appropriate circumstances, one or more steps can be performed in an order other than the order described.

[0215] Although the present application is described in terms of methods, one of ordinary skill in the art will appreciate that the present application is also directed to various components for performing at least some of the aspects and features of the methods, whether by hardware components, software, or any combination thereof. Accordingly, the technical solutions of the present application can be embodied in the form of a software product. The appropriate software product can be stored in a pre-recorded storage device or other similar non-volatile or non-transitory computer-readable medium, including a DVD, a CD-ROM, a USB flash disk, a removable hard disk, or other storage medium. The software product includes instructions tangibly stored thereon, which enable a processing device (e.g., a personal computer, a server, or a network device) to perform examples of the methods disclosed herein.

[0216] The present application can be implemented in other specific forms without departing from the subject matter of the claims. The described exemplary embodiments are merely illustrative in all aspects and are not restrictive. Features selected from one or more of the above-described embodiments can be combined to create alternative embodiments that are not explicitly described, and features suitable for such combinations can be understood within the scope of the present application.

[0217] All values and subranges within the scope of the disclosure are also disclosed. Furthermore, although the systems, devices, and processes disclosed and shown herein can include particular numbers of elements / components, these systems, devices, and components can be modified to include more or less of such elements / components. For example, although any disclosed element / component can be a single number, embodiments disclosed herein can be modified to include multiple such elements / components. The subject matter described herein is intended to cover and encompass all suitable technical modifications.

Claims

1. A foldable touch screen device, characterized by, comprising: a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section, the touch screen element being foldable at the fold region; a processor; a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to: display a first object in the first section; detect initiation of a drag gesture at a location of the first section corresponding to the first object; detect a path of the drag gesture proximate to or across a first one of one or more shortcut overlays displayed in the fold region; trigger a first action corresponding to the first one of the one or more shortcut overlays.

2. The foldable touch screen device of claim 1, wherein, the foldable touch screen device is further to, prior to detecting the path of the drag gesture proximate to or across the first one of the one or more shortcut overlays: detect the drag gesture at a location of the first section proximate to the fold region; display the one or more shortcut overlays in the fold region.

3. The foldable touch screen device of claim 1 or 2, wherein, the foldable touch screen device is further to: detect cessation of the drag gesture; responsive to detecting the cessation of the drag gesture, remove the display of the one or more shortcut overlays from the fold region.

4. The foldable touch screen device of any one of claims 1-3, wherein, the one or more shortcut overlays are window management overlays.

5. The foldable touch screen device of any one of claims 1-3, wherein, the one or more shortcut overlays are parallel horizon overlays.

6. The foldable touch screen device of any one of claims 1-5, wherein, the foldable touch screen device is further to: detect the drag gesture across the fold region at a second one of the one or more shortcut overlays; trigger a second action corresponding to the second one of the one or more shortcut overlays.

7. A foldable touch screen device, characterized by comprising: a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section, the touch screen element being foldable at the fold region; a processor; a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to, during execution of an e-reader application: detect a drag gesture in a first direction along the fold region, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; insert a virtual bookmark at a current display location of an electronic document.

8. The foldable touch screen device of claim 7, wherein, the drag gesture originates at or proximate to a first edge of the fold region.

9. The foldable touch screen device of claim 7 or 8, wherein, the foldable touch screen device is further to: detect a drag gesture in a second direction along the fold region substantially opposite the first direction; remove the virtual bookmark.

10. A foldable touch screen device, characterized by comprising: a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section, the touch screen element being foldable at the fold region; a processor; a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to: displaying content using a full screen mode; detecting a drag gesture in a first direction along the fold region, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; displaying the content using a dual screen mode.

11. A foldable touch screen device, characterized by Including: a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section, the touch screen element being foldable at the fold region; a processor; a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to: detect a first touch gesture at the fold region; display a defined area containing application shortcuts.

12. A foldable touch screen device, characterized by Including: a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section, the touch screen element being foldable at the fold region; a processor; a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to, during execution of a particular application: display one or more application control overlays defined by the particular application at the fold region; detect a touch gesture at a location of the fold region corresponding to one of the one or more application control overlays; trigger a first action corresponding to the one of the one or more application control overlays.

13. A foldable touch screen device, characterized by Including: a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section, the touch screen element being foldable at the fold region; a processor; a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to: detect a grip pose of the foldable touch screen device; in response to detecting the grip pose, display a thumb input overlay at the fold region; detect a touch gesture at a location of the fold region corresponding to the thumb input overlay; trigger a first action corresponding to the touch gesture and the location corresponding to the thumb input overlay.

14. The foldable touch screen device of claim 13, wherein, the foldable touch screen device is executing a video game application, the first action being a video game control action.

15. The foldable touch screen device of claim 13, wherein, the foldable touch screen device is executing a camera application, the first action being a camera control action.

16. A foldable touch screen device, characterized by Including: a flexible touch screen element having a first section, a second section, and a fold region separating the first section and the second section, the touch screen element being foldable at the fold region; a processor; a memory storing instructions that, when executed by the processor, cause the foldable touch screen device to: display one or more user-modifiable objects; receive a selection of one of the one or more user-modifiable objects; display one or more shortcut overlays corresponding to actions to modify the one of the one or more user-modifiable objects at the fold region; In response to selecting one of the one or more shortcut overlays, modifying the one of the one or more user-modifiable objects in accordance with the action corresponding to the selected one of the one or more shortcut overlays.

17. A computer-implemented method, comprising: Comprising: displaying a first object in a first partition of a flexible touch screen element; detecting initiation of a drag gesture at a location of the first partition corresponding to the first object; detecting a path of the drag gesture proximate to or across a first one of one or more shortcut overlays displayed in a fold region of the flexible touch screen element; triggering a first action corresponding to the first one of the one or more shortcut overlays.

18. The method of claim 17, wherein, The method further comprises: detecting the drag gesture at a location of the first partition proximate to the fold region prior to detecting the path of the drag gesture proximate to or across the first one of the one or more shortcut overlays; displaying the one or more shortcut overlays in the fold region.

19. The method of claim 17 or 18, wherein, The method further comprises: detecting cessation of the drag gesture; in response to detecting the cessation of the drag gesture, removing the display of the one or more shortcut overlays from the fold region.

20. The method of any one of claims 17-19, wherein, The one or more shortcut overlays are window management overlays.

21. The method of any one of claims 17-19, wherein, The one or more shortcut overlays are parallel universe overlays.

22. The method of any one of claims 17-21, wherein, The method further comprises: detecting the drag gesture across a second one of the one or more shortcut overlays; triggering a second action corresponding to the second one of the one or more shortcut overlays.

23. A computer-implemented method, comprising: Comprising: detecting a drag gesture in a first direction along a fold region of a flexible touch screen element, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; inserting a virtual bookmark at a current display location of an electronic document.

24. The method of claim 23, wherein, The drag gesture originates at or proximate to a first edge of the fold region.

25. The method of claim 23 or 24, wherein, The method further comprises: detecting a drag gesture in a second direction along the fold region substantially opposite the first direction; removing the virtual bookmark.

26. A computer-implemented method, comprising: Comprising: displaying content on a flexible touch screen element using a full screen mode; detecting a drag gesture in a first direction along a fold region of the flexible touch screen element, wherein the first direction is substantially parallel to a longitudinal axis of the fold region; displaying the content using a dual screen mode.

27. A computer-implemented method, comprising: Comprising: detecting a first touch gesture at a fold region of a flexible touch screen element; displaying a defined region containing application shortcuts.

28. A computer-implemented method, comprising: Comprising: displaying one or more application control overlays defined by an executing application at a fold region of a flexible touch screen element; detecting a touch gesture at a location of the fold region corresponding to one of the one or more application control overlays; triggering a first action corresponding to the one of the one or more application control overlays.

29. A computer-implemented method, comprising: Comprising: Detecting a holding posture of a foldable touch screen device, wherein the foldable touch screen device has a flexible touch screen element; In response to detecting the holding posture, displaying a thumb input overlay at a folding region of the flexible touch screen element; Detecting a touch gesture at a location of the folding region corresponding to the thumb input overlay; Triggering a first action corresponding to the touch gesture and the location corresponding to the thumb input overlay.

30. The method of claim 29, wherein, The foldable touch screen device is executing a video game application, and the first action is a video game control action.

31. The method of claim 29, wherein, The foldable touch screen device is executing a camera application, and the first action is a camera control action.

32. A computer-implemented method, comprising: Comprising: Displaying one or more user-modifiable objects on a flexible touch screen element; Receiving a selection of one of the one or more user-modifiable objects; Displaying one or more shortcut overlay layers corresponding to actions to modify the one of the one or more user-modifiable objects at a folding region of the flexible touch screen element; In response to selecting one of the one or more shortcut overlay layers, modifying the one of the one or more user-modifiable objects according to the action corresponding to the selected one of the one or more shortcut overlay layers.

Images