Electronic device for displaying visual object for providing plurality of screens, and method therefor
The electronic device enables efficient multitasking by allowing users to create split-screen arrangements through intuitive input gestures, addressing the challenge of setting up multiple applications simultaneously on a single screen.
Patent Information
- Application Number
- PCT/KR2024/002912
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-07-12
- Filing Date
- 2024-03-06
- Publication Date
- 2024-12-05
AI Technical Summary
Users face difficulties in creating split screens on electronic devices, particularly from the home screen, as the process is laborious and unintuitive, making it time-consuming to operate multiple applications simultaneously.
An electronic device with a processor and display that allows users to select and move visual objects to create split-screen arrangements by identifying inputs such as drag gestures, enabling the simultaneous display of multiple applications within designated areas on the screen.
Facilitates intuitive and efficient multitasking by allowing users to easily arrange and display multiple applications in split-screen configurations, improving user experience and reducing the complexity of setting up multi-window views.
Smart Images

Figure KR2024002912_05122024_PF_FP_ABST
Abstract
Description
Electronic device and method for displaying visual objects for providing multiple screens
[0001] The present disclosure relates to an electronic device and method for displaying a visual object to provide multiple screens.
[0002] The shapes and sizes of electronic devices are becoming increasingly diverse as their functions diversify. To improve mobility, electronic devices with reduced size and / or volume are being designed. Electronic devices may include displays for visualizing information. Electronic devices can use these displays to provide users with various types of information.
[0003] According to an embodiment, an electronic device may include a display, a memory including one or more storage media storing instructions, and at least one processor including a processing circuit. The instructions, when individually and / or collectively executed by the at least one processor, may be configured to cause the electronic device to display a plurality of visual objects on the display. The instructions, when individually and / or collectively executed by the at least one processor, may be configured to cause the electronic device to identify a first input indicating a selection of a first visual object from among the plurality of visual objects. The instructions, when individually and / or collectively executed by the at least one processor, may be configured to cause the electronic device to, in response to a second input for a second visual object from among the plurality of visual objects, move a location of the second visual object from a first location to a second location in response to the second input. The instructions, when individually and / or collectively executed by the at least one processor, may be configured to cause the electronic device to display a third visual object on the display based on identifying that the second visual object in the second location at least partially overlaps a first portion of the first visual object. The instructions, when individually and / or collectively executed by the at least one processor, may be configured to cause the electronic device to, in response to a third input indicating selection of the third visual object, display a first screen corresponding to the first visual object within a first area of the display and display a second screen corresponding to the second visual object within a second area of the display as a split-view.The instructions, when individually and / or collectively executed by the at least one processor, may be configured to cause the electronic device to display a fourth visual object on the display based on identifying that the second visual object within the second location overlaps a second portion of the first visual object that is different from the first portion.
[0004] According to one embodiment, a method of an electronic device may include an operation of displaying a plurality of visual objects on a display of the electronic device. The method may include an operation of identifying a first input indicating selection of a first visual object from among the plurality of visual objects. The method may include an operation of moving a position of a second visual object from a first position to a second position in response to a second input for a second visual object from among the plurality of visual objects in response to the second input. The method may include an operation of displaying a third visual object based on identifying that the second position of the second visual object at least partially overlaps a first portion of the first visual object. The method may further include an operation of displaying a first screen corresponding to the first visual object in a first area of the display and a second screen corresponding to the second visual object in a split-view in a second area of the display in response to a third input indicating selection of the third visual object. The method may include displaying a fourth visual object on the display based on identifying that the second visual object within the second location overlaps a second portion of the first visual object that is different from the first portion.
[0005] According to one embodiment, an electronic device may include a display and a processor. The processor may be configured to display a visual object for simultaneously providing multiple screens within a list of software applications while displaying the list. The processor may be configured to, in response to an input indicating a selection of the visual object, display a first screen among the multiple screens based on a size of an active area of the display. The processor may be configured to, in response to the input, display a second screen among the multiple screens in an overlapping manner on the first screen displayed in the active area.
[0006] According to one embodiment, a method of an electronic device may include an operation of displaying a visual object for providing a plurality of screens simultaneously within a list of software applications installed in the electronic device while the list is displayed. The method may include an operation of displaying a first screen among the plurality of screens based on a size of an active area of a display of the electronic device in response to an input indicating a selection of the visual object. The method may include an operation of displaying a second screen among the plurality of screens in an overlapping manner on the first screen displayed in the active area in response to the input.
[0007] FIG. 1 illustrates exemplary states of an electronic device according to one embodiment.
[0008] FIG. 2 illustrates an example block diagram of an electronic device according to one embodiment.
[0009] FIG. 3 illustrates an example of a flowchart of a method performed by an electronic device, according to one embodiment.
[0010] FIGS. 4A to 4C illustrate an example of the operation of an electronic device displaying a visual object for providing multiple screens.
[0011] FIGS. 5A to 5D illustrate an example of the operation of an electronic device displaying a visual object for providing multiple screens.
[0012] Figures 6a and 6b illustrate an example of the operation of an electronic device based on the positions of overlapping visual objects.
[0013] Figure 7 illustrates an example of the operation of an electronic device based on the positions of overlapping visual objects.
[0014] FIGS. 8A to 8E illustrate an example of a flowchart of an electronic device according to one embodiment.
[0015] FIG. 9 illustrates an example of the operation of an electronic device that identifies an input representing an overlap of visual objects.
[0016] FIG. 10 illustrates an example of a flowchart of an electronic device according to one embodiment.
[0017] FIGS. 11A and 11B illustrate an example of the operation of an electronic device that identifies an input for generating a visual object providing multiple screens.
[0018] Figures 12a to 12c illustrate an example of an operation of an electronic device performed based on a virtual space.
[0019] FIGS. 13A and 13B illustrate an example of the operation of an electronic device connected to one or more external displays.
[0020] Figure 14 illustrates an example of the operation of an electronic device for adjusting a layout corresponding to a visual object.
[0021] FIG. 15 illustrates an example of operation of an electronic device according to one embodiment.
[0022] FIG. 16 is a block diagram of an electronic device within a network environment according to various embodiments.
[0023] Hereinafter, various embodiments of this document are described with reference to the attached drawings.
[0024] The various embodiments of this document and the terminology used therein are not intended to limit the technology described in this document to a specific embodiment, but should be understood to include various modifications, equivalents, and / or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expression may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and / or B", "A, B, or C", or "at least one of A, B, and / or C" may include all possible combinations of the items listed together. Expressions such as "first", "second", "first", or "second" may modify the corresponding components regardless of order or importance, and are only used to distinguish one component from another, but do not limit the corresponding components. When it is said that a component (e.g., a first component) is “(functionally or communicatively) connected” or “connected” to another component (e.g., a second component), said component may be directly connected to said other component, or may be connected via another component (e.g., a third component).
[0025] The term "module" as used in this document includes a unit composed of hardware, and may be used interchangeably with terms such as "component" or "circuit." A module may be an integrally composed component, or a minimal unit or portion thereof that performs one or more functions. For example, a module may be composed of an application-specific integrated circuit (ASIC).
[0026] Users of electronic devices may increasingly desire to display and utilize multiple applications on their devices. Splitting the screen allows multiple applications to be displayed simultaneously on a single display, with each application screen appearing within its own designated area of the display. However, creating split screens has traditionally been difficult for users, especially from the home screen. Setting up split screens can be a laborious, complex, and unintuitive process. When a user wants to use split screens to operate two or more applications simultaneously, going through this process each time can be time-consuming and frustrating.
[0027] FIG. 1 illustrates, by way of example, states (191, 192, 193, 194, and 195) of an electronic device (101) according to one embodiment. States (191, 192, 193, 194, and 195) illustrate some examples of different possible arrangements of the display when presenting content to a user. In one embodiment, the electronic device (101) may be referred to as a terminal (or user terminal) in terms of being owned by a user. A terminal may include, for example, a personal computer (PC) such as a laptop or desktop. A terminal may include, for example, a smartphone, a smartpad, and / or a tablet PC. A terminal may include a smart accessory such as a smartwatch and / or a head-mounted device (HMD). One or more hardwares included in the electronic device (101) of FIG. 1 are described with reference to FIG. 2 and / or FIG. 16.
[0028] Referring to FIG. 1, different states (191, 192, 193, 194, 195) of an electronic device (101) are illustrated, distinguished by a screen displayed to a user. That is, the screen may be divided into a plurality of different methods or configurations when displaying content to a user. Hereinafter, a screen may refer to a user interface (UI) displayed within at least a portion of the display (110). The screen may include, for example, Windows of a Windows operating system and / or an activity of an Android operating system. The embodiment is not limited thereto. For example, the screen displayed by the electronic device (101) may be formed in an external space by light output from the electronic device (101) to the external space. For example, the screen may be formed on a plane onto which the light is projected within the external space. For example, the screen may be displayed three-dimensionally in the form of a hologram within the external space.
[0029] According to one embodiment, an electronic device (101) may include one or more software applications to support various functions. The electronic device (101) may display a list of functions supported by the electronic device (101) on the display (110). When multiple software applications are installed on the electronic device (101), a list of the multiple software applications may be displayed on the display (110) as a list of functions supported by the electronic device (101). Referring to FIG. 1, a state (191) of the electronic device (101) displaying a screen including a list of software applications is exemplarily illustrated. For example, within state (191), the electronic device (101) may display a screen provided from a designated software application, such as a launcher application. The screen displayed by the electronic device (101) within state (191) may be referred to as a launcher screen and / or a home screen.
[0030] Referring to FIG. 1, the screen displayed by the electronic device (101) within the state (191) may be divided into different regions (112, 114, 116, 118). Each of the regions (112, 114, 116, 118) may be formed along the width direction among the width direction and the height direction of the display (110) within the portrait mode of the electronic device (101). The mode of the electronic device (101) may include the portrait mode and the landscape mode described above. The portrait mode may be a mode in which a first direction of a relatively short edge of the display (110) corresponds to the width direction of the screen displayed on the display (110), and a second direction of a relatively long edge corresponds to the height direction of the screen. Landscape mode may be a mode in which the first direction of the display (110) corresponds to the height direction of the screen displayed on the display (110), and the second direction corresponds to the width direction of the screen. An electronic device (101) in landscape mode may arrange regions (112, 114, 116, 118) along the first direction and / or the height direction of the display (110).
[0031] Referring to FIG. 1, an area (112) within a home screen displayed by an electronic device (101) within a state (191) may be referred to as a status bar. Within the area (112), the electronic device (101) may display text, icons, thumbnails, and / or images indicating the time, the state of charge (SOC) of the battery, and / or the sensitivity of the wireless communication network. The text and / or icons displayed in the area (112) are not limited to the above examples, and the electronic device (101) may display text and / or icons in the area (112) while a software application (or system process) is running. In one embodiment, based on (or in response to) a user input to the area (112), the electronic device (101) may display a notification panel (or notification drawer). The user input may be identified, for example, by a swipe gesture performed along a designated direction of the display (110) (e.g., in the direction of the first edge of the display (110) or a second edge shorter than the first edge).
[0032] Referring to FIG. 1, an area (118) displayed by an electronic device (101) within a state (191) may be referred to as a navigation bar. While screens of the electronic device (101) including the area (118) are illustratively illustrated, the electronic device (101) may at least temporarily hide the area (118) within the display (110). Within the area (118), the electronic device (101) may display executable objects for switching the screen displayed via the display (110). The executable objects displayed in the area (118) may include at least one of a visual object (e.g., a home button) for displaying a launcher screen and / or a home screen, such as a screen displayed on the display (110) within the state (191), a visual object (e.g., a multi-tasking button) for displaying a list of screens displayed by the electronic device (101), or a visual object (e.g., a back button) for switching to a screen that was displayed immediately before the screen currently displayed on the display (110) among the screens. A visual object may mean an object that can be deployed within a screen for transmitting and / or interacting with information. A visual object may be referred to as an icon, a visual element, a UI element, a view object, and / or a view element. A visual object may include text, an image, an icon, a video, a button, a checkbox, a radio button, a text box, a slider, and / or a table.
[0033] Referring to FIG. 1, an area (114) displayed by an electronic device (101) within a state (191) may be referred to as a workspace area, a launcher area, and / or a home area. Within the area (114), the electronic device (101) may display icons representing software applications. In terms of displaying the icons arranged in a grid, the area (114) may be referred to as a grid area. Referring to FIG. 1, visual objects (131, 132, 133) in the form of icons representing software applications installed on the electronic device (101) may be displayed within the area (114). The visual objects and icons may be used interchangeably. Each of the visual objects (131, 132, 133) may correspond to different software applications installed on the electronic device (101). For example, a visual object (131) including an icon for executing an email application for opening an email, a visual object (132) including an icon representing an alarm application for providing an alarm, and / or a visual object (133) corresponding to a weather application for providing weather information are exemplarily illustrated, but the embodiment is not limited thereto. Providing input to a visual object can cause an application associated with the selected visual object to be executed, such that a screen associated with the application is displayed on the display.
[0034] Referring to FIG. 1, the electronic device (101) may display a widget (122) within an area (114) along with an icon corresponding to a software application. The widget (122) may be placed within the area (114) to visualize information identified by the electronic device (101) and / or to more quickly receive input for executing a specific function. Although the widget (122) in the form of a text box for quickly receiving a query for a search is illustrated as an example, the shape, color, and / or size of the widget (or gadget) displayable through the area (114) is not limited to the widget (122) of FIG. 1.
[0035] According to one embodiment, the electronic device (101) may display visual objects (120) within an area (114) for switching a combination of icons and / or widgets (122) displayed through the area (114). The visual objects (120) may be referred to as a pager and / or a page controller. While visual objects (120) including a plurality of dots are exemplarily illustrated, the embodiment is not limited thereto, and a page controller including different numbers may be displayed. Based on an input indicating a selection of any one of the visual objects (120), the electronic device (101) may change the icons and / or widgets (122) displayed in the area (114).
[0036] Referring to FIG. 1, an area (116) displayed by the electronic device (101) within a state (191) may be referred to as a hotseat and / or a dock area. Within the area (116), the electronic device (101) may display icons corresponding to pre-registered software applications. For example, the electronic device (101) may display at least one icon registered by a user input within the area (116). Within the state (191), the electronic device (101) may maintain displaying icons within the area (116) independently of changing icons and / or widgets (122) within the area (114) based on visual objects (120).
[0037] According to one embodiment, the electronic device (101) may identify, within a state (191), an input indicating execution of at least one software application. Based on the input, the electronic device (101) may display, within the display (110), a screen (or a combination of screens) provided from at least one software application corresponding to the input. For example, within the state (191), in response to an input indicating selection of a visual object (131), the electronic device (101) may switch to a state (192) to provide a screen of a software application corresponding to the visual object (131). The input indicating selection of the visual object (131) may be identified by a tap gesture of tapping a portion of the display (110) on which the visual object (131) is displayed and / or a mouse input of clicking the visual object (131). A tap gesture may be a touch on a portion of the display for less than a specified amount of time.
[0038] Referring to FIG. 1, in a state (192), the electronic device (101) may execute a software application (e.g., an email application) corresponding to a visual object (131) and display a screen provided from the executed software application based on the size of an active area (or display area) of the display (110). Since the electronic device (101) displays a screen based on the size of the active area, the width and / or height of the screen displayed on the display (110) may substantially match the width and / or height of the active area.
[0039] Referring to FIG. 1, in response to an input indicating selection of a visual object (132) within a state (191), the electronic device (101) may switch to a state (193) and display a screen provided from an alarm application corresponding to the visual object (132). Within the state (193), the screen displayed within the display (110) may occupy the entire area of the active area of the display (110).
[0040] According to one embodiment, the electronic device (101) may display a visual object for providing a combination of screens based on multi-tasking within an area (114) where icons (e.g., visual objects (131, 132, 133)) for executing a single software application are displayed. Referring to FIG. 1, visual objects (141, 142, 143, 144) that are displayed to provide a combination of screens based on multi-tasking and have the form of icons are exemplarily illustrated. Each of the visual objects (141, 142, 143, 144) may include images and / or icons representing screens to be displayed by the electronic device (101). For example, within the visual object (141) for providing a combination of screens provided from an email application and an alarm application, the electronic device (101) may display icons corresponding to each of the email application and the alarm application. The layout of icons displayed within a visual object (141) may correspond to the layout (or positional relationship and / or arrangement) of screens to be displayed within a display (110) by input to the visual object (141).
[0041] Referring to FIG. 1, in a state (191) displaying a visual object (141) including icons arranged along the vertical direction of the display (110), the electronic device (101) may switch to a state (194) based on an input indicating selection of the visual object (141). Hereinafter, the vertical direction may correspond to the direction of the first edge of the display (110) or a second edge shorter than the first edge in the portrait mode, and may correspond to the second edge in the landscape mode. In the state (194), the electronic device (101) may display screens corresponding to icons displayed through the visual object (141) along the vertical direction of the display (110). For example, the electronic device (101) may display the screens in a first region (151) including the top of the display (110) and a second region (152) including the bottom of the display (110). For example, if an icon corresponding to an alarm application is arranged below an icon corresponding to an email application along the vertical direction within a visual object (141), the electronic device (101) may display a screen provided from the alarm application below a screen provided from the email application along the vertical direction based on an input to the visual object (141). In the state (194) of FIG. 1, the areas (151, 152) where the screens of the email application and the alarm application are respectively displayed may be arranged along the vertical direction within the display (110). Between the areas (151, 152), the electronic device (101) may display a boundary line (163) (or frame line) parallel to the horizontal direction of the display (110). Hereinafter, the horizontal direction of the display (110) may correspond to the direction of the second edge shorter than the first edge of the display (110) within the portrait mode, and to the first edge within the landscape mode.
[0042] Referring to FIG. 1, in a state (194) of displaying a combination of a plurality of screens, the electronic device (101) may display handles (161, 162) for adjusting the position and / or size of each of the plurality of screens, together with a boundary line (163) for distinguishing the plurality of screens. A handle (161) superimposed on a first area (151) where a first screen provided from an email application is displayed may be displayed to receive an input for changing the position and / or size of the first screen within the display (110). Based on an input indicating selection of the handle (161), the electronic device (101) may display a menu including options for controlling the first screen displayed through the first area (151).
[0043] Referring to FIG. 1, similarly to the handle (161) in the first area (151), the electronic device (101) can identify an input for changing the position and / or size of the second screen within the display (110) through the handle (162) superimposed on the second area (152) where the second screen provided from the alarm application is displayed. Based on the input indicating movement of the handle (162), the electronic device (101) can change the positional relationship of the areas (151, 152). Similarly, based on the input indicating movement of the boundary line (163), the electronic device (101) can change the ratio of the areas (or sizes) of the display (110) occupied by each of the areas (151, 152). The ratio may include the ratio between the edges of the areas (151, 152) and / or the ratio of the areas of the areas (151, 152).
[0044] Referring to FIG. 1, a visual object (142) is illustrated for providing a combination of screens provided from an email application and an alarm application. Within the visual object (142), icons corresponding to the email application and the alarm application may be arranged along a direction of a second edge shorter than a first edge of the display (110). For example, within the visual object (142), an icon corresponding to the alarm application may be arranged to the left of an icon corresponding to the email application. In response to an input indicating selection of the visual object (142), the electronic device (101) may display screens provided from the email application and the alarm application along the direction of the second edge within the display (110). For example, the electronic device (101) may display a screen provided from the alarm application on a portion including a left side of the display (110), and may display a screen provided from the email application on a portion including a right side of the display (110). Similar to the handles (161, 162) in the state (194), the electronic device (101) may display handles for changing the position and / or size of the screens, overlapping each portion of the display (110) where the screens are displayed.
[0045] Referring to FIG. 1, a visual object (143) for providing a combination of screens provided from three software applications (e.g., an alarm application, an email application, and a weather application) may be displayed within a display (110). Within the visual object (143), the electronic device (101) may guide the arrangement and / or positional relationship of the screens to be displayed within the display (110) by an input indicating selection of the visual object (143) based on the arrangement and / or positional relationship of icons of the software applications.
[0046] Referring to FIG. 1, within a state (194), the electronic device (101) may execute a multi-tasking-based function by using a first region (151) and a second region (152) positioned below the first region (151) along one direction of the display (110) (e.g., a vertical direction corresponding to the first edge of the display (110) or a second edge shorter than the first edge). Picture-by-picture (PBP) may include a state (194) of displaying screens provided from a plurality of software applications by using the regions (151, 152) positioned side by side. When displaying screens based on PBP, the electronic device (101) may display the screens horizontally or vertically. Screens displayed within the display (110) based on PBP may not overlap each other within the display (110). For example, among the screens displayed based on PBP, a first screen may be displayed alongside a second screen among the screens within the display (110). Based on PBP, the electronic device (101) may tile the screens within the display (110).
[0047] Referring to FIG. 1, a visual object (144) for providing a combination of screens provided from software applications (e.g., an alarm application and an email application) based on a picture-in-picture (PIP) that is distinct from the PBP may be displayed within the display (110). The electronic device (101) may display the visual object (144) including an icon corresponding to the alarm application and an icon corresponding to the email application. Within the visual object (144), the icons may be arranged based on a layout of screens to be displayed on the display (110) (e.g., a layout based on the PIP) based on an input related to the visual object (144). For example, within the visual object (144), an icon corresponding to the email application may be superimposed on an icon corresponding to the alarm application based on the PIP layout. Based on an input to the visual object (144) displayed within the state (191), the electronic device (101) may switch to the state (195).
[0048] Referring to FIG. 1, within a state (195), the electronic device (101) may display a first screen provided from an alarm application in a first area (171) having a size corresponding to an active area of the display (110). The electronic device (101) may display a second screen provided from an email application in a second area (172) overlapping the first area (171). The PIP may include a state (195) that displays screens provided from a plurality of software applications using the overlapping areas (171, 172). Within the state (195), the second screen may be displayed to obscure (or cover) at least a portion of the first screen. The arrangement of the areas (171, 172) in which the first screen and the second screen are displayed within the state (195) may correspond to the arrangement of icons included in the visual object (144). Similar to the handles (161, 162) in the state (194), the electronic device (101) may display handles (181, 182) for adjusting the position and / or size of the screens displayed in each of the areas (171, 172).
[0049] As shown in states (194, 195) of FIG. 1, the electronic device (101) can simultaneously display screens provided from software applications that are substantially simultaneously executed based on PBP and / or PIP. A state (e.g., state (194)) in which screens are displayed in areas (151, 152) so as not to overlap each other based on PBP may be referred to as a multi-window. A state (e.g., state (195)) in which screens are displayed in areas (171, 172) that at least partially overlap based on PIP may be referred to as a pop-up window. In some embodiments, the pop-up window may correspond to a screen having a relatively small size among the screens displayed based on PIP. The states (e.g., states (194, 195)) of the electronic device (101) that provide the above screens based on PBP and / or PIP may be referred to as split screen, multi-window, multi-tasking, split window, split-view and / or multi-view.
[0050] As described above, according to one embodiment, the electronic device (101) can display visual objects (141, 142, 143, 144) for simultaneously displaying screens based on a multi-window within a launcher screen. In response to selecting the visual objects (141, 142, 143, 144), the electronic device can substantially simultaneously execute and display two or more applications represented by the selected visual objects. The electronic device (101) can display the visual objects (141, 142, 143, 144) based on an input (e.g., a drag input) for moving an icon (e.g., the visual objects (131, 132, 133)) corresponding to one software application. For example, the electronic device (101) may display another visual object (e.g., visual objects (141, 142, 144)) to simultaneously provide screens related to the visual objects (131, 132) based on an input of moving a visual object (132) over a visual object (131). In other words, the electronic device may generate and display the visual objects (141, 142, 144) in response to receiving a user input. The user input may be a touch gesture for dragging a first visual object (132) to a second visual object (132) such that portions of the visual objects overlap. The electronic device (101) may determine the layout (or arrangement or configuration) of the screens to be displayed through the other visual objects based on the position and / or direction of the visual object (132) moved over the visual object (131) by the input. In other words, when a generated visual object (141, 142, 143, 144) is selected by a user, the electronic device can determine the layout of the multi-window based on the layout depicted within the visual object (141, 142, 143, 144).The layout of the visual objects (141, 142, 143, 144) may be determined based on how a portion of the visual object (132) overlaps a portion of the visual object (131) by an input used to generate the visual objects (141, 142, 143, 144). As in states (194, 195) of FIG. 1, the electronic device (101) may determine the layout among PIP and / or PBP. According to one embodiment, an operation of the electronic device (101) identifying the input is described with reference to FIGS. 4A to 4C, 5A to 5D, 6A to 6B, 7, 9, 11A to 11B, and 12A to 12C. An operation of an electronic device (101) displaying screens based on the other visual objects is described with reference to FIGS. 13a and 13b. An operation of an electronic device (101) changing the layout of the screens to be displayed through the other visual objects is described with reference to FIGS. 14 and 15.
[0051] FIG. 2 illustrates an example of a block diagram of an electronic device (101) according to one embodiment. The electronic device (101) of FIG. 1 may be an example of the electronic device (101) of FIG. 2. The electronic device (101) may be classified into an electronic device (101-1) having a bar-type smartphone shape, an electronic device (101-2) having a deformable shape, and / or an electronic device (101-3) having a wearable shape by a user, depending on its shape, size, and / or form factor. Operations related to the electronic device (101-2) are described with reference to FIGS. 5A to 5D. Operations related to the electronic device (101-3) are described with reference to FIGS. 12A to 12C.
[0052] Referring to FIG. 2, according to one embodiment, an electronic device (101) may include at least one of a processor (210), a memory (215), a display (110), and a sensor (225). The processor (210), the memory (215), the display (110), and the sensor (225) may be electrically and / or operably coupled with each other by an electronic component (or electrical component), such as a communication bus (202). Hereinafter, operably coupled hardware may mean that a direct connection or an indirect connection is established between the hardwares, either wired or wireless, so that a second hardware is controlled by a first hardware among the hardwares. Although illustrated based on different blocks, the embodiment is not limited thereto, and some of the hardware of FIG. 2 (e.g., at least a portion of the processor (210) and the memory (215)) may be included in a single integrated circuit such as a system on a chip (SoC). The type and / or number of hardware included in the electronic device (101) is not limited to that illustrated in FIG. 2. For example, the electronic device (101) may include only some of the hardware illustrated in FIG. 2.
[0053] According to one embodiment, the processor (210) of the electronic device (101) may include hardware and / or circuitry for processing data based on one or more instructions. The hardware for processing data may include, for example, an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). The number of processors (210) may be one or more. For example, the processor (210) may have a multi-core processor structure such as a dual core, a quad core, or a hexa core.
[0054] According to one embodiment, the memory (215) of the electronic device (101) may include hardware components for storing data and / or instructions input to and / or output from the processor (210). The memory (215) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). The volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disc, solid state drive (SSD), and embedded multi media card (eMMC).
[0055] According to one embodiment, the display (110) of the electronic device (101) can output visualized information (e.g., the screen of FIG. 1) to the user. For example, the display (110) can be controlled to visualize information generated by the processor (210) and / or the graphic processing unit (GPU). The display (110) can include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs can include organic LEDs (OLEDs). The display (110) can include a flat panel display (FPD) and / or electronic paper. The embodiment is not limited thereto, and the display (110) can have an at least partially curved shape or a deformable shape (e.g., a flexible display).
[0056] According to one embodiment, the sensor (225) of the electronic device (101) may generate electrical information that may be processed by the processor (210) and / or the memory (215) from non-electronic information related to the electronic device (101). For example, the sensor (225) may include a global positioning system (GPS) sensor for detecting a geographic location of the electronic device (101). In addition to the GPS method, the sensor (225) may generate information indicating the geographic location of the electronic device (101) based on a global navigation satellite system (GNSS) such as, for example, Galileo or Beidou (compass). The information may be stored in the memory (215), processed by the processor (210), and / or transmitted to another electronic device distinct from the electronic device (101) via a communication circuit. The sensor (225) is not limited to those described above, and may include an image sensor, a light sensor, and / or a ToF sensor for detecting electromagnetic waves including light.
[0057] According to one embodiment, the sensor (225) of the electronic device (101) may include a touch sensor (227) for detecting the presence of an external object (e.g., a user's finger) on the display (110). The touch sensor (227) may be referred to as a touch sensor panel (TSP). The touch sensor (227) may detect a touch or proximity (within a predetermined range) of the external object on the display (110). For example, based on the TSP, the electronic device (101) may detect an external object that is in contact with the display (110) or floating on the display (110). In response to detecting the external object, the electronic device (101) may execute a function associated with a specific visual object among the visual objects displayed on the display (110) that corresponds to a location of the external object on the display (110).
[0058] Although not illustrated, in one embodiment, the electronic device (101) may include output means for outputting information in a form other than a visualized form. For example, the electronic device (101) may include a speaker for outputting an acoustic signal. For example, the electronic device (101) may include a motor for providing haptic feedback based on vibration.
[0059] According to one embodiment, one or more instructions (or commands) representing operations and / or actions to be performed on data by the processor (210) may be stored within the memory (215) of the electronic device (101). A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or a software application (or applications). For example, the electronic device (101) and / or the processor (210) may perform at least one of the operations of FIG. 3, FIGS. 8A to 8E, and / or FIG. 10 when a set of a plurality of instructions distributed in the form of an operating system, firmware, driver, and / or application is executed. Hereinafter, the fact that an application is installed in an electronic device (101) may mean that one or more instructions provided in the form of an application are stored in the memory (215) of the electronic device (101), and that the one or more applications are stored in a format executable by the processor (210) of the electronic device (101) (e.g., a file having an extension specified by the operating system of the electronic device (101)).
[0060] Referring to FIG. 2, a launcher application (230) is illustrated as an example of a software application installed in the memory (215) of the electronic device (101). When the launcher application (230) is running, the processor (210) can display a launcher screen on the display (110). The state (191) of FIG. 1 may be an exemplary state in which the launcher screen is displayed based on the execution of the launcher application (230). Through the launcher screen provided by the launcher application (230), the processor (210) can identify an input for executing one or more software applications. The input may include an input indicating a selection of one or more icons included in the launcher screen (e.g., visual objects (131, 132, 133, 141, 142, 143, 144) of FIG. 1), as described above with reference to FIG. 1. The processor (210) can identify the input based on data acquired by the touch sensor (227). The processor (210) can receive data acquired by the touch sensor (227).
[0061] According to one embodiment, the processor (210) of the electronic device (101) may add an icon to the launcher screen, edit an icon displayed on the launcher screen, and / or remove an icon while the launcher application (230) is running. For example, the processor (210) may add an icon to the launcher screen in response to an input identified through the touch sensor (227) for substantially simultaneously executing a plurality of software applications based on multi-window, split-view, multi-view, multi-tasking, and / or split-window. The icon may be displayed to switch to a state and / or mode of the electronic device (101) for providing a plurality of screens, referred to as split-screen, multi-window, multi-tasking, split-window, split-view, and / or multi-view, such as states (194, 195) of FIG. 1. The input may include a gesture of stacking or piling up visual objects (e.g., icons) corresponding to the plurality of software applications. Hereinafter, a gesture for overlapping visual objects may include a gesture (e.g., a drag gesture) for overlapping another visual object beyond a designated area and / or surface area on a specific visual object. Based on the positional relationship and / or direction of the overlapping visual objects by the input, the electronic device (101) may identify the layout of screens provided from the plurality of software applications. In response to a second input indicating selection of the icon added based on the first input, the electronic device (101) may display the screens on the display (110) based on the identified layout. Hereinafter, an example of an operation performed by the electronic device (101) to add the icon is described with reference to FIG. 3.
[0062] FIG. 3 illustrates an example of a flowchart of an electronic device according to one embodiment. The electronic device (101) and / or the processor (210) of FIGS. 1 and 2 may perform the operations described with reference to FIG. 3.
[0063] Referring to FIG. 3, in operation (310), according to one embodiment, a processor of an electronic device may display a plurality of visual objects. For example, the processor may display visual objects associated with one or more software applications on a display (e.g., display (110) of FIGS. 1 and 2). Each of the plurality of visual objects of operation (310) may include an icon representing a software application associated with the visual object. For example, each of the plurality of visual objects of operation (310) may include icons and / or text (e.g., names and / or package names of the one or more software applications) corresponding to one or more software applications. On the display, the processor may display a plurality of visual objects having positions aligned based on a two-dimensional grid. The processor may perform operation (310) while the launcher application (230) of FIG. 2 is running. The state in which the processor performs operation (310) may include state (191) of FIG. 1.
[0064] In one embodiment, a processor may display a list of software applications installed on an electronic device. To visualize at least a portion of the list, the processor may perform operation (310). For example, the plurality of visual objects displayed by operation (310) may correspond to at least a portion of the list of software applications. For example, the plurality of visual objects may each correspond to a plurality of software applications included in the list.
[0065] Referring to FIG. 3, in operation (320), according to one embodiment, a processor of an electronic device may identify a first input indicating selection of a first visual object from among a plurality of visual objects. For example, the processor may identify the first input of operation (320) based on identifying a long-touch gesture for the first visual object (e.g., a gesture for touching the first visual object for a period exceeding 1 second to 1.5 seconds). The long-touch gesture may be identified by the processor receiving sensor data of a touch sensor of the electronic device (e.g., the touch sensor 227 of FIG. 2). The first input of operation (320) is not limited to the long-touch gesture. The processor may identify the first input indicating selection of the first visual object from among a plurality of visual objects based on the touch sensor. The processor that identifies the first input may display the first visual object among the plurality of visual objects displayed based on the operation (310) in a larger size than the other visual objects. The first input may be distinguished from an input for executing an application related to the selected visual object. For example, the input for executing the related application may be a touch on the visual object that lasts less than a predetermined time, while the first input may be a touch on the visual object that lasts more than a predetermined time. The embodiment is not limited thereto.
[0066] Referring to FIG. 3, in operation (330), according to an embodiment, a processor of an electronic device may, in response to a second input indicating movement of a second visual object among a plurality of visual objects, change a position of the second visual object according to the second input. The processor may identify the second input of operation (330) based on identifying a drag gesture for the second visual object. The drag gesture may be identified from sensor data of a touch sensor of the electronic device. The processor that has identified the drag gesture may move the second visual object displayed within the display along the path of the drag gesture. The processor may display the second visual object at a position on the display where the drag gesture is completed based on the completion (or release) of the drag gesture. The first visual object of operation (320) and the second visual object of operation (330) may include visual objects (131, 132, 133) of FIG. 1. The second input may occur after the first input, or may occur while the first input is being generated. For example, the first input may include a touch on the display that lasts longer than a predetermined time, and may then be released. The second input may include a drag gesture that moves a second visual object from a first location to a second location, and may occur after the first input is released. The second input may be initiated within a second predetermined time after the first input is released.
[0067] In another example, a first input may be generated that includes a touch on the display for a duration exceeding a predetermined time. While the touch remains in contact with the display, a second input may be performed. The first input may be released after the second input is initiated, after the second input is released, or substantially simultaneously with the release of the second input.
[0068] Referring to FIG. 3 , in operation (340), a processor of an electronic device according to an embodiment may identify whether a second visual object is at least partially overlapped with a first visual object by a second input. The processor may perform operation (340) while the second visual object is moved by the second input. The processor may perform operation (340) after the second input is released. Identifying whether the second visual object is at least partially overlapped with the first visual object may include comparing coordinates of the second visual object at a second location with coordinates of the first visual object to determine which of the coordinates match. The processor, having identified a second input including a drag gesture, may move the second visual object along a path of the drag gesture. If the terminal point of the drag gesture is contained within the first visual object or is located less than a specified distance from the first visual object, the processor may determine that the second visual object at least partially overlaps the first visual object. If the second visual object at least partially overlaps the first visual object (340-Yes), the processor may perform operation (350) of FIG. 3.
[0069] In one embodiment, the processor can perform operations (310, 320, 330, 340, 350, 360) of FIG. 3 independently of the first input of operation (320). For example, the processor can identify a second input of operation (330) for changing the position of the second visual object without the first input of operation (320). Having identified the second input independently of the first input, the processor can identify, based on operation (340), whether the second visual object at least partially overlaps another visual object. That is, upon termination of the drag gesture over the second visual object, the processor can determine whether the new position of the second visual object at least partially overlaps a visual object displayed on the display. That is, the first visual object may not be required to be explicitly selected prior to termination of the drag gesture over the second visual object. A processor that has identified a second visual object that at least partially overlaps another visual object may perform operation (350) of FIG. 3.
[0070] Referring to FIG. 3 , in operation (350), according to an embodiment, a processor of an electronic device may display a third visual object for displaying screens corresponding to a first visual object and a second visual object based on a multi-window. Based on identifying that the second visual object at least partially overlaps the first visual object as a result of a second input, the processor may display a third visual object for simultaneously providing all screens corresponding to the first visual object and the second visual object. The third visual object may replace one or all of the first visual object and the second visual object on the display. The third visual object of operation (350) may include visual objects (141, 142, 143, 144) of FIG. 1 . Based on operation (350), the processor may display the third visual object within a list of software applications including a plurality of visual objects of operation (310). The processor may display the third visual object based on the size of other visual objects displayed on the display (e.g., the plurality of visual objects of operation (310)). For example, the processor may display the third visual object based on the size of an image (e.g., an icon) within the list representing the software application. The size of the third visual object may be the same as the size of one or more visual objects displayed within operation (310).
[0071] In one embodiment, the third visual object displayed based on the action (350) may include at least a portion of the first visual object of the action (320) and the second visual object of the action (340). If the first visual object includes a first icon for executing a first software application and the second visual object includes a second icon for executing a second software application, the third visual object of the action (350) may include the first icon and the second icon. When displaying a third visual object having the size of the first visual object and the second visual object, the first icon and the second icon that are smaller than the size may be integrated within the third visual object. The positional relationship of the first icon and the second icon within the third visual object may represent a layout of screens to be displayed within the display by an input indicating selection of the third visual object. For example, the processor may display a third visual object including a first icon and a second icon, arranged based on the positions of the second visual object overlapping the first visual object, in response to a second input.
[0072] Referring to FIG. 3, in operation (360), according to one embodiment, a processor of an electronic device may, in response to a third input indicating selection of a third visual object, display screens based on a layout corresponding to a position (or direction) of a second visual object overlapping a first visual object by the second input. That is, in response to a third input for selecting a third visual object, applications related to the first and second visual objects may be executed, an application screen for each of the executed applications may be displayed, and the application screens of the applications may be displayed according to the layout indicated by the third visual object. The third input may include a tap gesture of tapping the third visual object displayed on the display. The tap gesture may be identified based on sensor data of a touch sensor (e.g., touch sensor 227 of FIG. 2) connected to the processor.
[0073] In one embodiment, a processor that identifies a third input of operation (360) may execute software applications corresponding to each of the first visual object and the second visual object based on multi-tasking. The processor may arrange screens provided by the executed software applications based on the layout of operation (360). For example, screens simultaneously displayed in the display by operation (360) may have a positional relationship and / or layout represented by icons included in the third visual object (e.g., icons corresponding to each of the first visual object and the second visual object). The layout of operation (360) may be related to a position of a second visual object that at least partially overlaps the first visual object after a drag movement of the second input. The layout of operation (360) may be identified among PBP and / or PIP.
[0074] In one embodiment, if the second visual object does not overlap the first visual object (340-No), the processor may change the position of the second visual object according to the second input. Referring to FIG. 2, if the second visual object moved by the second input does not overlap the first visual object, a third visual object that provides all screens corresponding to the first and second visual objects may not be displayed. While the position of the second visual object is moved to a position corresponding to the end point of the second input, the first visual object may maintain its original position.
[0075] Although the tap gesture and / or the drag gesture have been described as examples of the first to third inputs, the embodiment is not limited thereto. In one embodiment, the electronic device can identify the first to third inputs described with reference to FIG. 3 based on at least one of a hand gesture, a direction of gaze, and / or a utterance based on a natural language sentence. According to one embodiment, an electronic device (e.g., the electronic device (101-3) of FIG. 2) that performs the operation of FIG. 3 based on a hand gesture is described with reference to FIGS. 12A to 12C.
[0076] Hereinafter, screens displayed by an electronic device performing the operation of FIG. 3 are described in an exemplary manner with reference to FIGS. 4a to 4c.
[0077] FIGS. 4A to 4C illustrate an example of an operation of an electronic device (101) that displays a visual object for providing multiple screens. The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operations of the electronic device (101) described with reference to FIGS. 4A to 4C. The operations of the electronic device (101) described with reference to FIGS. 4A to 4C may be related to at least one of the operations of FIG. 3.
[0078] Referring to FIGS. 4A to 4C, states (401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411) of an electronic device (101) are illustrated, which are distinguished according to the screen displayed on the display (110). The electronic device (101) can display the screens illustrated in FIGS. 4A to 4C in response to the execution of the launcher application (230) of FIG. 2. The electronic device (101) of FIGS. 4A to 4C may correspond to the electronic device (101-1) of FIG. 2.
[0079] Referring to FIG. 4A, within a state (401), the electronic device (101) may display a screen (e.g., a launcher screen and / or a home screen) for displaying a list of software applications within the display (110). Within the state (401), the electronic device (101) may display visual objects (131, 132) having a width (w1) and a height (h1). Each of the visual objects (131, 132) may correspond to a software application installed in the electronic device (101). Each of the visual objects (131, 132) may include an icon representing a software application corresponding to the visual object. Referring to FIGS. 4A to 4C, an operation of the electronic device (101) based on a visual object (131) corresponding to an email application and a visual object (132) corresponding to an alarm application is described, but the embodiment is not limited thereto.
[0080] Within state (401) of FIG. 4A, the electronic device (101) may identify a first input indicating selection of a visual object (131). For example, the electronic device (101) may identify the first input in response to detecting contact of an external object (e.g., a fingertip) on the display at a location (p1) where the visual object (131) is displayed on the display, the contact lasting for a period exceeding a specified period of time. Upon identifying the first input based on the location (p1) within state (401), the electronic device (101) may switch to state (402).
[0081] Referring to FIG. 4A, in response to a first input indicating selection of a visual object (131) based on a location (p1) within a state (402), the electronic device (101) may enlarge a visual object (131) among a plurality of visual objects. For example, the electronic device (101) may display a visual object (131) having a width (w2) exceeding a width (w1) and a height (h2) exceeding a height (h1). During a transition from state (401) to state (402), the visual object (131) may be enlarged based on a center point of the visual object (131). The electronic device (101) may enlarge the visual object (131) to provide a guide or indication that the visual object (131) will be used to create a multi-window.
[0082] In one embodiment, the visual object (131) may remain in an enlarged state while an external object is in contact with the visual object (131) for the first input. For example, when the external object is separated from the display (110), the electronic device (101) may reduce the width (w2) and the height (h2) of the visual object (131) to the width (w1) and the height (h1) of the visual object (131) within the state (401). The embodiment is not limited thereto. For example, after the visual object (131) is enlarged based on the first input, the electronic device (101) may continue to display the visual object (131) according to the width (w2) and the height (h2) even after the first input is released.
[0083] Within state (402) of FIG. 4A, the electronic device (101) may identify a second input indicating movement of a different visual object than the enlarged visual object (131). For example, the electronic device (101) may identify the second input based on an external object contacting a location (p2) within a portion of the display (110) on which the visual object (132) is displayed. The second input may include a gesture (e.g., a drag gesture) in which the external object contacting the location (p2) is moved out of the portion of the display (110) on which the visual object (132) is displayed, while in contact with the display (110). In one embodiment, the electronic device (101) may identify the second input for a period of time during which the first input is maintained. The embodiment is not limited thereto, and the electronic device (101) may identify the second input independently of whether the first input is maintained after the first input. The operation of the electronic device (101) for the second input may be performed while the first input is maintained, or may be performed after the first input is completed. The first input may be maintained while contact with an external object on the visual object (131) occurs. The first input may be completed when contact with the external object on the visual object (131) ceases.
[0084] Referring to FIG. 4A, exemplary drag gestures are illustrated, starting at location (p2) and moving along different paths (421, 422, 423). An electronic device (101) that identifies a second input based on the drag gestures may move a visual object (132) within the display (110) based on any one of the paths (421, 422, 423). For example, the electronic device (101) may display an animation such as a visual object (132) following an external object being dragged on the display (110). According to one embodiment, the electronic device (101) may display a visual object (or visual cue or container) (e.g., visual objects (432, 442, 452) of FIG. 4A) to guide the area based on identifying that the visual object (132) moved according to the second input has moved into an area formed within the display (110) based on the enlarged visual object (131).
[0085] Within the state (402) of FIG. 4A, the electronic device (101) that identifies a second input for moving the visual object (132) to a location (q1) adjacent to the visual object (131) along the path (421) may switch to the state (403). Within the state (403), the electronic device (101) may display a visual object (432) for adjusting the layout of screens corresponding to the first visual object and the second visual object corresponding to the first input and the second input, respectively. Referring to FIG. 4A, the visual object (432) having a rectangular shape with rounded corners is illustrated, but the embodiment is not limited thereto. The ratio between the width and the height of the visual object (432) displayed on the display (110) (e.g., aspect ratio) may correspond to the ratio between the width and the height of the display (110) and / or the active area. For example, the processor may display a visual object (432) representing a reduced display (110).
[0086] In the exemplary state (403) of FIG. 4A, the electronic device (101) that has identified a visual object (132) that has moved to a position (q1) adjacent to the visual object (131) along the -y direction of the display (110) may display the visual object (132) below the visual object (131) along the -y direction within the visual object (432). The visual objects (131, 132) displayed side by side within the visual object (432) may represent the layout of screens to be provided through the visual object (434) that will be generated by the second input. In the state (403), the electronic device (101) that has identified a visual object (132) that has moved to a different position than the position (q1) by the second input may change the positional relationship of the visual objects (131, 132) within the visual object (132) based on the different position of the visual object (132).
[0087] Referring to FIG. 4A, in a state (403) where a visual object (132) is moved into a visual object (432), the electronic device (101) may, based on identifying a release of a second input for moving the visual object (132) into a location of the visual object (432), display a visual object (434) for providing a plurality of screens according to a layout corresponding to the location of the visual object (132) within the visual object (432). The visual object (434) may be displayed when the second input is released. For example, the electronic device (101) that identifies a second input released at a location (q1) may display a visual object (434) related to a location and / or direction of the visual object (132) disposed within the visual object (432) based on the location (q1). The embodiment is not limited thereto, and after identifying the release of the second input, the electronic device (101) may display a visual object (434) in response to a tap gesture on a portion of the display (110) that is not occupied by an executable object. The portion of the display (110) that is not occupied by an executable object may be referred to as a blank area and / or an idle area. An input indicating a selection of an idle area may be included in an input for completing (or aborting) selecting multiple screens.
[0088] Referring to FIG. 4A, a visual object (434) displayed based on the release of the second input within a state (403) may include images (436, 438) corresponding to each of the visual objects (131, 132). Based on the positional relationship of the visual objects (131, 132) arranged along the -y direction, the electronic device (101) may display a visual object (434) including images (436, 438) arranged along the -y direction. The visual object (434) including a line positioned between the images (436, 438) is illustrated as an example, but the embodiment is not limited thereto. Each of the images (436, 438) within the visual object (434) may be icons representing software applications corresponding to the visual objects (131, 132). The electronic device (101) can display a visual object (434) having a width (w1) and a height (h1) of the visual object (131) in the state (401). Based on the release of the second input, the electronic device (101) can stop displaying at least one of the visual objects (131, 132, 432) within the display (110). Based on the release of the second input, the electronic device (101) can display the visual object (434) at a location of the visual object (131) corresponding to the first input.
[0089] Within the state (402) of FIG. 4A, the electronic device (101) that identifies a second input for moving the visual object (132) to a location (q2) adjacent to the visual object (131) along the path (422) may switch to the state (404). Within the state (404), the electronic device (101) may display a visual object (442) having a rectangular shape that wraps all of the visual objects (131, 132). An aspect ratio of the visual object (442) may match an aspect ratio of the display (110). Within the visual object (442), the electronic device (101) may display the visual objects (131, 132) based on the positional relationship between the location (q2) and the visual object (131). When the visual object (132) is moved in the +y direction of the visual object (131) along the path (422), the electronic device (101) can display the visual object (132) on top of the visual object (131) along the +y direction within the visual object (442).
[0090] In state (404) of FIG. 4A, the electronic device (101) that has identified the release of the second input for movement of the visual object (132) may display a visual object (444) for simultaneously providing screens corresponding to the visual objects (131, 132). In response to the release of the second input, the electronic device (101) may display a visual object (444) having a width (w1) and a height (h1) within the launcher screen. In one embodiment, the electronic device (101) may replace the visual object (131) within the launcher screen with the visual object (444). Based on the release of the second input, the electronic device (101) may hide at least one of the visual objects (131, 132, 442) from the display (110) and display the visual object (444). Within the visual object (444), the electronic device (101) can arrange images (446, 448) corresponding to each of the visual objects (131, 132) based on the positional relationship of the visual objects (131, 132) at the time the second input is released. The arrangement of the images (446, 448) within the visual object (444) can represent the arrangement of screens to be displayed within the display (110) based on the input related to the visual object (444).
[0091] Within the state (402) of FIG. 4a, the electronic device (101) that identifies a second input for moving the visual object (132) to a location (q3) adjacent to the visual object (131) along the path (423) may switch to the state (405). Within the state (405), the electronic device (101) may display a visual object (452) that includes all of the visual objects (131, 132). When the visual object (132) is moved in the +x direction of the visual object (131) along the path (423), the electronic device (101) may display the visual object (452) in the form of the display (110) in the landscape mode. For example, the ratio of the width and height of the visual object (452) may match the ratio of the height and width of the display (110) in the state (405). Within the visual object (452), the electronic device (101) can display the visual object (132) to the left of the visual object (131) along the +x direction.
[0092] In the state (405) of FIG. 4A, when a second input for movement of the visual object (132) is released at a location (q2) within the visual object (452), the electronic device (101) may display a visual object (454) to simultaneously provide screens corresponding to the visual objects (131, 132) within the visual object (452). The width (w1) and the height (h1) of the visual object (454) displayed by the electronic device (101) may match the width (w1) and the height (h1) of the visual object (131) within the state (401). Based on the release of the second input, the electronic device (101) may stop displaying at least one of the visual objects (131, 132, 452) and display the visual object (454) at the location where the visual object (131) was displayed. The electronic device (101) can display a visual object (454) including images (456, 458) related to visual objects (131, 132). Within the visual object (454), the arrangement of the images (456, 458) can correspond to the arrangement of the visual objects (131, 132) within the visual object (452) at the time when the second input is released.
[0093] Referring to FIG. 4A, in one embodiment where the height of the display (110) is longer than the width, the electronic device (101) can display multiple screens based on a portrait mode or a landscape mode. Based on an input for visual objects (434, 444) including images (e.g., images (436, 438, 446, 448)) arranged along a vertical direction (e.g., the direction of the y-axis) of the display (110), the electronic device (101) can display multiple screens in the portrait mode. Based on an input for visual objects (454) including images (456, 458) arranged along a direction of a second edge shorter than a first edge of the display (110), the electronic device (101) can display multiple screens in the landscape mode.
[0094] Referring to states (403, 404, 405) of FIG. 4A, the layout of visual objects (434, 444, 454) and / or screens may depend on the position and / or arrangement of visual objects (131, 132) within the visual cue. Referring to FIG. 4B, an exemplary state (406) is illustrated in which visual objects (434, 454) of FIG. 4A are displayed within the display (110). Referring to FIG. 4B, the electronic device (101) may display within the display (110) visual objects (434, 454) for executing multiple software applications at once, together with a visual object (e.g., visual objects (131, 132)) for executing one software application.
[0095] Within state (406) of FIG. 4B, in response to an input indicating selection of a visual object (434), the electronic device (101) may switch to state (407). Within state (407), the electronic device (101) may display screens provided from software applications arranged along the vertical direction of the display (110) and represented by the visual object (434). For example, the electronic device (101) may display a second screen in a second area (152) below a first area (151) of the display (110) in which the first screen is displayed, along the -y direction within the display (110). The electronic device (101) may display handles (161, 162) for controlling the position and / or size of the first screen and the second screen, respectively, in the areas (151, 152). Through the boundary line (163) between the areas (151, 152), the electronic device (101) can identify an input for adjusting the height, size and / or area of the areas (151, 152).
[0096] Within state (406) of FIG. 4B, based on an input indicating selection of a visual object (454), the electronic device (101) may switch to state (408). Within state (408), the electronic device (101) may display screens provided from software applications corresponding to the visual object (454) based on a landscape mode. For example, the electronic device (101) may display a first screen in a first area (481) of the display (110), and display a second screen in a second area (482) to the right of the first area (481) along the +y direction. In each of the areas (481, 482), the electronic device (101) may display handles (483, 484) for controlling the position and / or size of the first screen and the second screen.
[0097] Referring to FIG. 4B, based on an input indicating selection of any one of the visual objects (434, 454), the electronic device (101) can display screens related to the visual object corresponding to the input on each of portions (e.g., portions (151, 152, 481, 482)) of an active area of the display (110) divided along the direction of the edge of the display (110). In response to a first input, in a state (402) in which the visual object (131) is enlarged, based on a second input for moving the visual object (132) to a position adjacent to the visual object (131) (e.g., positions (q1, q2, q3)), the electronic device (101) can display a visual object (e.g., visual objects (434, 444, 454)) for simultaneously displaying screens according to the PBP. Within the state (402), when the visual object (132) moves to a position at least partially overlapping with the visual object (131) according to the second input, the electronic device (101) may display the visual object for simultaneously displaying screens according to PIP and / or PBP. The operation of the electronic device (101) based on the visual object (132) moving to a position at least partially overlapping with the visual object (131) is described with reference to FIGS. 6A to 6B and / or 7.
[0098] Referring to FIG. 4C, states (409, 410, 411) of the electronic device (101) are illustrated, which are distinguished according to the screen displayed on the display (110) within the state (409) included in the landscape mode. Within the state (409), the electronic device (101) may display a screen (e.g., a launcher screen and / or a home screen) for displaying a list of software applications within the display (110). Within the screen, the electronic device (101) may display visual objects (131, 132, 133) having a width (w1) and a height (h1). The visual objects (131, 132, 133) of FIG. 4C may correspond to the visual objects (131, 132, 133) of FIG. 4A. For example, the electronic device (101) of FIG. 4a may be switched to the state (409) of FIG. 4c based on identifying a motion in which the display (110) and / or the electronic device (101) rotates 90º relative to the direction of the normal of the display (110).
[0099] Within state (409) of FIG. 4c, the electronic device (101) can identify a first input indicating selection of a visual object (131). For example, the electronic device (101) can identify the first input based on an external object coming into contact with the display (110) at a location (p1) where the visual object (131) is displayed for at least a specified period of time (or a predetermined length of time). The electronic device (101) that has identified the first input can switch from state (409) of FIG. 4c to state (410).
[0100] Within the state (410) of FIG. 4c, the electronic device (101) that has identified the first input indicating selection of the visual object (131) may enlarge the visual object (131) corresponding to the first input. As the visual object (131) is enlarged, the width (w2) and the height (h2) of the visual object (131) displayed within the state (410) of FIG. 4c may be greater than the width (w1) and the height (h1) of another visual object (e.g., the visual object (132)). By displaying the visual object (131) with a width (w2) and a height (h2) that are greater than the width (w1) and the height (h1), respectively, the electronic device (101) may provide an indication to the user that the visual object (131) is in a state in which it can receive additional input for creating a multi-window visual object.
[0101] In one embodiment, the display of the visual object (131) according to the width (w2) and the height (h2) may be maintained while the wearable device (101) receives the first input. For example, based on the release, termination, and / or interruption of the first input, the wearable device (101) may reduce the width (w2) and the height (h2) of the visual object (131) to the width (w1) and the height (h1). The embodiment is not limited thereto, and even after the first input is released, the wearable device (101) may maintain the width (w2) and the height (h2) of the visual object (131) for a specified period of time.
[0102] Referring to FIG. 4c, within the enlarged state (410) of the visual object (131), the electronic device (101) can identify a second input indicating movement of a different visual object than the visual object (131). For example, in response to an external object contacting a location (p2) within a portion of the display (110) where the visual object (132) is displayed, the electronic device (101) can identify the second input. For example, the electronic device (101) can identify a second input that moves from the location (p2) along the path (491) to a location (q3) adjacent to the visual object (131). Based on the second input, the electronic device (101) can change the location of the visual object (132) on the display (110). Based on identifying a visual object (132) moved to a position (q3) within an area formed within a display (110) based on a visual object (131), the electronic device (101) can display a visual object (452) for guiding the area. Referring to FIG. 4C, a state (411) of the electronic device (101) displaying a visual object (452) based on the second input is illustrated.
[0103] Referring to FIG. 4c, the electronic device (101) that has identified a visual object (132) that has moved to a position (q3) adjacent to the visual object (131) along the -y direction may display a visual object (452) having a shape and / or size that includes all of the visual objects (131, 132). The visual object (452) may have a shape and / or aspect ratio of the display (110) in landscape mode. In the state (411) of FIG. 4c, when a second input for moving the visual object (132) is released within the visual object (452), the electronic device (101) may display a visual object (454) for providing screens corresponding to the visual objects (131, 132) grouped by the visual object (452) based on a multi-window. The visual object (454) and the images (456, 458) included in the visual object (454) of FIG. 4c may correspond to the visual object (454) and the images (456, 458) of FIG. 4a.
[0104] After the state (411) of FIG. 4c, the electronic device (101) displaying the visual object (454) on the display (110) may, in response to an input indicating selection of the visual object (454), execute all software applications (e.g., software applications corresponding to each of the images (456, 458) within the visual object (454)) displayed by the visual object (454) based on the landscape mode. Based on the execution of the software applications, the electronic device (101) may display screens provided by the software applications based on the landscape mode. For example, within the landscape mode, the electronic device (101) may display the screens on different parts of the display (110). For example, the electronic device (101) may display a second screen on the left side of a first screen among the screens within the display (110) in the landscape mode.
[0105] FIGS. 5A to 5D illustrate an example of an operation of an electronic device (101-2) that displays a visual object for providing multiple screens. The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operations of the electronic device (101-2) described with reference to FIGS. 5A to 5D. The operations of the electronic device (101-2) described with reference to FIGS. 5A to 5D may be related to at least one of the operations of FIG. 3.
[0106] Referring to FIGS. 5A to 5D , states (501, 502, 503, 504, 505, 506, 507) of the electronic device (101-2) distinguished according to the screen shown on the display (110) are illustrated. The electronic device (101-2) of FIGS. 5A to 5D may correspond to the electronic device (101-2) of FIG. 2 . For example, the electronic device (101-2) may include a first housing (591), a second housing (592), and a hinge assembly (593) for rotatably coupling the first housing (591) and the second housing (592). The display (110) of the electronic device (101-2) may be disposed on the first housing (591), the hinge assembly (593), and the second housing (592). The display (110) of the electronic device (101-2) may be referred to as a flexible display.
[0107] Referring to FIG. 5A, within a state (501) of displaying a list of software applications, the electronic device (101-2) may display visual objects (511, 512, 513) for executing different software applications. The visual objects (511, 512, 513) may each have the same width (w1) and height (h1). Within the state (501) of displaying the visual objects (511, 512, 513) based on the launcher screen, the screen displayed by the electronic device (101-2) may be divided into different areas (112, 114, 118). Within an area (114) referred to as a workspace area, a launcher area, and / or a home area, the electronic device (101-2) may display a widget (514) together with icons representing software applications.
[0108] Within state (501) of FIG. 5A, the electronic device (101-2) may identify a first input indicating selection of a visual object (511). The first input may include a long-touch (or long-press) gesture for a location (p1) of the display (110) where the visual object (511) is displayed. The electronic device (101-2) that has identified the first input may switch to state (502). While switching from state (501) to state (502), the electronic device (101-2) may enlarge the visual object (511) corresponding to the first input. For example, the width (w2) and the height (h2) of the visual object (511) within state (502) may be greater than the width (w1) and the height (h1) of the visual object (511) within state (501). By enlarging the visual object (511), the electronic device (101-2) can guide that it has switched to a state (502) for adding a new visual object to provide a plurality of screens including a screen corresponding to the visual object (511) based on PBP and / or PIP. By enlarging the visual object (511), the electronic device (101-2) can more easily receive additional input for combining the screens.
[0109] Referring to FIG. 5B, in a state (502) where the visual object (511) is enlarged, the electronic device (101-2) can identify a second input indicating movement of a different visual object than the visual object (511). For example, the electronic device (101-2) can identify the second input based on a drag gesture initiated from a location (p2) on the display (110) where the visual object (512) is displayed. When the visual object (512) is moved to an area (520) adjacent to the visual object (511) based on the second input, the electronic device (101-2) can display a visual object (e.g., visual objects (525, 526, 527)) for providing a combination of screens corresponding to the visual objects (511, 512). The layout of the screens to be included in the above combination can be determined based on the position of the visual object (512) moved into the area (520).
[0110] Referring to FIG. 5b, the electronic device (101-2) that identifies a visual object (512) moving into an area (520) based on a second input may display a visual object (524) for guiding the area (520). The ratio between the width and the height (e.g., aspect ratio) of the visual object (524) may match the ratio between the width and the height of the display (110). Referring to FIG. 5b, different examples of the second input are illustrated. Referring to FIG. 5b, the electronic device (101-2) that identifies a visual object (512) moving from a location (p2) along a path (521) to a location (q1) within the area (520) may display a visual object (524-1) for guiding the area (520). The electronic device (101-2) can display a visual object (512) within a visual object (524-1) based on a location (q1) for the visual object (511). The electronic device (101-2), which has identified a second input released at the location (q1), can display a visual object (525) including images (528, 529) that are associated with each of the visual objects (511, 512) and arranged based on the location (q1) for the visual object (511). The electronic device (101-2) can display a visual object (525) within the launcher screen that has a width (w1) and a height (h1) of another visual object that are different from the visual objects (511, 512). In response to an input indicating a selection of a visual object (525), the electronic device (101-2) can display a plurality of screens within the display (110) having a layout corresponding to the arrangement of images (528, 529) within the visual object (525).
[0111] Referring to FIG. 5B, the electronic device (101-2), which has identified a visual object (512) moving from a location (p2) along a path (522) to a location (q2) within an area (520), may display a visual object (524-2) guiding the area (520). The electronic device (101-2) may determine the locations of the visual objects (511, 512) within the visual object (524-2) based on the location (q2) for the visual object (511). When the visual object (512) moves to a location (q2) adjacent to the visual object (511) along the horizontal direction of the display (110) (e.g., the direction of the second edge longer than the first edge of the display (110), the electronic device (101-2) may display the visual objects (511, 512) arranged along the horizontal direction within the visual object (524-2). In response to a second input released at location (q2), the electronic device (101-2) may display a visual object (526) that is arranged based on location (q2) and includes images (528, 529) corresponding to each of the visual objects (511, 512). The visual object (526) may be displayed at a location within the display (110) where the visual object (511) was displayed. The embodiment is not limited thereto, and the electronic device (101-2) may display the visual object (526) on a portion of the display (110) that is not occupied by an executable object (e.g., an icon for executing a software application such as the visual object (513)) within a launcher screen.
[0112] Referring to FIG. 5B, the electronic device (101-2), which has identified a visual object (512) moving from a location (p2) along a path (523) to a location (q3) within an area (520), may display a visual object (524-3) guiding the area (520). While displaying the visual object (524-3) having the aspect ratio of the display (110), the electronic device (101-2) may display visual objects (511, 512) arranged based on the location (q3) within the visual object (524-3). When a second input for movement of the visual object (512) is released at the location (q3), the electronic device (101-2) may add a visual object (527) for displaying screens having a layout corresponding to the location (q3) for the visual object (511) within the launcher screen. A visual object (527) may include images (528, 529) associated with each of the visual objects (511, 512). Within the visual object (527), the layout of the images (528, 529) may correspond to the layout of screens to be displayed within the display (110) based on the visual object (527).
[0113] Referring to FIG. 5b, while visual objects (511, 512) are arranged within a visual object (524) representing an area (520), the electronic device (101-2) may generate or add a visual object (e.g., visual objects (525, 526, 527)) to provide screens corresponding to the visual objects (511, 512) within the visual object (524) based on an input indicating a selection of a blank area (or idle area) of the display (110).
[0114] Although the operation of the electronic device (101-2) for generating visual objects (525, 526, 527) for providing two screens simultaneously has been described with reference to FIGS. 5A and 5B, the embodiment is not limited thereto. Referring to FIG. 5C, in a state (503) in which the visual object (524-1) of FIG. 5B is displayed, the electronic device (101-2) may identify a third input for moving a visual object (513) located outside the visual object (524-1) into the visual object (524-1). According to one embodiment, the electronic device (101-2) may determine a layout for a plurality of screens based on the positional relationship between the visual objects (511, 512) included in the visual object (524-1) and the visual object (513) moved into the visual object (524-1) by the third input.
[0115] Referring to FIG. 5c, in a state (503) of displaying a visual object (524-1) including visual objects (511, 512) arranged along a vertical direction of the display (110) (e.g., the direction of the first edge among the first edge of the display (110) or a second edge shorter than the first edge), the electronic device (101-2) may display a visual object (524-4) including visual objects (511, 512, 513) based on a third input for moving the visual object (513) to a position (q4) within the visual object (524-1) along a path (531). Within the visual object (524-4), the visual objects (511, 512) may be arranged based on the position (q4) of the visual object (513) moved by the third input. Based on the visual object (513) moved to the location (q4) within the visual object (524-4) based on the third input, the electronic device (101-2) can display a visual object (533) to provide three screens corresponding to the visual objects (511, 512, 513). The visual object (533) can include images (535, 536, 537) corresponding to the visual objects (511, 512, 513) included within the visual object (524-4). The positional relationship of the images (535, 536, 537) within the visual object (533) can correspond to the positional relationship of the visual objects (511, 512, 513) within the visual object (524-4).
[0116] Referring to FIG. 5c, within a state (503), based on a third input indicating that a visual object (513) is to be moved to a location (q5) within a visual object (524-1) along a path (532), the electronic device (101-2) may display an object (524-5) including visual objects (511, 512, 513). The electronic device (101-2) may display a visual object (524-5) including visual objects (511, 512, 513) arranged based on the location (q5). After displaying the visual object (524-5), based on an input indicating selection of a blank area (or idle area) of the display (110), the electronic device (101-2) may display a visual object (534) to provide a plurality of screens having a layout corresponding to the arrangement of visual objects (511, 512, 513) within the visual object (524-5). Within the visual object (534), the electronic device (101-2) may display images (535, 536, 537) representing each of the visual objects (511, 512, 513). The arrangement of the images (535, 536, 537) within the visual object (534) may match the arrangement of the visual objects (511, 512, 513) within the visual object (524-5). The arrangement of images (535, 536, 537) within a visual object (534) may correspond to the layout of screens to be displayed within the display (110) by an input indicating a selection of the visual object (534).
[0117] Referring to FIG. 5d, an exemplary state (504) of an electronic device (101-2) displaying visual objects (526, 527, 533) described with reference to FIGS. 5a to 5c is illustrated. The visual objects (526, 527, 533) may have the same size as other visual objects (or icons) within a launcher screen displayed via the display (110).
[0118] Within state (504) of FIG. 5d, in response to an input indicating selection of a visual object (533), the electronic device (101-2) may switch to state (505). Within state (505), the electronic device (101-2) may concurrently execute software applications corresponding to the visual object (533). Based on the concurrently executed software applications, the electronic device (101-2) may display screens provided from the software applications within the display (110). Within state (505), the layout of the screens displayed within the display (110) may correspond to the arrangement of images and / or icons included in the visual object (533). For example, based on an input for a visual object (533) including icons (or images) representing a weather application, an alarm application, and an email application, respectively, the electronic device (101-2) may display screens provided from the weather application, the alarm application, and the email application within the display (110). Within the display (110), the positional relationship of the first area (551) where the third screen provided from the alarm application is displayed, the second area (552) where the second screen provided from the email application is displayed, and the third area (553) where the first screen provided from the alarm application is displayed may be matched to the positional relationship within the visual objects (533) of the icon representing the alarm application, the icon representing the email application, and the icon representing the alarm application. In each of the areas (551, 552, 553), the electronic device (101-2) may display handles (554, 555, 556) for adjusting the position and / or size of the areas (551, 552, 553).
[0119] In state (504) of FIG. 5d, in response to an input indicating selection of a visual object (527), the electronic device (101-2) may switch to state (506). When icons representing each of an email application and an alarm application are arranged along the vertical direction of the display (110) within the visual object (527), the electronic device (101-2) may display screens provided from the email application and the alarm application side by side along the vertical direction within the display (110). For example, with respect to a first area (561) in which a first screen provided from the email application is displayed, a second area (562) in which a second screen provided from the alarm application is displayed may be positioned below the first area (561) along the vertical direction within the display (110). In each of the areas (561, 562), the electronic device (101-2) may display handles (563, 564) for adjusting the position and / or size of the areas (561, 562).
[0120] Within the state (504) of FIG. 5d, in response to an input indicating selection of a visual object (526), the electronic device (101-2) may switch to a state (507). Based on the input for the visual object (526) in which icons corresponding to different software applications (e.g., an email application and an alarm application) are arranged along the horizontal direction of the display (110), the electronic device (101-2) may switch to a state (507) in which screens provided from the software applications are arranged along the horizontal direction within the display (110). Referring to the state (504) of FIG. 5d, a screen provided from the email application may be displayed in a second area (572) of the display (110) that is distinct from a first area (571) occupied by the alarm application. Within a state (507) of displaying screens provided from different software applications through non-overlapping areas (571, 572), the electronic device (101-2) can display handles (572, 574) for adjusting the position and / or size of the areas (571, 572).
[0121] As described above, according to one embodiment, the electronic device (101-2) can create and / or add visual objects (e.g., visual objects (525, 526, 527, 533, 534)) to provide two or more screens simultaneously. The electronic device (101-2) can adjust the layout of the screens based on the position, direction, and / or path of a drag gesture for combining the visual objects (e.g., visual objects (511, 512, 513)) corresponding to the screens. In one embodiment, the electronic device (101-2) can execute a function for organizing software applications based on the launcher screen. For example, the electronic device (101-2) can support a function for creating, editing, and / or removing a folder capable of accommodating one or more software applications. Hereinafter, with reference to FIGS. 6A and 6B, the operation of an electronic device (101) for generating either a visual object for a folder or a visual object for a multi-window by using the position and / or direction in which the visual objects overlap is described.
[0122] FIGS. 6A and 6B illustrate an example of an operation of an electronic device (101) based on the positions of overlapping visual objects (131, 132). The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIGS. 6A and 6B. The operation of the electronic device (101) described with reference to FIGS. 6A and 6B may be related to at least one of the operations of FIG. 3.
[0123] Referring to FIGS. 6A and 6B , states (403, 601, 602, 603) of the electronic device (101) distinguished according to the screen shown on the display (110) are illustrated. The state (403) of FIGS. 6A and 6B may correspond to the state (403) of FIG. 4A . Referring to FIG. 6A , a drag gesture is exemplarily illustrated for moving a visual object (132) at a position (q1) adjacent to a visual object (131) to a position (q2) on the visual object (131) along a path (620). According to one embodiment, the electronic device (101) may determine the layout of screens corresponding to the visual objects (131, 132) based on the visual object (132) moved to a position (q2) overlapping the visual object (131) by the drag gesture. The electronic device (101) can generate a visual object (630) for providing the above screens based on the determined layout.
[0124] Referring to FIG. 6a, an electronic device (101) that has identified a visual object (132) that is moved to a position (q2) overlapping with a visual object (131) can compare the areas (611, 612, 613, 614, 616, 616, 617, 618, 619) formed on the visual object (131) and the position (q2) to determine the layout of screens corresponding to the visual object (131, 132). Referring to FIG. 6a, areas (611, 612, 613, 614, 616, 616, 617, 618, 619) formed on a visual object (131) having a rectangular shape with rounded corners are illustrated as an example. Each of the regions (612, 614, 616, 618) may include each of the vertices (e.g., rounded corners) of the visual object (131). Each of the regions (611, 613, 615, 617) may include each of the edges connecting the vertices. The region (619) may correspond to an inner region of the visual object (131) defined by the vertices and the edges. For example, the regions (611, 612, 613, 614, 615, 616, 617, 618) excluding the region (619) may be formed along the boundary line of the visual object (131).
[0125] Referring to FIG. 6A, an electronic device (101) that identifies a visual object (132) that has moved to one of the areas (611, 612, 613, 614, 615, 616, 617, 618) formed along the boundary line of a visual object (131) may display a visual object (e.g., a visual object (630)) for providing screens related to the visual objects (131, 132) based on a layout corresponding to the positional relationship of the visual objects (131, 132). For example, an electronic device (101) that identifies a visual object (132) that has moved to areas (611, 613, 615, 617) corresponding to edges of the visual object (131) may display a visual object for providing screens having a layout of a PBP. For example, if the visual object (132) and / or the position (q2) to which the visual object (132) has moved overlaps with or is included in an area (e.g., area (611)) corresponding to an edge connecting the vertices of the visual object (131), the electronic device (101) may generate a visual object for displaying screens related to the visual objects (131, 132) side by side based on the positional relationship of the area with respect to the visual object (131).
[0126] In one embodiment, an electronic device (101) that identifies a visual object (132) that overlaps areas (612, 614, 616, 618) corresponding to vertices of a visual object (131) may display a visual object for providing screens having a layout of a PIP. In the state (403) of FIG. 6A, an electronic device (101) that identifies an input for moving a visual object (132) to a location (q2) within an area (618) along a path (620) may generate a visual object (630) for overlapping a second screen corresponding to the visual object (132) at a location adjacent to a vertex corresponding to the area (618) within a first screen corresponding to the visual object (131). The electronic device (101) that has identified the visual object (132) moved to the location (q2) can change the color of the area (618) including the location (q2) to a different color from the colors of other areas (611, 612, 613, 614, 615, 616, 617, 619). In the state (601) after identifying the input, the electronic device (101) can display the generated visual object (630) within the display (110). For example, based on identifying that the visual object (132) is overlapped on a vertex of the visual object (131) (e.g., a vertex corresponding to the area (618)) according to an input including a drag gesture performed along a path (620), the electronic device (101) can display the visual object (630) for displaying the second screen as an overlap on the first screen.
[0127] Referring to FIG. 6A, the visual object (630) may include a first image corresponding to the first screen. The visual object (630) may include a second image having a smaller size than the first image and corresponding to the second screen. The positional relationship between the first image and the second image may correspond to the positional relationship between the visual object (131) and the area (618). For example, the second image may be positioned adjacent to a vertex of the first image corresponding to a vertex of the visual object (131) included in the area (618). The embodiment is not limited thereto, and the second image may be overlapped on a vertex of the first image corresponding to a vertex of the visual object (131) included in the area (618) within the visual object (630). In response to an input indicating selection of a visual object (630), the electronic device (101) can display a screen associated with the second image as an overlay on the screen associated with the first image, at a location adjacent to a vertex of the display (110) corresponding to the vertex of the first image on which the second image is overlaid.
[0128] Within the state (601) of FIG. 6A, in response to an input indicating selection of a visual object (630), the electronic device (101) may switch to the state (602). Within the state (602) for displaying a plurality of screens related to the visual object (630), the electronic device (101) may display a first screen among the plurality of screens based on the size of the active area of the display (110). The first screen may be provided from a software application corresponding to the visual object (131). The electronic device (101) may display a second screen among the plurality of screens as an overlay on the first screen displayed in the active area of the display (110). The second screen may be provided from a software application corresponding to the visual object (132). For example, the electronic device (101) may, in response to an input indicating selection of a visual object (630), display a second screen as an overlay on a first screen that occupies the entire active area of the display (110).
[0129] Referring to FIG. 6A, in a state (602) of displaying a plurality of screens based on a visual object (630), portions (641, 652) where the plurality of screens are displayed may at least partially overlap. A portion (641) where a first screen is displayed among the plurality of screens may correspond to the entire area of the active area of the display (110). A portion (642) where a second screen is displayed among the plurality of screens may have a smaller size than the portion (641). Since the second screen is displayed in a portion (642) that overlaps the portion (641), at least a portion of the first screen may be covered by the second screen. The electronic device (101) may display handles (643, 644) for adjusting the position and / or size of each of the portions (641, 642) on each of the portions (641, 642).
[0130] As described above, according to one embodiment, the electronic device (101) may support a function for creating and / or adding a visual object (e.g., a visual object (630) of FIG. 6A) for providing multiple screens using PBP and / or PIP based on areas (611, 612, 613, 614, 615, 616, 617, 618) formed along a boundary of the visual object (131). Based on an area (619) corresponding to an internal area of the visual object (131), the electronic device (101) may support a function for creating and / or adding a folder.
[0131] Referring to FIG. 6B, an electronic device (101) that identifies an input for moving a visual object (132) to a location (q3) within an area (619) spaced apart from a boundary of a visual object (131) along a path (650) within a state (403) may create a folder containing the visual objects (131, 132). The electronic device (101) that identifies the input may switch from the state (403) to the state (603) and display a visual object (660) representing the folder. The electronic device (101) may display the visual object (660) based on the location of the visual object (131) within the display (110). The visual object (660) may include images (or icons) related to the visual objects (131, 132). The visual object (660) may have the same size as another visual object displayed together with the visual object (660) within the display (110), for example, a visual object (133) corresponding to a weather application.
[0132] According to one embodiment, the electronic device (101) may create a directory in a memory (e.g., memory (215) of FIG. 2) containing all software applications corresponding to the visual objects (131, 132) based on identifying a release of an input for movement of the visual object (132) after the visual object (132) is moved into the visual object (131) along the path (650). The visual object (660) of FIG. 6B may correspond to the directory.
[0133] Referring to FIG. 6B, in a state (603) in which a visual object (660) corresponding to a folder (or directory) including visual objects (131, 132) is displayed, in response to an input indicating selection of the visual object (660), the electronic device (101) may display a visual object (670) for displaying a list of software applications included in the folder on at least a portion of the display (110). Within the visual object (670), the electronic device (101) may display text (672) assigned to the folder (or directory). Within the visual object (670), the electronic device (101) may display visual objects (131, 132) representing each of the software applications included in the folder. In response to an input indicating selection of any one of the visual objects (131, 132), the electronic device (101) may execute one software application associated with the input. The electronic device (101) can display a screen provided from the executed software application based on the size of the active area of the display (110).
[0134] As described above, according to one embodiment, the electronic device (101) may create a folder containing software applications or create a visual object for providing a multi-window based on the software applications as icons (e.g., visual objects (131, 132)) corresponding to different software applications are overlapped by a user input. Based on the position and / or direction in which the icons overlap, the electronic device (101) may selectively create either the folder or the visual object.
[0135] FIG. 7 illustrates an example of an operation of an electronic device (101) based on the positions of overlapping visual objects (131, 132). The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIG. 7. The operation of the electronic device (101) described with reference to FIG. 7 may be related to at least one of the operations of FIG. 3.
[0136] Referring to FIG. 7, an exemplary state (701) of an electronic device (101) is illustrated that identifies a first input indicating selection of a visual object (131) while visual objects (131, 132) corresponding to software applications are displayed. For example, the first input may include a long-touch gesture for a location (p1) of a display (110) where the visual object (131) is displayed. Based on the first input, the electronic device (101) may display the visual object (131) based on a width (w2) and a height (h2) that exceed the width (w1) and the height (h1) of another visual object (132).
[0137] Within state (701) of FIG. 7, the electronic device (101) may identify a second input indicating movement of the visual object (132). The second input may include a drag gesture that is initiated at a location (p2) of the display (110) where the visual object (132) is displayed and is performed along a path (711). Based on the second input, the electronic device (101) may move the visual object (132) displayed within the display (110) along the path (711). According to one embodiment, the electronic device (101) may determine a layout of screens corresponding to the visual objects (131, 132) based on a location and / or direction of the visual object (132) overlapping the visual object (131) according to the second input. The electronic device (101) may create and add a visual object for displaying the screens based on the determined layout.
[0138] Referring to FIG. 7, the electronic device (101) that has identified a visual object (132) that has moved to a position (q3) on the edge of the visual object (131) along a path (711) may display a visual object (730) that covers a portion of the visual object (131) that includes the position (q3). The visual object (730) may be referred to as a visual cue for guiding the layout of screens corresponding to the visual objects (131, 132). By using the visual object (730), the electronic device (101) may guide or indicate that the screens will be displayed based on the shape of the visual object (131) distinguished by the visual object (730).
[0139] Referring to FIG. 7, in response to the release of a second input for moving the visual object (132) based on the visual object (132) moved to the position (q3), within a state in which the visual object (730) is displayed, the electronic device (101) may display a visual object (741) for displaying screens corresponding to the visual objects (131, 132). The electronic device (101) may display the visual object (741) having a width (w1) and a height (h1) within the display (110). The electronic device (101) may display the visual object (741) including images (742, 743) corresponding to the visual objects (131, 132). Within the visual object (741), the image (742) corresponding to the visual object (131) may have a position corresponding to a portion of the visual object (131) that is not overlapped with the visual object (730). Within the visual object (741), an image (743) corresponding to the visual object (132) may have a position corresponding to a portion of the visual object (131) that overlaps the visual object (730). In response to an input indicating selection of the visual object (741), the electronic device (101) may display a plurality of screens having a layout corresponding to the arrangement of the images (742, 743) within the display (110). For example, the electronic device (101) may display screens having a layout of PBP based on the arrangement of the images (742, 743).
[0140] Referring to FIG. 7, the electronic device (101) that has identified a visual object (132) that has moved to a location (q2) within an area (721) corresponding to a vertex of the visual object (131) along a path (711) can highlight the area (721). For example, among the areas (721, 722, 723, 724) corresponding to each of the vertices of the visual object (131), the electronic device (101) can display the color of the area (721) in a different color from the colors of the other areas (721, 722, 723). In a state where a visual object (132) moved to a location (q2) is identified, in response to a release of a drag gesture for moving the visual object (132), the electronic device (101) may display a visual object (750) for displaying screens corresponding to the visual objects (131, 132). The electronic device (101) may display the visual object (750) for displaying the screens based on a layout related to the location (q2) for the visual object (131). The visual object (750) may include the visual object (131) and an image (752) superimposed on the visual object (131). The image (752) may include an icon of a software application corresponding to the visual object (132). The location of the image (752) within the visual object (750) may correspond to the location (q2) of the visual object (132) superimposed on the visual object (131).
[0141] In one embodiment, in response to an input indicating selection of a visual object (750), the electronic device (101) may display screens having a layout of a PIP. For example, the electronic device (101) may display a first screen corresponding to a visual object (131) based on a size of the display (110), and may display a second screen corresponding to a visual object (132) at a location adjacent to a vertex of the display (110) corresponding to a vertex of the visual object (131) associated with a location (q2).
[0142] Hereinafter, with reference to FIGS. 8A to 8E, the operation of an electronic device (101) for creating and adding visual objects (e.g., visual objects (434, 444, 454, 525, 526, 527, 533, 534, 630, 741, 750)) described with reference to FIGS. 4A to 7 is described.
[0143] FIGS. 8A to 8E illustrate an example of a flowchart of an electronic device according to one embodiment. The electronic device (101) and / or the processor (210) of FIGS. 1 to 2 may perform the operations described with reference to FIGS. 8A to 8E. The operations of the electronic device described with reference to FIGS. 8A to 8E may be related to the operations of the electronic device (101) described with reference to FIGS. 3 to 7.
[0144] Referring to FIG. 8A, in operation (810), according to one embodiment, a processor of an electronic device may increase a size of a first visual object based on a first input indicating selection of the first visual object from among a plurality of visual objects displayed on a display. The first input may include a tap gesture identified by a touch sensor of the electronic device (e.g., the touch sensor (227) of FIG. 2). The enlarged state (402) of the visual object (131) of FIG. 4A and / or the enlarged state (502) of the visual object (511) of FIG. 5A may be exemplary states of the electronic device in which operation (810) is performed. Based on operation (810), the processor may display the size of the first visual object to be larger than the sizes of other visual objects.
[0145] Referring to FIG. 8A, in operation (820), a processor of an electronic device according to an embodiment may move a second visual object among a plurality of visual objects in response to a second input indicating movement of the second visual object. The second input of operation (820) may include a drag gesture with respect to the second visual object. Drag gestures performed along paths (421, 422, 423) of FIG. 4A, drag gestures performed along paths (521, 522, 523) of FIG. 5B, drag gestures performed along paths (531, 532) of FIG. 5C, drag gestures performed along paths (620, 650) of FIGS. 6A-6B, and / or drag gestures performed along path (711) of FIG. 7 may be included in the second input of operation (820). While identifying a second input including a drag gesture, the processor can move a second visual object along a path of the drag gesture.
[0146] Referring to FIG. 8A, in operation (830), a processor of an electronic device according to an embodiment may display a third visual object for providing multiple screens using a multi-window based on identifying a second visual object overlapping a first visual object according to a second input. The processor may determine a layout of the multiple screens to be provided through the third visual object using a position and / or direction in which the second visual object is overlapping on the first visual object. The third visual object displayed based on operation (830) may include visual objects (434, 444, 454) of FIG. 4A, visual objects (525, 526, 527) of FIG. 5B, visual objects (533, 534) of FIG. 5C, visual objects (630) of FIG. 6A, and / or visual objects (741, 750) of FIG. 7. The third visual object displayed based on the action (830) may have the size of the first visual object prior to having its size increased based on the action (810). Within the third visual object, the processor may display thumbnails (or reduced images) for the first visual object and the second visual object based on the position at which the second visual object overlaps the first visual object.
[0147] Referring to FIG. 8A, in operation (840), a processor of an electronic device according to an embodiment may simultaneously display multiple screens based on a third input indicating selection of a third visual object. States (407, 408) of FIG. 4B, states (505, 506, 507) of FIG. 5D, and / or state (602) of FIG. 6A may be examples of states of an electronic device that performed operation (840). The processor may display multiple screens of operation (840) based on a layout corresponding to an overlapping relationship between the first visual object and the second visual object of operation (830). The above layout can be determined from among PBP for displaying screens in non-overlapping portions (e.g., portions (151, 152, 481, 482) of FIG. 4b, portions (551, 552, 553, 561, 562, 571, 572) of FIG. 5d) of a display (e.g., display (110) of FIGS. 1 and 2), and PIP in which one of the plurality of screens overlaps another.
[0148] Hereinafter, the operation of the electronic device for the third visual object of operations (830, 840) is described with reference to FIGS. 8B to 8E. Within the description of FIGS. 8B to 8E, descriptions that overlap with the operations and / or descriptions of FIG. 8A may be omitted.
[0149] Referring to FIG. 8B, in an operation (832) following operations (810, 820) of FIG. 8A, a processor of an electronic device according to an embodiment may display a third visual object for providing multiple screens according to a multi-window, using the position of the second visual object with respect to the first visual object, based on identifying a second visual object moved to a position adjacent to the first visual object according to a second input. Based on the second visual object moved to a position adjacent to the first visual object, the processor may display a visual cue (e.g., visual objects (432, 442, 452) of FIG. 4A, visual object (524) of FIGS. 5B to 5C, visual object (432) of FIGS. 6A to 6B) for guiding the creation of the third visual object. Based on the positional relationship between the first visual object and the second visual object within the visual cue, the processor can determine the layout of the plurality of screens to be displayed by an input for selecting a third visual object. The state (406) of FIG. 4B, the state (504) of FIG. 5D, and / or the states (601, 603) of FIGS. 6A to 6B may be examples of states of an electronic device that performed operation (832).
[0150] Referring to FIG. 8B, in operation (840), according to one embodiment, a processor of an electronic device may simultaneously display multiple screens based on a third input indicating a selection of a third visual object displayed based on operation (832). The processor performing operation (840) of FIG. 8B may display multiple screens positioned based on the PBP on the display.
[0151] Referring to FIG. 8C, in operation (834) following operations (810, 820) of FIG. 8A, the processor of the electronic device according to one embodiment may display a third visual object for providing multiple screens using a multi-window based on identifying a second visual object moved to the location of the vertex of the first visual object according to a second input. The second input for moving the second visual object to the location of the vertex of the first visual object in operation (834) may include a drag gesture performed along the path (620) of FIG. 6A and / or a drag gesture for moving the visual object (132) to the location (q2) along the path (711) of FIG. 7. The third visual object of FIG. 8C may include the visual object (630) of FIG. 6A and / or the visual object (750) of FIG. 7. The third visual object of FIG. 8c can be generated to display multiple screens having a layout of PIP based on the vertices of the display corresponding to the vertices.
[0152] Referring to FIG. 8C , in operation (844), according to an embodiment, a processor of an electronic device may display a first screen corresponding to a first visual object and a second screen corresponding to a second visual object and superimposed on a portion of the first screen, based on a third input indicating selection of a third visual object. The first screen may be displayed based on execution of a software application corresponding to the first visual object. The second screen may be displayed based on execution of a software application corresponding to the second visual object. The processor may display the first screen having a size of an active area of the display, and may display the second screen as an overlay on the first screen. The processor may display the second screen at a position overlapping a vertex of the display corresponding to a vertex of the first visual object of operation (834), or adjacent to the vertex of the display. Within the display, the width and height of the first screen may be greater than the width and height of the second screen. Based on the operation (844), the processor can display a first screen and a second screen having a layout of PIP.
[0153] Referring to FIG. 8D , in operation (836) following operations (810, 820) of FIG. 8A , a processor of an electronic device according to an embodiment may identify whether a second visual object has moved to an area adjacent to a first visual object in response to a second input. The area of operation (836) may include an area (520) of FIG. 5B and / or a portion of a display on which a visual object (131) of FIG. 7 is displayed. Based on the second visual object moving to the area of operation (836), the processor may visualize or highlight the area. For example, the processor may display visual objects (432, 442, 452) of FIGS. 4A and 6A and / or visual object (524) of FIGS. 5B to 5C . If the second visual object has moved to an area adjacent to the first visual object (836 - Yes), the processor may perform operation (846). If the second visual object overlaps the first visual object, or if the second visual object is located outside the area of the action (836) (836-No), the processor may perform the action (837).
[0154] Referring to FIG. 8D , in operation (846), according to an embodiment, a processor of an electronic device may display a third visual object for displaying screens corresponding to a first visual object and a second visual object in different parts of a display using a multi-window, based on a position of a second visual object moved by a second input. The processor may identify a layout corresponding to a position of the second visual object with respect to the first visual object (or an area adjacent to the first visual object where the second visual object is positioned), identified based on operation (836). The processor may display a third visual object for displaying the screens according to the layout. The processor may display icons representing each of the first visual object and the second visual object within the third visual object, according to the layout. The visual objects (434, 444, 454) of FIG. 4a, the visual objects (525, 526, 527) of FIG. 5b and / or the visual objects (533, 534) of FIG. 5c may be examples of a third visual object indicated by the action (846).
[0155] Referring to FIG. 8D , in operation (837), the processor of the electronic device according to one embodiment may identify whether the second visual object has moved to an area overlapping a vertex of the first visual object based on or in response to a second input. The area of operation (837) may include areas (612, 614, 616, 618) of FIG. 6A and areas (721, 722, 723, 724) of FIG. 7 . If the second visual object has moved to an area overlapping a vertex of the first visual object (837 - Yes), the processor may perform operation (847). If the second visual object does not overlap a vertex of the first visual object (837 - No), the processor may perform operation (860).
[0156] Referring to FIG. 8D , in operation (847), according to one embodiment, the processor of the electronic device may display a fourth visual object for displaying second screens corresponding to the second visual object based on a vertex of the first visual object overlapping the second visual object, together with a first screen corresponding to the first visual object. The fourth visual object of operation (847) may include the visual object (144) of FIG. 1 , the visual object (630) of FIG. 6A , and / or the visual object (750) of FIG. 7 .
[0157] Referring to FIG. 8d, within operation (860), the processor of the electronic device according to one embodiment may execute a function corresponding to the positional relationship between the second visual object and the first visual object. For example, the processor may perform operation (838) described with reference to FIG. 8e. Referring to FIG. 8e, within operation (838) following operations (810, 820), the processor of the electronic device according to one embodiment may identify whether the second visual object has moved to an area overlapping with the edge of the first visual object according to the second input. The area overlapping with the edge of the first visual object of operation (838) may include areas (611, 613, 615, 617) of FIG. 6b. If the second visual object has moved to the area of operation (838) (838-Yes), the processor may perform operation (848). If the second visual object moves to an area different from the area of the action (838) (838-No), the processor can perform the action (839).
[0158] Referring to FIG. 8E, in operation (848), a processor of an electronic device according to an embodiment may display a fifth visual object for displaying screens corresponding to the first visual object and the second visual object in different portions of the display using a multi-window, based on an edge of the first visual object overlapping the second visual object. In an embodiment, based on an input indicating selection of the fifth visual object, the processor may display a plurality of screens having a layout corresponding to the edge of the first visual object described in operation (838). The visual object (741) of FIG. 7 may be an example of a fifth visual object displayed based on operation (848).
[0159] Referring to FIG. 8E, in operation (839), the processor of the electronic device according to one embodiment may identify whether the second visual object moves to an inner region of the first visual object according to the second input. The inner region of operation (839) may include region (619) of FIG. 6B. If the processor identifies the second visual object that has moved to the inner region of the first visual object according to the second input (839—Yes), the processor may perform operation (849). If the second visual object is located in a different region than the inner region of operation (839—No), the processor may perform operation (850).
[0160] Referring to FIG. 8E, in operation (849), according to one embodiment, the processor of the electronic device may display a sixth visual object representing a folder including a first visual object and a second visual object. The visual object (660) of FIG. 6B may be an example of the sixth visual object of operation (849). In response to an input indicating selection of the sixth visual object, the processor may display a visual object (e.g., visual object (670) of FIG. 6B) corresponding to the folder including the first visual object and the second visual object. Based on operation (849), the processor may simultaneously support both a function for creating a folder and a function for adding a visual object for a multi-window.
[0161] Based on displaying the third through sixth visual objects in operations (846, 847) of FIG. 8d and operations (848, 849) of FIG. 8e, the processor may at least temporarily stop displaying the first visual object and / or the second visual object on the display. For example, a processor that has displayed at least one of the third through sixth visual objects may hide and / or remove the first visual object and / or the second visual object within the display.
[0162] Referring to FIG. 8E, within operation (850), the processor may display a second visual object moved according to the second input at a location distinct from the first visual object. If the second visual object is moved to a different area from all of the areas of operations (836, 837, 838, 839), the processor may display the second visual object at a location independent of the first visual object.
[0163] FIG. 9 illustrates an example of an operation of an electronic device (101) that identifies an input indicating an overlap of visual objects (131, 132, 133). The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIG. 9. The operation of the electronic device (101) described with reference to FIG. 9 may be related to at least one of the operations of FIG. 3 and / or FIGS. 8A to 8E.
[0164] Referring to FIG. 9, states (901, 902, 903) of an electronic device (101) are illustrated that receive inputs overlapping at least two of visual objects (131, 132, 133). Within state (901), the electronic device (101) may enlarge a visual object (131) based on a first input indicating selection of the visual object (131) (e.g., a long-touch gesture performed on a location (p1). For example, a width (w2) and a height (h2) of the visual object (131) within state (901) may be greater than a width (w1) and a height (h1) of another visual object (132). While receiving the first input, the electronic device (101) may identify a second input indicating movement of the visual object (132). The embodiment is not limited thereto, and the electronic device (101) may identify a second input after the first input. The second input may include a drag gesture performed by a fingertip that is initiated at a location (p2) of the display (110) where the visual object (132) is displayed and is dragged along a path (911). When the electronic device (101) identifies the visual object (132) that has moved to a location (q2) within the visual object (131) based on the second input, the electronic device (101) may switch from state (901) to state (902).
[0165] Within the state (902) of FIG. 9, the electronic device (101) may display a visual object (920) having the form of a container containing visual objects (131, 132), similar to the states (403, 404) of FIG. 4A and the state (503) of FIG. 5C. Within the state (902) of displaying the visual object (920), based on identifying a tap gesture on a blank area (or idle area) of the display (110), the electronic device (101) may generate and / or display a visual object for providing screens corresponding to the visual objects (131, 132) included in the visual object (920). Within the state (902) of FIG. 9, the electronic device (101) may identify a third input for adding a software application to the visual object (920). For example, the third input may include a drag gesture performed by a fingertip that starts from a location (p3) where a visual object (133) is displayed and is dragged along a path (922). Based on the third input, the electronic device (101) that has identified the visual object (133) as having moved to a location (q3) within the visual object (920) may switch from state (902) to state (903).
[0166] Within state (903) of FIG. 9, the electronic device (101) can display visual objects (131, 132, 133) identified by the first input to the third input within a visual object (920). The electronic device (101) can display the visual objects (131, 132, 133) in different portions within the visual object (920). Within state (903), based on identifying a tap gesture for a blank area (or idle area) of the display (110), the electronic device (101) can generate and / or display a visual object (e.g., visual objects (533, 534) of FIG. 5c) to simultaneously provide three screens corresponding to the visual objects (131, 132, 133) within the visual object (920).
[0167] As described above, according to one embodiment, the electronic device (101) can sequentially identify or add screens to be displayed based on a multi-window based on the first to third inputs performed discretely.
[0168] FIG. 10 illustrates an example of a flowchart of an electronic device according to one embodiment. The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operations of the electronic device (101) described with reference to FIG. 10. The operations of the electronic device (101) described with reference to FIG. 10 may be related to at least one of the operations of FIG. 3 and / or FIGS. 8A to 8E.
[0169] Referring to FIG. 10, in operation (1010), according to one embodiment, a processor of an electronic device may display a plurality of visual objects. The processor may perform operation (1010) of FIG. 10 similarly to operation (310) of FIG. 3. For example, the processor may perform operation (1010) based on the execution of the launcher application (230) of FIG. 2.
[0170] Referring to FIG. 10 , in operation (1020), a processor of an electronic device according to one embodiment may identify a first input indicating selection of a first visual object from among a plurality of visual objects. The processor may perform operation (1020) of FIG. 10 similarly to operation (320) of FIG. 3 . The first input of operation (1020) may correspond to the first input of operation (810) of FIGS. 8A to 8E .
[0171] Referring to FIG. 10 , in operation (1030), according to one embodiment, a processor of an electronic device may identify a second input indicating that a second visual object among a plurality of visual objects overlaps a first visual object corresponding to the first input. The second input identified based on operation (1020) may include the second input of operation (330) of FIG. 3 and / or the second input of operation (820) of FIGS. 8A to 8E .
[0172] Referring to FIG. 10 , in operation (1040), according to an embodiment, a processor of an electronic device may determine whether a difference between a timing at which a second input is identified and a timing at which a first input is identified is less than a threshold. Based on operation (1030), the processor may determine whether the second input is identified within a specified period of time (e.g., a time interval formed after the first input by the threshold) after the first input. The specified period of time may be configured to determine whether the second input is an input indicating addition of a visual object for a multi-window or an input for creating a folder. If the difference in operation (1030) is less than the threshold (1040 - Yes), the processor may perform operation (1050). If the difference in operation (1030) is greater than or equal to the threshold (1040 - No), the processor may perform operation (1060).
[0173] Referring to FIG. 10, in operation (1050), according to an embodiment, a processor of an electronic device may display a third visual object for providing all screens corresponding to a first visual object and a second visual object based on a multi-window. The third visual object may be provided to execute software applications corresponding to the screens based on multi-tasking. The third visual object of operation (1050) may correspond to the third visual object of operation (350) of FIG. 3. The third visual object may correspond to a function for displaying the screens based on a layout corresponding to a position and / or direction of a second visual object superimposed on the first visual object by a second input.
[0174] Referring to FIG. 10, in operation (1060), according to one embodiment, the processor of the electronic device may display a fourth visual object corresponding to a folder that includes all of the first visual object and the second visual object. The fourth visual object of operation (1060) may include the visual object (660) of FIG. 6B. Referring to operations (1040, 1050, 1060), although an embodiment of displaying a third visual object based on the first input and the second input identified along an interval below a threshold has been described, the embodiment is not limited thereto. In one embodiment, the processor may perform operation (1050) if the difference between the timings at which the first input and the second input are identified is greater than or equal to the threshold, and may perform operation (1060) if the difference is less than the threshold.
[0175] Hereinafter, with reference to FIGS. 11A and 11B, exemplary operation of an electronic device providing a menu including options for creating a visual object providing multiple screens is described.
[0176] FIGS. 11A and 11B illustrate an example of an operation of an electronic device (101) that identifies an input for generating a visual object that provides multiple screens. The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIGS. 11A and 11B . The operation of the electronic device (101) described with reference to FIGS. 11A and 11B may be related to at least one of the operations of FIG. 3 , FIGS. 8A to 8E , and / or FIG. 10 .
[0177] Referring to FIGS. 11A and 11B , different states (1101, 1102, 1103, 1104, 1105, 1106, 1107) of an electronic device (101) that has received an input for generating a visual object providing multiple screens are illustrated. Within the state (1101) of FIG. 11A , the electronic device (101) may display a screen including visual objects (131, 132) corresponding to software applications. Based on a designated input for the visual object (132) displayed on the display (110), the electronic device (101) may display a visual object (1110) in the form of a menu including options related to the visual object (132). The designated input may include a long-touch gesture performed for a specified period of time.
[0178] Referring to FIG. 11A, options included in a menu in a state (1101) in which a visual object (1110) having a menu form corresponding to a visual object (132) is displayed are exemplarily illustrated. For example, as options displayed in the visual object (1110), the electronic device (101) may display at least one of a visual object (1111) corresponding to a first option for identifying an input for selecting one or more visual objects (or icons) displayed on the display (110), a visual object (1112) corresponding to a second option for removing the visual object (132) from a list of software applications provided through a launcher screen, a visual object (1113) corresponding to a third option for removing the software application corresponding to the visual object (132) from a memory of the electronic device (101) (e.g., the memory (215) of FIG. 2), or a visual object (1114) corresponding to a fourth option for displaying information on a software application corresponding to the visual object (132). In response to an input indicating a selection of any one of the visual objects (1111, 1112, 1113, 1114), the electronic device (101) can execute a function corresponding to an option selected by the input among the first option to the fourth option.
[0179] State (1102) of FIG. 11A may be an exemplary state of the electronic device (101) switched from state (1101) in response to an input indicating selection of a visual object (1111). Based on the input indicating selection of a visual object (1111), the electronic device (101) may execute a function for editing a list of software applications displayed through a launcher screen. Within state (1102) of FIG. 11A, the electronic device (101) may display visual objects (1120-1, 1120-2, 1120-3) overlapping on vertices of the visual objects displayed on the screen to identify an input indicating selection of the visual objects (e.g., visual objects (131, 132)). Although visual objects (1120-1, 1120-2, 1120-3) having the form of check boxes are illustrated as an example, the embodiment is not limited thereto.
[0180] Within the exemplary state (1102) of FIG. 11A, the electronic device (101) may display a visual object (1122) in the form of a menu containing options to be executed based on a visual object selected from among the visual objects (1120-1, 1120-2, 1120-3). During the transition from state (1101) to state (1102), the electronic device (101) may hide or remove the visual object (1110) that was displayed within the display (110). As options displayed within the visual object (1122), the electronic device (101) may display at least one of a visual object (1124) corresponding to a first option for removing a software application corresponding to a visual object selected based on the visual objects (1120-1, 1120-2, 1120-3), a visual object (1126) corresponding to a second option for creating a folder containing the visual object selected based on the visual objects (1120-1, 1120-2, 1120-3), or a visual object (1128) corresponding to a third option for creating a visual object that simultaneously provides screens corresponding to the visual objects selected based on the visual objects (1120-1, 1120-2, 1120-3).
[0181] Referring to FIG. 11A, within an exemplary state (1102) in which an input indicating selection of visual objects (131, 132) is identified based on visual objects (1120-1, 1120-2), the electronic device (101) may identify an input indicating selection of a visual object (1128). The electronic device (101) having identified the input within the state (1102) may switch to the state (1103) of FIG. 11A. Within the state (1103), the electronic device (101) may display a visual object (1130) within the display (110) to simultaneously provide screens corresponding to the visual objects (131, 132) selected based on the visual objects (1120-1, 1120-2). Referring to FIG. 11A, each of the images (1131, 1132) included in the visual object (1130) may correspond to icons included in each of the visual objects (131, 132). In response to an input indicating selection of the visual object (1130), the electronic device (101) may display a plurality of screens arranged within the display (110) based on the arrangement of the visual objects (131, 132) within the visual object (1130). The plurality of screens may have a layout corresponding to the arrangement of the visual objects (131, 132).
[0182] Referring to FIG. 11b, in a state (1104) in which visual objects (131, 132, 133) are displayed based on a launcher screen, the electronic device (101) can identify a drag gesture that is initiated at a location (p1) in which the visual object (132) is displayed and moves along a path (1140) to a location (q1) in which the visual object (1131) is displayed. Based on the drag gesture, the electronic device (101) can display the visual objects (131, 132) in an overlapping manner within the display (110). Based on the visual objects (131, 132) displayed in an overlapping manner, the electronic device (101) can switch to a state (1105) of FIG. 11b. Within the state (1105) of FIG. 11b, the electronic device (101) may display a visual object (1150) in the form of a menu containing executable options based on the visual objects (131, 132). For example, as options displayed within the visual object (1150), the electronic device (101) may display a visual object (1151) corresponding to a first option for creating a folder (or directory) containing the overlapping visual objects (131, 132) and / or a visual object (1152) corresponding to a second option for creating and / or displaying a visual object for providing screens corresponding to the overlapping visual objects (131, 132) based on multi-tasking.
[0183] Within state (1105) of FIG. 11B, in response to an input indicating a selection of a visual object (1151) corresponding to a first option, the electronic device (101) may switch to state (1106). Within state (1106), the electronic device (101) may display a visual object (1160) corresponding to a folder containing visual objects (131, 132). Based on the input, the electronic device (101) that has created a folder containing visual objects (131, 132) may refrain from displaying the visual objects (131, 132) within the display (110). For example, the visual objects (131, 132) may be displayed based on an input indicating a selection of a visual object (1160) corresponding to the folder. In response to an input indicating a selection of a visual object (1160), the electronic device (101) may display a visual object (1162) for displaying the visual objects (131, 132) contained in the folder within the display (110). The visual object (1162) of FIG. 11b may correspond to the visual object (670) of FIG. 6b.
[0184] Within the state (1105) of FIG. 11B, in response to an input indicating selection of a visual object (1152) corresponding to the second option, the electronic device (101) may switch to the state (1107). Within the state (1107), the electronic device (101) may display a visual object (1130) for simultaneously displaying screens corresponding to the visual objects (131, 132). The electronic device (101) displaying the visual object (1130) may stop displaying the visual objects (131, 132) within the display (110). The electronic device (101) may display the visual object (1130) on a portion of the display (110) where the visual object (131) was displayed. The visual object (1130) may include images (1131, 1132) representing each of the visual objects (131, 132).
[0185] Within state (1107) of FIG. 11B, based on an input indicating selection of a visual object (1130), the electronic device (101) may switch to state (1108). Within state (1108), the electronic device (101) may execute all of a plurality of software applications (e.g., an email application and an alarm application) represented by the visual object (1130) based on multitasking. The electronic device (101) may display screens provided from the software applications executed based on multitasking based on the positional relationship of the images (1131, 1132) within the display (110). For example, the screens may be arranged within the display (110) along a vertical direction of the display (110).
[0186] Although the operation of the electronic device (101) that identifies a touch input that has come into contact with the display (110) based on the touch sensor (227) of FIG. 2 has been described, the embodiment is not limited thereto. Hereinafter, with reference to FIGS. 12A to 12C, one embodiment of the electronic device (101) including an HMD (head-mounted display) is described.
[0187] FIGS. 12A to 12C illustrate an example of an operation of an electronic device (101-3) performed within a virtual space. The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operations of the electronic device (101-3) described with reference to FIGS. 12A to 12C. The operations of the electronic device (101-3) described with reference to FIGS. 12A to 12C may be related to at least one of the operations of FIG. 3, FIGS. 8A to 8E, and / or FIG. 10.
[0188] Referring to FIGS. 12A to 12C , different states (1201, 1202, 1203) of an electronic device (101-3) displaying at least a portion of a virtual space are illustrated. The electronic device (101-3) is a device that can be worn on a user's head and may be referred to as a wearable device. The electronic device (101-3) may be designed to provide a user experience for augmented reality (AR), virtual reality (VR), mixed reality (MR), extended reality (XR), and / or video (or visible) see-through (VST). For example, the electronic device (101-3) may include one or more sensors and / or cameras to identify the location, motion (e.g., motion of the head of the user (1210)), direction of gaze, and / or hand gesture (e.g., gesture performed by hands (1211, 1212)) of the user (1210) wearing the electronic device (101-3).
[0189] In a state (1201) of FIG. 12A, according to an embodiment, the electronic device (101-3) may display at least a portion of a virtual space within a field-of-view (FoV) of the user (1210) while the electronic device (101-3) is worn by the user (1210). Referring to FIG. 12A, an exemplary state (1201) of the electronic device (101-3) is illustrated that displays five virtual objects (virtual objects A to E) corresponding to different software applications within the FoV. According to an embodiment, the electronic device (101-3) may display five virtual objects having a two-dimensional plane shape within a three-dimensional virtual space. In a state (1201) of FIG. 12A, virtual objects A to E may be provided from software applications installed in the electronic device (101-3). For example, each of virtual objects A to E may have a screen shape provided by a software application.
[0190] Within the state (1201) of FIG. 12A, according to one embodiment, the electronic device (101-3) can identify the position and / or posture of hands (1211, 1212). For example, the electronic device (101-3) can identify a hand (1211) at a position (p1) in external space, which is mapped to a position (v1) within the FoV where the E virtual object (1210) is displayed. Referring to FIG. 12A, the electronic device (101-3) that has identified the hand (1211) including two fingers in the form of a ring, such as a pinch gesture, can identify a first input indicating selection of a virtual object (e.g., the E virtual object (1210)) corresponding to the position (v1). While the pinch gesture by the hand (1211) is maintained, the electronic device (101-3) can identify the hand (1212) at a position (p2) in the external space corresponding to the position (v2) in the C virtual object (1220). When the hand (1212) having the posture of the pinch gesture is moved in the external space along any one of the paths (1231, 1232), the processor can move the C virtual object (1220) along the paths (1241, 1242) in the virtual space corresponding to the paths (1231, 1232). According to one embodiment, the electronic device (101-3) may generate and / or display a virtual object that performs the function of a shortcut for displaying a combination of screens corresponding to the plurality of virtual objects based on an input of overlapping a plurality of virtual objects (e.g., C virtual object (1220) and E virtual object (1210)), similar to an input of overlapping visual objects in the form of icons representing the software applications of FIGS. 4A to 9.
[0191] Referring to FIG. 12A, based on identifying a hand (1212) moved from a position (p2) to a position (p3) along a path (1231) within a state (1201), the electronic device (101-3) can move a C virtual object (1220) from a position (v2) to a position (v3) along a path (1241) within the FoV. The position (v3) within the FoV can correspond to the position (p3) of the hand (1212) within the external space. Referring to FIG. 12A, the electronic device (101-3) can identify a C virtual object (1220) moved to a position (v3) on a line connecting vertices on a boundary line of the E virtual object (1210). In response to an input to move the C virtual object (1220) to a position (v3) on the edge of the E virtual object (1210), the electronic device (101-3) may switch to the state (1202) of FIG. 12b.
[0192] Referring to FIG. 12B, within a state (1202), the electronic device (101-3) may display a virtual object (1250) for displaying a combination of an E virtual object (1210) and a C virtual object (1220). The virtual object (1250) may include a combination of images corresponding to each of the E virtual object and the C virtual object. The electronic device (101-3) may combine the images based on the layout of the screens to be displayed within the FoV based on the selection of the virtual object (1250).
[0193] Referring to FIG. 12B, in a state (1202) where a virtual object (1250) is displayed, the electronic device (101-3) can identify a pinch gesture of a hand (1212) performed at a location (p5) in an external space. If a location (v5) within the FoV corresponding to the location (p5) overlaps with a virtual object (1250) for simultaneously displaying multiple screens, the electronic device (101-3) can identify an input indicating selection of the virtual object (1250) based on the pinch gesture of the hand (1212). Based on the input, the electronic device (101-3) can display the screens within the FoV.
[0194] Referring to FIG. 12B, a panel (1253) is illustrated, on which screens (1251, 1252) are arranged, displayed by an electronic device (101-3) that has identified an input indicating selection of a virtual object (1250). The electronic device (101-3) may display the panel (1253) in the form of a two-dimensional plane floating within the FoV of a user (1210) wearing the electronic device (101-3). Within the panel (1253), a screen (1251) provided from a software application corresponding to the E virtual object (1210) of FIG. 12A and a screen (1252) provided from a software application corresponding to the C virtual object (1220) of FIG. 12A may be displayed along the horizontal direction of the FoV. For example, the screen (1251) may be positioned to the right of the screen (1252) along the horizontal direction within the panel (1253). The arrangement within the panel (1253) of the screens (1251, 1252) may correspond to the arrangement of images combined within the virtual object (1250). The arrangement within the panel (1253) of the screens (1251, 1252) may correspond to the positional relationship of the E virtual object (1210) to the C virtual object (1220) that was moved from the state (1201) of FIG. 12A to the position (v3) that was identified for the creation of the virtual object (1250).
[0195] Referring to FIG. 12A, within a state (1201), the electronic device (101-3) can identify a hand (1212) that moves from a position (p2) to a position (p4) along a path (1232) and has a posture of a pinch gesture. Based on identifying the hand (1212), the electronic device (101-3) can move a C virtual object (1220) from a position (v2) to a position (v4) along a path (1242) within the FoV corresponding to the path (1232). Based on the input for moving the C virtual object (1220) to a position (v4) on a vertex of the E virtual object (1210), the electronic device (101-3) can generate a virtual object for providing all screens related to the E virtual object (1210) and the C virtual object (1220).
[0196] Referring to FIG. 12C, a state (1203) of an electronic device (101-3) displaying a virtual object (1260) for providing all screens related to the E virtual object (1210) and the C virtual object (1220) based on the input is illustrated. Within the state (1203), the electronic device (101-3) can display a virtual object (1260) including a combination of images corresponding to the E virtual object and the C virtual object. Within the state (1203) of displaying the virtual object (1260) based on an input for moving the C virtual object (1220) to a position (v4) on a vertex of the E virtual object (1210), the electronic device (101-3) can display a virtual object (1260) including images arranged based on a positional relationship of the E virtual object (1210) and the C virtual object (1220) where the images are overlapped on the position (v4). The positional relationship of images within the virtual object (1260) can guide the display of screens related to the E virtual object (1210) and the C virtual object (1220) based on PIP.
[0197] Referring to FIG. 12C, in a state (1203) of displaying a virtual object (1260), the electronic device (101-3) can identify a pinch gesture of a hand (1212) performed at a location (p6) in an external space. If a location (v6) within the FoV corresponding to the location (p6) overlaps with the virtual object (1260), the electronic device (101-3) can identify an input indicating selection of the virtual object (1260) based on the pinch gesture. Based on the input, the electronic device (101-3) can display screens provided from a plurality of software applications corresponding to the virtual object (1260) within the FoV.
[0198] Referring to FIG. 12C, a panel (1263) is illustrated, on which screens (1261, 1262) are arranged, displayed by an electronic device (101-3) that has identified an input indicating selection of a virtual object (1260). The panel (1263), similar to the panel (1253) of FIG. 12B, may have the form of a two-dimensional plane floating within the FoV of a user (1210) wearing the electronic device (101-3). Within the panel (1263), the electronic device (101-3) may display a screen (1262) provided from a software application corresponding to the C virtual object (1220) of FIG. 12A, overlaid on a screen (1261) provided from a software application corresponding to the E virtual object (1210) of FIG. 12A. For example, within a panel (1263), screens (1261, 1262) may be arranged based on a PIP layout. The positional relationship of the screens (1261, 1262) may correspond to the positional relationship of the E virtual object (1210) to the C virtual object (1220) that was moved to the position (v4) within the state (1201) of FIG. 12A.
[0199] As described above, the head-mounted electronic device (101-3) can display multiple virtual objects within the FoV based on VR, AR, and / or VST. The electronic device (101-3) can display one virtual object (e.g., the virtual object (1250) of FIG. 12b and / or the virtual object (1260) of FIG. 12c) having the function of a shortcut key for providing the multiple virtual objects based on the layout of the PBP and / or the PIP.
[0200] FIGS. 13A and 13B illustrate an example of an operation of an electronic device (101) connected to one or more external displays (1310, 1320). The electronic device (101) of FIGS. 1 and 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIGS. 13A and 13B. The operation of the electronic device (101) described with reference to FIGS. 13A and 13B may be related to at least one of operation (360) of FIG. 3 and operations (840, 844) of FIGS. 8A to 8C.
[0201] Referring to FIG. 13A, an exemplary state (1301) of an electronic device (101) connected to an external display (1310) is illustrated. The electronic device (101) may be connected to the external display (1310) via a wired connection such as a universal serial bus (USB), a digital visual interface (DVI), a high definition multimedia interface (HDMI), a display port (DP), and / or a d-subminiature (D-sub). The embodiment is not limited thereto, and the electronic device (101) may be connected to the external display (1310) via a wireless connection such as a wireless display (WiDi) and / or a WiFi-direct (e.g., Miracast). In the state (1301) connected to the external display (1310), the electronic device (101) may display a screen provided from a software application through the external display (1310).
[0202] Within the state (1301) of FIG. 13A, the electronic device (101) may display a screen (e.g., a launcher screen) including a list of software applications on the display (110). Within the screen, the electronic device (101) may display visual objects (141, 142) for providing a plurality of screens based on the execution of a plurality of software applications. The visual objects (141, 142) may be added within the launcher screen based on the operation of the electronic device (101) described with reference to FIGS. 4A to 11B.
[0203] Referring to FIG. 13A, in response to an input indicating selection of a visual object (141), the electronic device (101) may display screens corresponding to the visual object (141) on the external display (1310). The layout of the screens displayed on the external display (1310) may correspond to the arrangement of icons (e.g., icons representing each of an alarm application and an email application) included in the visual object (141). In the exemplary state (1301) of FIG. 13A, based on an input for a visual object (141) including icons arranged along the horizontal direction of the display (110), the electronic device (101) may display screens arranged along the horizontal direction of the external display (1310) on the external display (1310). Each of the screens may be provided from software applications corresponding to each of the icons. For example, the electronic device (101) can execute the software applications substantially simultaneously based on an input indicating a selection of a visual object (141). By transmitting a signal to the external display (1310) for displaying screens provided from the executed software applications, the electronic device (101) can cause the external display (1310) to display the screens.
[0204] Referring to FIG. 13A, in response to an input indicating selection of a visual object (142), the electronic device (101) can display screens corresponding to the visual object (142) on an external display (1310). Based on the input indicating selection of a visual object (142) for providing screens of an email application and an alarm application, the electronic device (101) can execute the email application and the alarm application. The electronic device (101) can transmit a signal to the external display (1310) for displaying screens provided from the email application and the alarm application.
[0205] Referring to FIG. 13A, the arrangement and / or layout of screens displayed through the external display (1310) may be related to the positions of icons (e.g., icons corresponding to each of an email application and an alarm application) that were overlapped to create a visual object (142). Based on an input indicating selection of a visual object (142), the electronic device (101) may display screens provided from software applications based on a layout (e.g., a layout based on PBP) arranged along the vertical direction of the external display (1310). The layout may correspond to the arrangement of icons that were overlapped to create the visual object (142). The embodiment is not limited thereto, and the electronic device (101) may display screens using a layout based on PIP.
[0206] Referring to FIG. 13B, an exemplary state (1302) of an electronic device (101) connected to a plurality of external displays (1310, 1320) is illustrated. In the state (1302) in which the plurality of external displays (1310, 1320) are connected, the electronic device (101) can control the external displays (1310, 1320) using information about the arrangement (or positional relationship) of the plurality of external displays (1310, 1320). The arrangement of the plurality of external displays (1310, 1320) can be related to coordinates of pixels of the external displays (1310, 1320) and the display (110) of the electronic device (101). Using the arrangement and / or coordinates, the electronic device (101) can support continuous movement of visual objects between the display and the external displays (1310, 1320). For example, the electronic device (101) may support a function in which a visual object moves from one edge of an external display (1310) to one edge of an external display (1320). The electronic device (101) may identify an arrangement of multiple external displays (1310, 1320) based on user input.
[0207] Referring to FIG. 13b, an electronic device (101) connected to external displays (1310, 1320) can adjust the mapping between the screens and the external displays (1310, 1320) by using the arrangement of the external displays (1310, 1320) based on an input indicating a selection of a visual object (141) for displaying screens provided from a plurality of software applications (e.g., an email application and an alarm application) along the vertical direction of the display (110). For example, the electronic device (101) may, within a state (1302) connected to a plurality of external displays (1310, 1320), identify a third input indicating a selection of a visual object (e.g., visual objects (141, 142)), and may use information representing the arrangement of the plurality of external displays (1310, 1320) to assign each of the screens to the plurality of external displays (1310, 1320).
[0208] For example, if the arrangement of the external displays (1310, 1320) corresponds to the layout of the visual object (141), the electronic device (101) can allocate screens provided based on the visual object (141) to each of the external displays (1310, 1320). Referring to FIG. 13B, the electronic device (101) that receives an input for the visual object (141) for providing screens arranged along the vertical direction of the display (110) can identify information indicating the arrangement of the external displays (1310, 1320). Based on the information, if the external displays (1310, 1320) are set to be arranged along the vertical direction, the electronic device (101) can transmit signals for the screens to each of the external displays (1310, 1320). For example, based on an input indicating selection of a visual object (141) in which icons corresponding to an alarm application and an email application are sequentially displayed along a vertical direction, the electronic device (101) may transmit a first screen corresponding to the alarm application and a second screen corresponding to the email application to each of the external displays (1310, 1320) arranged along the vertical direction. When the external displays (1310, 1320) are arranged along the horizontal direction of the display (110), the electronic device (101) may transmit a signal for displaying a combination of screens related to the visual object (141) to one of the external displays (1310, 1320) based on an input indicating selection of the visual object (141).
[0209] Referring to FIG. 13B, according to one embodiment, the electronic device (101) may identify an input indicating selection of a visual object (142) for displaying screens along a horizontal direction. When external displays (1310, 1320) are arranged along the horizontal direction of the display (110), the electronic device (101) that has identified the input may transmit signals for displaying screens related to the visual object (142) to each of the external displays (1310, 1320). When the external displays (1310, 1320) are arranged sequentially along the horizontal direction and icons corresponding to an alarm application and an email application are arranged sequentially within the visual object (142), the electronic device (101) may provide a first screen corresponding to the alarm application to the external display (1310) and a second screen corresponding to the email application to the external display (1320).
[0210] As described above, according to one embodiment, the electronic device (101) can display screens based on PBP and / or PIP within the display (110). When connected to one or more external displays (e.g., external displays (1310, 1320)), the electronic device (101) can display screens based on PBP and / or PIP using the one or more external displays.
[0211] FIG. 14 illustrates an example of an operation of an electronic device (101) for adjusting a layout corresponding to a visual object (141). The electronic device (101) of FIG. 1 and / or FIG. 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIG. 14. The operation of the electronic device (101) described with reference to FIG. 14 may be related to at least one of the operations of FIG. 3, FIGS. 8A to 8E, and / or FIG. 10.
[0212] Referring to FIG. 14, an exemplary state (1401) of an electronic device (101) displaying a visual object (141) for providing screens based on PBP and / or PIP is illustrated. The visual object (141) may be generated and / or displayed based on the operation of the electronic device (101) described with reference to FIGS. 4A to 11B. For example, an electronic device (101) that identifies an input for superimposing a second icon corresponding to an alarm application on a first icon corresponding to an email application may display a visual object (141) for providing screens having a layout corresponding to the positional relationship between the first icon and the second icon. The visual object (141) may include a first image (1411) (e.g., an icon representing an email application) and a second image (1412) (e.g., an icon representing an alarm application) arranged based on the layout. In response to an input indicating a selection of a visual object (141) (e.g., an input including a tap gesture for the visual object (141), the electronic device (101) may display a plurality of screens (e.g., screens provided from an email application and an alarm application) within the display (110) based on the layout.
[0213] According to one embodiment, the electronic device (101) may identify an input indicating a change in the layout of screens to be provided through a visual object (141). The input may include a long-touch gesture with respect to the visual object (141). In response to the input, the electronic device (101) may display one or more visual objects indicating options for changing the layout at locations at least partially overlapping the visual object (141). Within state (1401) of FIG. 14, the electronic device (101) may, in response to the input indicating a change in the layout associated with the visual object (141), display visual objects (1421, 1422) at least partially overlapping the visual object (141). Each of the visual objects (1421, 1422) may correspond to options for adjusting the layout of screens to be displayed by the input indicating selection of the visual object (141).
[0214] Referring to FIG. 14, a visual object (1421) displayed as an overlap on a vertex of a visual object (141) may correspond to an option for swapping positions of screens to be displayed through the visual object (141). For example, an electronic device (101) displaying a visual object (141) including a first image (1411) and a second image (1412) sequentially arranged along a vertical direction of a display (110) may, in response to an input to the visual object (1421), swap positions of the first image (1411) and the second image (1412) within the visual object (141). Based on the swapping of positions, the electronic device (101) may display a visual object (1430) including the first image (1411) and the second image (1412) sequentially arranged along a vertical direction.
[0215] Referring to FIG. 14, a visual object (1422) displayed as an overlay on a different vertex of a visual object (141) than the vertex where the visual object (1421) is displayed may correspond to an option for changing the direction in which screens to be displayed through the visual object (141) are arranged. For example, an electronic device (101) displaying a visual object (141) for displaying screens along the vertical direction of the display (110) may change the direction in which the screens are arranged from the vertical direction to the horizontal direction in response to an input indicating selection of the visual object (1422). In response to an input indicating selection of the visual object (1422), the electronic device (101) may play an animation indicating rotation of the visual object (141), such as the visual object (1440). While rotating the visual object (141) based on the above animation, the electronic device (101) may refrain from displaying visual objects (1421, 1422) corresponding to the options for adjusting the layout.
[0216] Referring to FIG. 14, in response to an input indicating a selection of a visual object (1422), the electronic device (101) may display a visual object (1450) for displaying screens arranged along a horizontal direction of the display (110). The visual object (1450) may display a first image (1411) and a second image (1412) arranged along the horizontal direction. The electronic device (101) displaying the visual object (1450) for providing screens arranged along the horizontal direction may display a visual object (1423) for exchanging the positions of the screens. Based on the input indicating a selection of a visual object (1423) displayed as an overlap on a vertex of the visual object (1450), the electronic device (101) may display a visual object (1460) in which the positions of the first image (1411) and the second image (1412) corresponding to the screens are exchanged.
[0217] In one embodiment, the electronic device (101) may, in response to an input regarding a blank area (or idle area) within the display (110), stop providing options for changing the layout of screens to be presented via the visual object (141). For example, based on an input indicating selection of a blank area of the display (110), the electronic device (101) may stop displaying the visual objects (1421, 1422, 1423). Based on the input, the electronic device (101) may stop (or complete) adjusting the layout of screens to be presented via the visual object (141).
[0218] FIG. 15 illustrates an example of an operation of an electronic device (101) according to one embodiment. The electronic device (101) of FIG. 1 and / or FIG. 2 and / or the processor (210) of FIG. 2 may perform the operation of the electronic device (101) described with reference to FIG. 15. The operation of the electronic device (101) described with reference to FIG. 15 may be related to at least one of the operations of FIG. 3, FIGS. 8A to 8E, and / or FIG. 10.
[0219] Referring to FIG. 15, states (1501, 1502, 1503) of an electronic device (101) that identify an input indicating removal of a visual object (141) for providing multiple screens are illustrated. Within state (1501) of FIG. 15, the electronic device (101) may display the visual object (141) based on a width (w1) and a height (h1). Within the state (1501) displaying the visual object (141), the electronic device (101) may switch to state (1502) based on identifying a long-touch gesture for the visual object (141) (e.g., an input for adjusting the layout of screens to be displayed by the visual object (141). Within state (1502), the electronic device (101) may display the visual object (141) based on an enlarged width (w2) and an enlarged height (h2). Within the state (1502), the electronic device (101) can display visual objects (1421, 1422) at each of the vertices of the visual object (141). The visual objects (1421, 1422) can correspond to options for changing the layout of screens to be provided by the visual object (141).
[0220] Within state (1502) of FIG. 15, the electronic device (101) may identify an input indicating removal of a visual object (141). The input may include drag gestures performed simultaneously by a specified number of external objects. For example, the electronic device (101) may identify a first drag gesture that starts at a location (p1) within the visual object (141) and moves out of the visual object (141) along a direction (1521). For example, the electronic device (101) may identify a second drag gesture that is identified simultaneously with the first drag gesture and starts at a location (p2) within the visual object (141) and moves out of the visual object (141) along a direction (1522). Based on the first and second drag gestures being performed substantially simultaneously, the electronic device (101) may identify an input indicating removal of the visual object (141). Based on an input indicating removal of a visual object (141), the electronic device (101) can switch from state (1502) to state (1503).
[0221] Referring to an exemplary state (1503) of FIG. 15, based on an input indicating removal of a visual object (141), the electronic device (101) may at least temporarily stop displaying the visual object (141) within the display (110). In one embodiment, the electronic device (101) may display visual objects (131, 132) within the display (110), the visual objects including icons representing software applications associated with the visual object (141). By removing the visual object (141) and displaying the visual objects (131, 132), the electronic device (101) may provide an animation (or visual effect) such that the visual objects (131, 132) are segmented from the visual object (141).
[0222] As described above, according to one embodiment, the electronic device (101) may display a visual object (141) for providing multiple screens simultaneously based on a single input. The multiple screens to be displayed by the visual object (141) may have a layout corresponding to the positional relationship of icons that were overlapped with each other by the input for creating the visual object (141). The layout of the screens that the electronic device (101) provides based on the visual object (141) may include a layout that displays screens in each of non-overlapping areas within the display (110), such as PBP, as well as a layout that displays other screens in an overlapping manner on a specific screen, such as PIP.
[0223] FIG. 16 is a block diagram of an electronic device (1601) within a network environment (1600) according to various embodiments. The electronic device (1601) of FIG. 16 may be an example of the electronic device (101) of FIGS. 1 to 15. Referring to FIG. 16, in the network environment (1600), the electronic device (1601) may communicate with the electronic device (1602) via a first network (1698) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (1604) or the server (1608) via a second network (1699) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1601) may communicate with the electronic device (1604) via the server (1608). According to one embodiment, the electronic device (1601) may include a processor (1620), a memory (1630), an input module (1650), an audio output module (1655), a display module (1660), an audio module (1670), a sensor module (1676), an interface (1677), a connection terminal (1678), a haptic module (1679), a camera module (1680), a power management module (1688), a battery (1689), a communication module (1690), a subscriber identification module (1696), or an antenna module (1697). In some embodiments, the electronic device (1601) may omit at least one of these components (e.g., the connection terminal (1678)), or may have one or more other components added. In some embodiments, some of these components (e.g., sensor module (1676), camera module (1680), or antenna module (1697)) may be integrated into a single component (e.g., display module (1660)).
[0224] The processor (1620) may control at least one other component (e.g., a hardware or software component) of the electronic device (1601) connected to the processor (1620) by executing, for example, software (e.g., a program (1640)), and may perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (1620) may store commands or data received from other components (e.g., a sensor module (1676) or a communication module (1690)) in a volatile memory (1632), process the commands or data stored in the volatile memory (1632), and store result data in a non-volatile memory (1634). According to one embodiment, the processor (1620) may include a main processor (1621) (e.g., a central processing unit or an application processor) or an auxiliary processor (1623) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (1621). For example, when the electronic device (1601) includes the main processor (1621) and the auxiliary processor (1623), the auxiliary processor (1623) may be configured to use less power than the main processor (1621) or to be specialized for a given function. The auxiliary processor (1623) may be implemented separately from the main processor (1621) or as a part thereof.
[0225] The auxiliary processor (1623) may control at least a portion of functions or states associated with at least one component (e.g., the display module (1660), the sensor module (1676), or the communication module (1690)) of the electronic device (1601), for example, on behalf of the main processor (1621) while the main processor (1621) is in an inactive (e.g., sleep) state, or together with the main processor (1621) while the main processor (1621) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (1623) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (1680) or a communication module (1690)). In one embodiment, the auxiliary processor (1623) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (1601) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (1608)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.
[0226] The memory (1630) can store various data used by at least one component (e.g., the processor (1620) or the sensor module (1676)) of the electronic device (1601). The data can include, for example, software (e.g., the program (1640)) and input data or output data for commands related thereto. The memory (1630) can include volatile memory (1632) or non-volatile memory (1634).
[0227] The program (1640) may be stored as software in memory (1630) and may include, for example, an operating system (1642), middleware (1644), or an application (1646).
[0228] The input module (1650) can receive commands or data to be used in a component of the electronic device (1601) (e.g., a processor (1620)) from an external source (e.g., a user) of the electronic device (1601). The input module (1650) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).
[0229] The audio output module (1655) can output audio signals to the outside of the electronic device (1601). The audio output module (1655) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.
[0230] The display module (1660) can visually provide information to an external party (e.g., a user) of the electronic device (1601). The display module (1660) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (1660) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.
[0231] The audio module (1670) can convert sound into an electrical signal, or vice versa. According to one embodiment, the audio module (1670) can acquire sound through the input module (1650), output sound through the sound output module (1655), or an external electronic device (e.g., electronic device (1602)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (1601).
[0232] The sensor module (1676) can detect the operating status (e.g., power or temperature) of the electronic device (1601) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (1676) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.
[0233] The interface (1677) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (1601) with an external electronic device (e.g., the electronic device (1602)). In one embodiment, the interface (1677) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
[0234] The connection terminal (1678) may include a connector through which the electronic device (1601) may be physically connected to an external electronic device (e.g., the electronic device (1602)). In one embodiment, the connection terminal (1678) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).
[0235] The haptic module (1679) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (1679) may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
[0236] The camera module (1680) can capture still images and videos. In one embodiment, the camera module (1680) may include one or more lenses, image sensors, image signal processors, or flashes.
[0237] The power management module (1688) can manage the power supplied to the electronic device (1601). According to one embodiment, the power management module (1688) can be implemented as at least a part of, for example, a power management integrated circuit (PMIC).
[0238] A battery (1689) may power at least one component of the electronic device (1601). In one embodiment, the battery (1689) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.
[0239] The communication module (1690) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (1601) and an external electronic device (e.g., electronic device (1602), electronic device (1604), or server (1608)), and the performance of communication through the established communication channel. The communication module (1690) may operate independently from the processor (1620) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1690) may include a wireless communication module (1692) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (1694) (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device (1604) via a first network (1698) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (1699) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network or a wide area network)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1692) may use subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (1696) to identify or authenticate the electronic device (1601) within a communication network such as the first network (1698) or the second network (1699).
[0240] The wireless communication module (1692) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (1692) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (1692) may support various technologies for securing performance in high-frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (1692) may support various requirements specified in the electronic device (1601), an external electronic device (e.g., the electronic device (1604)), or a network system (e.g., the second network (1699)). According to one embodiment, the wireless communication module (1692) may support a peak data rate (e.g., 20 Gbps or more) for eMBB implementation, a loss coverage (e.g., 164 dB or less) for mMTC implementation, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC implementation.
[0241] The antenna module (1697) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (1697) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (1697) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (1698) or the second network (1699), may be selected from the plurality of antennas by, for example, the communication module (1690). A signal or power may be transmitted or received between the communication module (1690) and the external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (1697).
[0242] According to various embodiments, the antenna module (1697) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.
[0243] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).
[0244] According to one embodiment, commands or data may be transmitted or received between the electronic device (1601) and an external electronic device (1604) via a server (1608) connected to a second network (1699). Each of the external electronic devices (1602 or 1504) may be the same or a different type of device as the electronic device (1601). According to one embodiment, all or part of the operations executed in the electronic device (1601) may be executed in one or more of the external electronic devices (1602, 1504, or 1508). For example, when the electronic device (1601) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (1601) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (1601). The electronic device (1601) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (1601) may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (1604) may include an Internet of Things (IoT) device. The server (1608) may be an intelligent server utilizing machine learning and / or a neural network.According to one embodiment, an external electronic device (1604) or server (1608) may be included within the second network (1699). The electronic device (1601) may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technology.
[0245] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.
[0246] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.
[0247] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0248] Various embodiments of the present document may be implemented as software (e.g., a program (1640)) including one or more instructions stored in a storage medium (e.g., an internal memory (1636) or an external memory (1638)) readable by a machine (e.g., an electronic device (1601)). For example, a processor (e.g., a processor (1620)) of the machine (e.g., an electronic device (1601)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.
[0249] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.
[0250] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
[0251] In one embodiment, a method for displaying visual objects to provide multiple screens simultaneously may be required. As described above, according to one embodiment, an electronic device (e.g., electronic device (101) of FIGS. 1 to 15) may include a display (e.g., display (110) of FIGS. 1 to 15), a memory (e.g., memory (215) of FIG. 2), and a processor (e.g., processor (210) of FIG. 2). The processor may be configured to display a plurality of visual objects on the display (e.g., visual objects (131, 132, 133) of FIGS. 1, 4A to 4C, 6A to 6B, 7, 9, 11A to 11B, and visual objects (511, 512, 513) of FIGS. 5A to 5D). The processor may be configured to identify a first input indicating selection of a first visual object from among the plurality of visual objects. The processor may be configured to, in response to a second input for a second visual object from among the plurality of visual objects, move the position of the second visual object from a first position to a second position in response to the second input. The processor may be configured to display a third visual object (e.g., visual objects (141, 142, 143) of FIG. 1 , visual objects (434, 444, 454) of FIG. 4a , visual objects (525, 526, 527) of FIG. 5b , visual objects (533, 534) of FIG. 5c , visual object (741) of FIG. 7 , visual object (1130) of FIGS. 11a-11b ) on the display based on identifying that the second visual object within the second location at least partially overlaps the first portion of the first visual object.The processor may be configured to, in response to a third input indicating selection of the third visual object (e.g., visual objects (141, 142, 143) of FIG. 1, visual objects (434, 444, 454) of FIG. 4a, visual objects (525, 526, 527) of FIG. 5b, visual objects (533, 534) of FIG. 5c, visual object (741) of FIG. 7, visual object (1130) of FIGS. 11a-11b), display a first screen corresponding to the first visual object within a first area of the display, and display a second screen corresponding to the second visual object within a second area of the display as a split-view. The processor may be configured to display a fourth visual object (e.g., visual object 144 of FIG. 1 , visual object 630 of FIG. 6A , visual object 750 of FIG. 7 ) on the display based on identifying that the second visual object within the second location overlaps a second portion of the first visual object that is different from the first portion. The processor may be configured to display the second screen as an overlay on the first screen in response to a fourth input indicating selection of the third visual object. In one embodiment, the electronic device may adjust the layout of screens to be displayed based on the visual object among PBP and / or PIP.
[0252] For example, the processor may be configured to, in response to identifying that the second visual object within the second location overlaps the second portion that includes a vertex of the first visual object, display the fourth visual object for displaying the second screen superimposed on the first screen.
[0253] For example, the processor may be configured to, in response to a fourth input indicating selection of the fourth visual object, display the second screen as an overlay as a pop-up window on the first screen occupying the entire active area of the display.
[0254] For example, the processor may be configured to display the third visual object for displaying the first screen corresponding to the first visual object and the second screen corresponding to the second visual object using a layout based on the edge, based on identifying that the second visual object within the second location overlaps the first portion including an edge connecting vertices of the first visual object.
[0255] For example, the processor may be configured to, in response to the third input, display the first screen and the second screen in each of the regions of the active area of the display divided along the direction of the edge.
[0256] For example, the processor may be configured to, in response to the first input, enlarge the first visual object among the plurality of visual objects. The processor may be configured to, based on identifying that the second visual object has been moved to a position adjacent to the enlarged first visual object in response to the second input, display a fourth visual object including the first visual object and the second visual object arranged based on the position.
[0257] For example, the processor may be configured to, in response to identifying a release of the second input, display the third visual object for providing the first screen and the second screen according to the layout corresponding to the position of the second visual object overlapping the fourth visual object at the timing when the second input is released, within a state where the second position of the second visual object overlaps the fourth visual object.
[0258] For example, the processor may be configured to, in response to identifying a release of the second input within a state where the second location of the second visual object is within the enlarged first visual object, create a directory in the memory of the electronic device containing all software applications corresponding to the first visual object and the second visual object.
[0259] For example, the processor may be configured to display a menu comprising a plurality of options based on identifying a release of the second input within a state where the second location of the second visual object at least partially overlaps the first visual object. The menu may include a fifth visual object (e.g., visual object (1152) of FIG. 11B ) corresponding to an option for displaying the third visual object. The menu may include a sixth visual object (e.g., visual object (1151) of FIG. 11B ) corresponding to an option for creating a directory comprising all software applications corresponding to the first visual object and the second visual object.
[0260] For example, the processor may be configured to, in response to a fifth input indicating a change in the layout of the screens to be provided via the third visual object, display one or more visual objects representing options for changing the layout, at locations at least partially overlapping the third visual object.
[0261] For example, the processor may be configured to, in response to identifying the third input, assign each of the screens to each display of the plurality of external displays using information representing the arrangement of the plurality of external displays, while the plurality of external displays (e.g., the external displays 1310 and 1320 of FIGS. 13A and 13B) are connected to the electronic device.
[0262] According to one embodiment, a method of an electronic device as described above may include an operation of displaying a plurality of visual objects on a display of the electronic device (e.g., operation 310 of FIG. 3 ). The method may include an operation of identifying a first input indicating a selection of a first visual object from among the plurality of visual objects (e.g., operation 320 of FIG. 3 ). The method may include an operation of moving a location of a second visual object from a first location to a second location in response to a second input indicating a movement of a second visual object from among the plurality of visual objects (e.g., operation 330 of FIG. 3 ). The method may include an operation of displaying a third visual object based on identifying that the second location of the second visual object at least partially overlaps a first portion of the first visual object. The method may further include, in response to a third input indicating selection of the third visual object, displaying a first screen corresponding to the first visual object within a first area of the display and displaying a second screen corresponding to the second visual object within a second area of the display as a split-view. The method may further include, in response to a fourth input indicating selection of the fourth visual object, displaying the second screen as an overlay on the first screen based on identifying that the second visual object within the second location overlaps a second portion of the first visual object that is different from the first portion. The method may further include, in response to a fourth input indicating selection of the fourth visual object, displaying the second screen as an overlay on the first screen.
[0263] For example, the act of displaying the third visual object may include an act of displaying the fourth visual object to display the second screen corresponding to the second visual object as an overlay on the first screen corresponding to the first visual object, based on identifying that the second visual object within the second location overlaps the second portion including a vertex of the first visual object.
[0264] For example, the act of displaying the screens may include, in response to a fourth input indicating selection of the fourth visual object, displaying the second screen as an overlay as a pop-up window for the first screen occupying the entire active area of the display.
[0265] For example, the act of displaying the third visual object may include, in response to identifying that the second visual object in the second location overlaps the first portion including an edge connecting vertices of the first visual object, displaying a third visual object for displaying a first screen corresponding to the first visual object and a second screen corresponding to the second visual object using a layout based on the edge. The act of displaying the screens may include, in response to the third input, displaying the first screen and the second screen in respective regions of an active area of the display divided along a direction of the edge.
[0266] For example, the operation of identifying the first input may include an operation of enlarging the first visual object among the plurality of visual objects in response to the first input. The method may include an operation of displaying a fourth visual object having a shape including the first visual object and the position based on identifying that the second visual object moved in response to the second input has moved to a position adjacent to the enlarged first visual object. The operation of displaying the third visual object may include an operation of displaying the third visual object for providing the screens according to the layout corresponding to the position of the second visual object overlaid on the fourth visual object at the timing when the second input is released, based on identifying a release of the second input while the second visual object is overlaid on the fourth visual object.
[0267] For example, the method may include an action of creating a directory in a memory of the electronic device, the directory containing all software applications corresponding to the first visual object and the second visual object, based on identifying a release of the second input while the second visual object is moved into the enlarged first visual object according to the second input.
[0268] For example, the changing action may include an action of displaying a menu including a plurality of options based on identifying a release of the second input, wherein the second visual object is at least partially overlapping the first visual object in response to the second input. The displaying action may include an action of displaying the menu including a fifth visual object corresponding to an option for displaying the third visual object and a sixth visual object corresponding to an option for creating a directory including all software applications corresponding to the first visual object and the second visual object.
[0269] For example, the act of displaying the third visual object may include, in response to a fourth input indicating a change in the layout of the screens to be provided through the third visual object, displaying one or more visual objects representing options for changing the layout at positions at least partially overlapping the third visual object.
[0270] For example, the act of displaying the screens may include an act of assigning each of the screens to the plurality of external displays using information representing an arrangement of the plurality of external displays based on identifying a third input indicating selection of the third visual object while the plurality of external displays are connected to the electronic device.
[0271] According to one embodiment, as described above, an electronic device may include a display and a processor. The processor may be configured to display a visual object for providing a plurality of screens simultaneously within a list of software applications while displaying the list. The processor may be configured to, in response to an input indicating a selection of the visual object, display a first screen among the plurality of screens based on a size of an active area of the display. The processor may be configured to, in response to the input, display a second screen among the plurality of screens in an overlapping manner on the first screen displayed in the active area.
[0272] For example, the processor may be configured to display the visual object based on the size of an image within the list representing the software application.
[0273] For example, the processor may be configured to display the visual object, which includes a first image corresponding to the first screen, a second image having a smaller size than the first image, and corresponding to the second screen.
[0274] For example, the processor may be configured to display the second image having a position overlapping a vertex of the first image. The processor may be configured, in response to the input, to display the second screen at a position adjacent to a vertex of the first screen corresponding to the vertex of the first image over which the second image is overlaid.
[0275] According to one embodiment of the present invention, a method of an electronic device as described above may include an operation of displaying a visual object for providing a plurality of screens simultaneously within a list of software applications installed in the electronic device while displaying the list. The method may include an operation of displaying a first screen among the plurality of screens based on a size of an active area of a display of the electronic device in response to an input indicating a selection of the visual object. The method may include an operation of displaying a second screen among the plurality of screens in an overlapping manner on the first screen displayed in the active area in response to the input.
[0276] For example, the act of displaying the visual object may include an act of displaying the visual object based on the size of an image in the list representing the software application.
[0277] For example, the action of displaying the visual object may include an action of displaying the visual object, the visual object including a first image corresponding to the first screen, having a smaller size than the first image, and including a second image corresponding to the second screen.
[0278] For example, the action of displaying the visual object may include an action of displaying the second image having a position overlapping a vertex of the first image. The action of displaying the second screen may include an action of displaying the second screen, in response to the input, at a position adjacent to a vertex of the first screen corresponding to the vertex of the first image over which the second image is overlaid.
[0279] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.
[0280] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.
[0281] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.
[0282] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.
[0283] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.
[0284] The embodiments described above and their technical features can be combined with each other in each and every combination, as long as there is no conflict between the two embodiments or features. That is, each and every combination of two or more of the embodiments described above are included and contemplated by the present disclosure. One or more features from one embodiment can be integrated with another embodiment and provide corresponding advantages or advantages.
[0285] Additional aspects of the invention are set forth in the numbered sections that follow.
[0286] Verse 1
[0287] In electronic devices,
[0288] display (110);
[0289] touch sensor (227); and
[0290] It includes a processor (210), and the processor,
[0291] Displaying a plurality of visual objects (131, 132, 133; 511, 512, 513) on the display, wherein the visual objects display applications executable by the electronic device;
[0292] Identifying a first touch on the visual object, wherein the first touch represents a first input for selecting a first visual object from the plurality of visual objects;
[0293] Identifying a second touch on the visual object, wherein the second touch represents a second input for selecting a second visual object from the plurality of visual objects;
[0294] Identifying movement of the second touch from a first location corresponding to the location of the second visual object to a second location;
[0295] In response to the identified movement of the second touch, moving the second visual object from the first location to the second location; and
[0296] In response to determining that the second visual object within the second location at least partially overlaps the first location, generating and displaying a third visual object;
[0297] In response to identifying a third touch on the display, the third touch represents a third input for selecting the third visual object, configured to display a screen of a first application associated with the first visual object displayed on a first portion of the display and a second application associated with the second visual object displayed on a second portion of the display.
[0298] Electronic devices.
[0299] Section 2
[0300] In the electronic device of claim 1, the processor is further configured to perform one or more of: identifying the second touch, identifying movement of the second touch, and moving the second visual object from the first location to the second location while the first touch on the first visual object is maintained.
[0301] Electronic devices.
[0302] Verse 3
[0303] In the electronic device of clause 1 or clause 2, the third visual object is configured according to the position of the overlapping portion of the first visual object with respect to the first visual object.
[0304] Electronic devices.
[0305] Verse 4
[0306] In any electronic device among clauses 1 to 3, the third visual object includes an image representing the first visual object and an image representing the second visual object.
[0307] Electronic devices.
[0308] Verse 5
[0309] In the electronic device of clause 4, the image representing the first visual object includes a small version of the first visual object, and the image representing the second visual object includes a small version of the second visual object.
[0310] Electronic devices.
[0311] Verse 6
[0312] In the electronic device of clause 4 or clause 5, the image representing the first visual object and the image representing the second visual object within the third visual object indicate an arrangement of the first part of the display with respect to a position of the second part of the display when the third visual object is selected.
[0313] Electronic devices.
[0314] Verse 7
[0315] In the electronic device of section 6, the processor,
[0316] Identifying a fourth touch on the third visual object, wherein the fourth touch is maintained for at least a predetermined period of time;
[0317] Identifying the movement of the fourth touch; and
[0318] Further configured to change the relative positions of the image representing the first visual object and the image representing the second visual object based on the identified movement of the fourth touch.
[0319] Electronic devices.
[0320] Verse 8
[0321] In the electronic device of section 7,
[0322] The movement of the fourth touch includes a movement indicating rotation of the third visual object, and
[0323] In response to the movement of the fourth touch, the positions of the image representing the first visual object and the image representing the second visual object within the third visual object are exchanged.
[0324] Electronic devices.
[0325] Verse 9
[0326] In any of the electronic devices described above, in response to determining that the second visual object within the second location at least partially overlaps the second location of the first visual object, generating and displaying the fourth visual object;
[0327] In response to identifying a fourth touch on the display, the fourth touch indicates a fourth input for selecting the fourth visual object, and displays the screen of the second application superimposed on a portion of the screen of the first application; and
[0328] The second position of the first visual object is different from the first part of the first visual object,
[0329] Electronic devices.
Claims
1. In an electronic device (101), display (110); A memory (215) comprising one or more storage media for storing instructions; and An electronic device comprising at least one processor (210) including a processing circuit, wherein the instructions, when individually and / or collectively executed by the at least one processor, Displaying a plurality of visual objects (131, 132, 133; 511, 512, 513) on the display; Identifying a first input indicating a selection of a first visual object from among the plurality of visual objects; In response to a second input for a second visual object among the plurality of visual objects, moving the position of the second visual object from a first position to a second position according to the second input; Based on identifying that the second visual object in the second location at least partially overlaps the first portion of the first visual object, displaying a third visual object (141, 142, 143; 434, 444, 454; 525, 526, 527; 533, 534; 741; 1130) on the display, and in response to a third input indicating selection of the third visual object (141, 142, 143; 434, 444, 454; 525, 526, 527; 533, 534; 741; 1130), configured to display a first screen corresponding to the first visual object within a first area of the display and a second screen corresponding to the second visual object within a second area of the display as a split-view; and configured to display a fourth visual object (144; 630; 750) on the display based on identifying that the second visual object within the second location overlaps a second portion of the first visual object that is different from the first portion. Electronic devices.
2. In claim 1, the processor, Further configured to display the fourth visual object for displaying the second screen superimposed on the first screen in response to identifying that the second visual object within the second location overlaps the second portion including a vertex of the first visual object, Electronic devices.
3. In claim 1 or claim 2, the processor, In response to a fourth input indicating selection of the fourth visual object, the second screen is further configured to be displayed as an overlay as a pop-up window on the first screen, occupying the entire active area of the display. Electronic devices.
4. In claims 1 to 3, the processor, Further configured to display the third visual object for displaying the first screen corresponding to the first visual object and the second screen corresponding to the second visual object using a layout based on the edge, based on identifying that the second visual object in the second location overlaps the first portion including an edge connecting the vertices of the first visual object. Electronic devices.
5. In claim 4, the processor, In response to the third input, the first screen and the second screen are further configured to be displayed in each of the regions of the active area of the display divided along the direction of the edge. Electronic devices.
6. In claims 1 to 5, the processor, In response to the first input, enlarging the first visual object among the plurality of visual objects; and Further configured to display a fourth visual object including the first visual object and the second visual object arranged based on the position, based on identifying that the second visual object is moved to a position adjacent to the enlarged first visual object according to the second input. Electronic devices.
7. In claims 1 to 6, the processor, Further configured to, in response to identifying the release of the second input, display the third visual object for providing the first screen and the second screen according to the layout corresponding to the position of the second visual object superimposed on the fourth visual object at the timing when the second input is released, within a state where the second position of the second visual object overlaps the fourth visual object. Electronic devices.
8. In claim 5, the processor, Further configured to, in response to identifying the release of the second input, within a state where the second location of the second visual object is within the enlarged first visual object, generate a directory in the memory of the electronic device, the directory including all software applications corresponding to the first visual object and the second visual object. Electronic devices.
9. In claims 1 to 8, the processor, Further configured to display a menu including a plurality of options based on identifying a release of the second input, within a state where the second location of the second visual object at least partially overlaps the first visual object; The above menu is, A fifth visual object (1152) corresponding to an option for displaying the third visual object; and A sixth visual object (1151) corresponding to an option for creating a directory containing all software applications corresponding to the first visual object and the second visual object, Electronic devices.
10. In claims 1 to 9, the processor, In response to a fifth input indicating a change in the layout of the screens to be provided through the third visual object, further configured to display one or more visual objects indicating an option for changing the layout, at positions at least partially overlapping the third visual object. Electronic devices.
11. In claims 1 to 10, the processor, Further configured to assign each of the screens to each display of the plurality of external displays (1310, 1320), in response to identifying the third input, while the plurality of external displays (1310, 1320) are connected to the electronic device, using information indicating the arrangement of the plurality of external displays. Electronic devices.
12. In the method of an electronic device, An action of displaying a plurality of visual objects on a display of said electronic device; An action for identifying a first input indicating a selection of a first visual object from among the plurality of visual objects; An action of moving a position of a second visual object from a first position to a second position in response to a second input for a second visual object among the plurality of visual objects; Based on identifying that the second location of the second visual object at least partially overlaps the first portion of the first visual object, the method further comprises the action of displaying a third visual object; in response to a third input indicating selection of the third visual object, the method further comprises the action of displaying a first screen corresponding to the first visual object, within a first area of the display, and displaying a second screen corresponding to the second visual object, within a second area of the display, as a split-view; and An operation of displaying a fourth visual object on the display based on identifying that the second visual object within the second location overlaps a second portion of the first visual object that is different from the first portion, method.
13. In claim 12, the action of displaying the third visual object is: An operation of displaying the fourth visual object for displaying the second screen corresponding to the second visual object as an overlay on the first screen corresponding to the first visual object based on identifying that the second visual object within the second location overlaps the second portion including the vertex of the first visual object, method.
14. In claims 12 to 13, the operation of displaying the screens comprises: In response to a fourth input indicating selection of the fourth visual object, the method comprises displaying the second screen as an overlay as a pop-up window on the first screen occupying the entire active area of the display. method.
15. In claims 12 to 14, the action of displaying the third visual object comprises: In response to identifying that the second visual object within the second location overlaps the first portion including an edge connecting vertices of the first visual object, the method comprises displaying a third visual object for displaying a first screen corresponding to the first visual object and a second screen corresponding to the second visual object using a layout based on the edge, The actions that display the above screens are: In response to the third input, the method comprises displaying the first screen and the second screen in each area of the active area of the display divided along the direction of the edge. method.
Citation Information
Patent Citations
Mobile terminal and multi-application operation method thereof
KR1020140106193A
Mobile terminal and method for controlling the same
KR1020170011539A
Electronic apparatus and method for displaying and transmitting image thereof
KR1020170102634A
Composition for enamel, method for preparation thereof and cooking appliance
KR1020200102758A
Display apparatus for excuting plurality of applications and method for controlling thereof
KR102183071B1
