Method and electronic device for performing context-based actions

By selecting and dragging and dropping graphic objects in electronic devices, identifying contexts and executing actions, the problem of cumbersome content sharing steps in the prior art is solved, and user experience and operation efficiency are improved.

CN112424740BActive Publication Date: 2024-11-26SAMSUNG ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980048064.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-07-25
Filing Date
2019-07-25
Publication Date
2024-11-26
Estimated Expiration
2039-07-25

AI Technical Summary

Technical Problem

Existing electronic devices require users to perform multiple steps when sharing content, resulting in increased time consumption and inconvenience to users.

Method used

The content sharing process is simplified by selecting and dragging and dropping graphic objects in an electronic device, identifying relevant contexts and performing actions based on that context.

Benefits of technology

Reduces user operation steps, improves communication convenience between applications, and enhances user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112424740B_ABST
    Figure CN112424740B_ABST
Patent Text Reader

Abstract

A system and method for performing context-based actions in an electronic device is provided. The method includes: selecting one or more first graphical objects in the electronic device; detecting a drag-and-drop event for the selected one or more first graphical objects, the drag-and-drop event involving manipulating the one or more first graphical objects to place them on one or more second graphical objects; in response to the detected drag-and-drop event, identifying a context associated with each of the one or more first graphical objects and the one or more second graphical objects, wherein the context of the one or more first graphical objects is shared with the one or more second graphical objects. In addition, the method includes performing one or more actions in the electronic device based on the identified context.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an electronic device. More specifically, the present disclosure relates to a method and an electronic device for performing a context-based action. Background Art

[0002] Generally, electronic devices dominate every aspect of modern life. Over a period of time, the way electronic devices display information on user interfaces has become smart, efficient, spontaneous, and unobtrusive. A user of an electronic device may often want to share some interesting content with his or her contacts while communicating through the electronic device. To facilitate content sharing, many applications can implement the functionality of content sharing.

[0003] In some existing methods, content sharing is performed through a series of operations described herein. The user opens a page in an application and selects content to be shared. In addition, the user clicks a share button on a menu, and after the electronic device receives a signal that the user clicks the share button, a share page corresponding to the page is displayed, which guides the user to edit the content to be shared in the share page. Finally, the content to be shared in the share page is sent to a server so that the server can share the content.

[0004] As an app has more and more functions, it accordingly comes with more and more app buttons. Therefore, in order to share content, the user needs to perform several operations, which ultimately takes longer time and may be troublesome for the user.

[0005] The above information is presented as background information and is only intended to assist in understanding the present disclosure. No determination or assertion is made that any of the above may be used as prior art with respect to the present disclosure. Summary of the invention

[0006] Various aspects of the present disclosure are to address at least the above problems and / or disadvantages and to provide at least the advantages described below. Therefore, one aspect of the present disclosure is to provide a method for performing context-based actions in an electronic device.

[0007] Another aspect of the present disclosure is to provide a method for selecting one or more first graphic objects in an electronic device.

[0008] Another aspect of the present disclosure is to provide a method for detecting a drag and drop event on selected one or more first graphic objects, the drag and drop event involving manipulating the one or more first graphic objects to place them on one or more second graphic objects.

[0009] Another aspect of the present disclosure is to provide a method for identifying a context associated with each of one or more first graphical objects and one or more second graphical objects in response to a detected drag and drop event.

[0010] Another aspect of the present disclosure is to provide a method for sharing a context of one or more first graphic objects with one or more second graphic objects.

[0011] Another aspect of the present disclosure is to provide a method for performing one or more actions in an electronic device based on an identified context.

[0012] According to another aspect of the present disclosure, a method for performing a context-based action in an electronic device is provided. The method includes: selecting one or more first graphic objects in the electronic device; detecting a drag-and-drop event for the selected one or more first graphic objects, the drag-and-drop event involving manipulating the one or more first graphic objects to place them on one or more second graphic objects; in response to the detected drag-and-drop event, identifying a context associated with each of the one or more first graphic objects and the one or more second graphic objects, wherein the context of the one or more first graphic objects is shared with the one or more second graphic objects; and performing one or more actions in the electronic device based on the identified context.

[0013] According to another aspect of the present disclosure, a method for performing a context-based action in an electronic device is provided. The method includes: selecting one or more first graphic objects in the electronic device; detecting a drag event of the selected one or more first graphic objects, the drag event involving manipulating the one or more first graphic objects; identifying a context associated with the one or more first graphic objects in response to the detected drag event; displaying one or more second graphic objects based on the identified context; and performing one or more actions in the electronic device based on the selection of the one or more second graphic objects.

[0014] According to another aspect of the present disclosure, a method for performing context-based actions in an electronic device is provided. The method includes: selecting one or more first graphic objects representing one or more applications in the electronic device; detecting a drag and drop event for the selected one or more first graphic objects; identifying a content type associated with the selected one or more first graphic objects; in response to the detected drag and drop event, determining one or more second graphic objects for performing one or more actions based on the identified content type; displaying the one or more second graphic objects; and performing the one or more actions in the electronic device based on the selection of the one or more second graphic objects.

[0015] According to another aspect of the present disclosure, an electronic device for performing context-based actions is provided. The electronic device includes a memory, at least one processor coupled to the memory, and a context action engine coupled to the memory and the at least one processor. The context action engine is configured to: select one or more first graphic objects in the electronic device; detect a drag and drop event for the selected one or more first graphic objects, the drag and drop event involving manipulating the one or more first graphic objects to place them on one or more second graphic objects; in response to the detected drag and drop event, identify a context associated with each of the one or more first graphic objects and the one or more second graphic objects, wherein the context of the one or more first graphic objects is shared with the one or more second graphic objects. The context action engine is also configured to: perform one or more actions in the electronic device based on the identified context.

[0016] According to another aspect of the present disclosure, an electronic device for performing context-based actions is provided. The electronic device includes a memory, at least one processor, and a context action engine coupled to the memory and the at least one processor. The context action engine is configured to: select one or more first graphic objects in the electronic device; detect a drag event of the selected one or more first graphic objects, the drag event involving manipulation of the one or more first graphic objects; in response to the detected drag event, identify a context associated with the one or more first graphics; display one or more second graphic objects based on the identified context; and perform one or more actions in the electronic device based on the selection of the one or more second graphic objects.

[0017] According to another aspect of the present disclosure, an electronic device for performing context-based actions is provided. The electronic device includes a memory, at least one processor, and a context action engine coupled to the memory and the processor. The context action engine is configured to: select one or more first graphic objects representing one or more applications in the electronic device; detect a drag and drop event for the selected one or more first graphic objects; identify the content type associated with the selected one or more first graphic objects; in response to the detected drag and drop event, determine one or more second graphic objects for performing one or more actions based on the identified content type; display the one or more second graphic objects; and based on the selection of the one or more second graphic objects, perform the one or more actions in the electronic device.

[0018] Other aspects, advantages, and salient features of the present disclosure will become apparent to those skilled in the art from the following detailed description of various embodiments of the present disclosure in conjunction with the accompanying drawings.

[0019] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other aspects, features and advantages of some embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0021] Figure 1A , Figure 1B and Figure 1C is an example illustration of an electronic device performing a context-based action according to various embodiments of the present disclosure;

[0022] Figure 2 is a block diagram of an electronic device according to an embodiment of the present disclosure;

[0023] Figure 3 is a flowchart of a method for performing a context-based action in an electronic device according to an embodiment of the present disclosure;

[0024] Figure 4A , Figure 4B , Figure 4C , Figure 4D , Figure 4E and Figure 4F is an example illustration for sharing content by dragging and dropping an application icon onto another application icon according to various embodiments of the present disclosure;

[0025] Figure 5 is an example diagram of an electronic device displaying multiple restaurants based on location according to an embodiment of the present disclosure;

[0026] Figure 6 is an example diagram of an electronic device performing an action when an application icon is dragged and dropped onto one or more application icons according to an embodiment of the present disclosure;

[0027] Figure 7 is an example diagram for sharing content by dragging and dropping a notification panel icon onto an application icon according to an embodiment of the present disclosure;

[0028] Fig. 8A , Figure 8B and Figure 8C is an example illustration of an electronic device performing a context-based action in an augmented reality (AR) mode according to various embodiments of the present disclosure;

[0029] Fig. 9 is a flowchart of a method for performing a context-based action in an electronic device based on a drag event of an application icon according to an embodiment of the present disclosure;

[0030] Fig.10is an example diagram of an electronic device displaying one or more actions based on an identified context according to an embodiment of the present disclosure;

[0031] Fig.11 is an example diagram of an electronic device displaying one or more actions when a drag gesture is performed on an application being accessed at the electronic device according to an embodiment of the present disclosure;

[0032] Fig.12 is a block diagram of a context action engine in an electronic device according to an embodiment of the present disclosure;

[0033] Fig.13 The action of registering an application based on content type according to an embodiment of the present disclosure is shown;

[0034] Fig.14 The invention shows that the content of an application is transmitted based on the context of the application according to an embodiment of the present disclosure;

[0035] Fig.15 Various operations for determining one or more related applications for performing one or more actions according to an embodiment of the present disclosure are shown;

[0036] Fig.16 is an example diagram of an electronic device displaying multiple applications related to an application selected for performing one or more actions according to an embodiment of the present disclosure;

[0037] Fig.17 Various operations performed when an application icon is dragged near another application icon according to an embodiment of the present disclosure are shown;

[0038] Fig.18 is an example diagram of an electronic device displaying a recommendation when an application icon is dragged and dropped onto one or more application icons according to an embodiment of the present disclosure;

[0039] Fig.19 is an example illustration of performing various actions on a connected device associated with an electronic device according to an embodiment of the present disclosure;

[0040] Fig. 20A and Fig. 20B is an example illustration of performing various actions on an IoT device in a smart home environment according to various embodiments of the present disclosure; and

[0041] Fig.21 It is an example diagram of executing actions in an electronic device based on the context of multiple applications according to an embodiment of the present disclosure.

[0042] Throughout the drawings, like reference numerals will be understood to refer to like parts, components, and structures. DETAILED DESCRIPTION

[0043] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. The following description includes various specific details to assist in understanding, but these specific details should be considered as merely exemplary. Therefore, it will be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, for the sake of clarity and brevity, descriptions of known functions and structures may be omitted.

[0044] The terms and words used in the following description and claims are not limited to their written meanings, but are merely used to achieve a clear and consistent understanding of the present disclosure. Therefore, it should be clear to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only, and not for the purpose of limiting the present disclosure as defined by the attached claims and their equivalents.

[0045] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.

[0046] The term "or" as used herein refers to a non-exclusive or, unless otherwise specified. The examples herein are intended only to facilitate understanding of the manner in which the embodiments are implemented, and to further enable those skilled in the art to implement the embodiments. Therefore, the examples should not be construed as limiting the scope of the embodiments herein.

[0047] As is common in the art, embodiments can be described and illustrated around blocks that perform desired functions. These blocks, which may be referred to herein as units, engines, managers, modules, etc., are physically implemented by analog and / or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hard-wired circuits, etc., and may be optionally driven by firmware and / or software. For example, the circuit may be specifically implemented in one or more semiconductor chips, or on substrate supports such as printed circuit boards, etc. The circuits constituting the blocks may be implemented by dedicated hardware or by a processor (e.g., one or more programmed microprocessors and associated circuits), or by a combination of dedicated hardware for performing some functions of the block and a processor for performing other functions of the block. Without departing from the scope of the present disclosure, each block of the embodiment may be physically divided into two or more interactive and discrete blocks. Similarly, without departing from the scope of the present disclosure, the blocks of the embodiment may be physically combined into more complex blocks.

[0048] Throughout the specification, the term "graphic object" refers to an application icon, a data item, a notification panel icon, a graphical user interface (GUI) element, a function icon, a service icon, a button, etc. The terms first graphical object and second graphical object are used for labeling purposes only and should not be construed as limiting the scope of the embodiments of the present disclosure.

[0049] Therefore, an embodiment of the present invention provides a method for performing context-based actions in an electronic device. The method includes selecting one or more first graphic objects in an electronic device. The method includes a method for detecting a drag-and-drop event for the selected one or more first graphic objects, the drag-and-drop event involving manipulating the one or more first graphic objects to place them on one or more second graphic objects. In addition, the method includes: in response to the detected drag-and-drop event, identifying a context associated with each of the one or more first graphic objects and the one or more second graphic objects, wherein the context of the one or more first graphic objects is shared with the one or more second graphic objects. In addition, the method includes performing one or more actions in the electronic device based on the identified context.

[0050] For example, consistent with one example embodiment, a user may press and hold his finger on a first graphical object (e.g., a content or video window) displayed on an electronic device while dragging or otherwise moving his finger to "pull" the first graphical object, and then lift his finger as if "dropping" the element on top of a second graphical object, a manipulation generally referred to as a "drag and drop" operation. By dragging and dropping a first graphical object on top of a second graphical object, the user may invoke various operations that make intuitive sense to the user based on the context represented by the graphical objects. For example, in some example embodiments, dragging and dropping a content or video window onto a graphical object representing a person or group of people (e.g., as may be the case with a friend list) may automatically invoke an operation to share the content or video displayed or presented in the content or video window of a content viewing application currently being executed on the electronic device with the specific person associated with the second graphical object.

[0051] In various embodiments, the method includes selecting one or more first graphical objects representing one or more applications in an electronic device. The method includes detecting a drag and drop event for the selected one or more first graphical objects. The method includes identifying a content type associated with the selected one or more first graphical objects. The method includes: in response to the detected drag and drop event, determining one or more second graphical objects based on the identified content type for performing the one or more actions. In addition, the method includes causing the one or more second graphical objects to be displayed. In addition, the method includes: performing one or more actions in the electronic device based on the selection of the one or more second graphical objects.

[0052] The proposed method enables users to perform contextual actions with fewer operations to enhance communication between applications. Using the proposed method, the context of application icons, notification panel icons can be identified, which enables electronic devices to perform context-based actions by dragging and dropping application icons on top of each other. Therefore, the proposed method provides an additional level of convenience for users, wherein users can perform various actions by dragging and dropping application icons on top of each other. Various example illustrations are provided in conjunction with the accompanying drawings to better understand the embodiments of the present disclosure.

[0053] Figure 1A , Figure 1B and Figure 1C 1 is an example diagram of an electronic device performing a context-based action according to various embodiments of the present disclosure. The electronic device may be, for example but not limited to, a cellular phone, a smart phone, a personal digital assistant (PDA), a tablet computer, a laptop computer, a smart watch, smart glasses, etc.

[0054] Reference Figure 1A In an embodiment, application icon 1 of electronic device 100 may be dragged and dropped onto application icon 2. When application icon 1 is dragged and dropped onto application icon 2, electronic device 100 performs one or more actions based on the context (i.e., current context) of application icon 1 and application icon 2. Based on the current context of the application icons (i.e., application icon 1 and application icon 2), an action is performed at electronic device 100 when the drop operation is completed. When application icon 1 is dragged and dropped onto application icon 2, the context of application icon 1 is shared with application icon 2 so that electronic device 100 can perform context-based actions.

[0055] Reference Figure 1B In another embodiment, the application icon can be dragged and dropped onto itself. Then, when the application icon 1 is dragged and dropped onto itself, the electronic device 100 performs one or more actions based on the context of the application icon. The electronic device 100 provides one or more recommended actions to the user based on the current context of the application icon.

[0056] Reference Figure 1C In another embodiment, the notification panel icon can be dragged and dropped onto the application icon. Then, when the notification icon is dragged and dropped onto the application icon, the electronic device 100 performs one or more actions based on the context of the notification panel icon and the application icon.

[0057] Therefore, by using the proposed method, the electronic device 100 performs an action based on the context of the application icon. Various actions performed by the electronic device 100 based on the context of the application icon are described in conjunction with the accompanying drawings.

[0058] Figure 2 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0059] Reference Figure 2 , the electronic device 100 includes an event detection engine 110, a context action engine 120, a communicator 130, a memory 150, at least one processor 140, a display screen 160 and a graphic object 170.

[0060] The event detection engine 110 may be configured to detect the occurrence of a drag event, a drop event, and a drag-and-drop event on the graphical object 170 .

[0061] The contextual action engine 120 may be configured to perform one or more actions based on the context of an application icon, notification panel icon, GUI element, etc. The contextual action engine 120 may be configured to identify the current context of an application icon and perform one or more actions based on the identified context.

[0062] In an embodiment, the contextual action engine 120 may be configured to identify the current context of an application icon by extracting the user's recent activities using graphical objects 170 such as application icons, notification panel icons, GUI elements, etc. Thus, the contextual action engine 120 may be configured to perform various actions based on the identified context.

[0063] The communicator 130 is configured to communicate internally between internal hardware components and to communicate with external devices via one or more networks. The communicator 130 is configured to communicate with the context action engine 120 to perform context-based actions in the electronic device 100 .

[0064] The processor 140 is coupled to the contextual action engine 120, the communicator 130, the memory 150, the display screen 160, and the graphic object 170. The processor 140 is configured to execute instructions stored in the memory 150 and perform various actions. The processor 140 in communication with the contextual action engine 120 obtains the context of the application icon, the notification panel icon, etc. When the user performs a drag and drop operation of an application icon onto another application icon, the contextual action engine 120 identifies the current context of the application icon and transmits the identified context of the application icon to the processor 140. Therefore, the processor 140 obtains the context of the application icon (i.e., the most recent activity) and executes one or more instructions to enable the electronic device 100 to perform context-based actions.

[0065] The memory 150 stores mappings of content-action and action-data packet names. The memory 150 also stores instructions to be executed by the processor 140. The memory 150 may include volatile storage elements or non-volatile storage elements or a combination thereof.

[0066] Various actions performed based on the identified context are displayed to the user on display screen 160 .

[0067] although Figure 2 Various hardware components of the electronic device 100 are shown, but it should be understood that other embodiments are not limited thereto. In other embodiments, the electronic device 100 may include fewer or more components. In addition, the labels or names of the components are only for illustrative purposes and do not limit the scope of the present disclosure. One or more components can be combined together to perform the same or substantially similar functions, thereby performing context-based actions in the electronic device 100.

[0068] Figure 3 is a flowchart of a method for performing context-based actions in an electronic device according to an embodiment of the present disclosure.

[0069] Reference Figure 3 , at operation 302 of flowchart 300, the method includes selecting one or more first graphical objects in the electronic device 100. At operation 304, the method includes detecting a drag and drop event for the selected one or more first graphical objects, the drag and drop event involving manipulating the one or more first graphical objects to place them on one or more second graphical objects.

[0070] At operation 306, the method includes identifying a context associated with each of the one or more first graphical objects and the one or more second graphical objects in response to the detected drag and drop event. At operation 308, the method includes performing one or more actions in the electronic device 100 based on the identified context.

[0071] The various actions, behaviors, blocks, operations, etc. in the flowchart diagram 300 may be performed in the order presented, in a different order, or simultaneously. Furthermore, in some embodiments, some actions, behaviors, blocks, operations, etc. may be omitted, added, modified, skipped, etc. without departing from the scope of the present disclosure.

[0072] Figure 4A , Figure 4B , Figure 4C , Figure 4D , Figure 4E and Figure 4F is an example illustration for sharing content by dragging and dropping an application icon onto another application icon according to various embodiments of the present disclosure. Figure 4B and Figure 4D The various application icons shown are for illustrative purposes only to describe the embodiments.

[0073] Reference Figure 4A, a conversation about song suggestions is ongoing in a chat application on the electronic device 100. Figure 4B As shown, the user is currently playing a song in a music application on another electronic device 100. The current context of the music application is the song playing in the music application, while the context of the chat application is the last message communicated in the chat thread. While listening to the song, the user realizes that the song was requested during the conversation, such as Figure 4C In addition, Figure 4D As shown, the user drags and drops the music application onto the chat application on the electronic device 100.

[0074] When the user drags and drops the music application onto the chat application, Figure 4D As shown, when the music application comes nearby while sharing, the graphical element is temporarily changed to represent the current context. Figure 4D As shown, when the chat application icon arrives near the music icon, a notification is displayed to the user indicating that a song is being shared through the chat application.

[0075] In addition, if Figure 4E As shown, the electronic device 100 displays a notification to the user to obtain user confirmation (in the chat application) for sharing the song with the recipient. Figure 4F As shown, when the user provides confirmation, the song is shared with the recipient.

[0076] Figure 5 is an example diagram of an electronic device displaying a plurality of restaurants based on locations according to an embodiment of the present disclosure.

[0077] Reference Figure 5 , drag and drop the location application icon onto the food application icon on the electronic device 100. When the location application is dragged and dropped onto the food application icon, the user's current location (i.e., the context of the location icon) is shared with the food application icon. Using the user's current location, the food application displays multiple restaurants near the user.

[0078] Figure 6 is an example illustration of an electronic device performing an action when an application icon is dragged and dropped onto one or more application icons according to an embodiment of the present disclosure.

[0079] Reference Figure 6 , the user is accessing a professional network application and is searching for various profiles on the electronic device 100. The user can drag and drop the professional network application icon onto the call icon or the email icon. When the professional network application icon is dragged and dropped onto the call icon, the electronic device 100 launches the call application with the mobile number of the profile that the user is accessing. In addition, Figure 6As shown, when the professional web application icon is dragged and dropped onto the email icon, the email application is launched with the email ID of the profile that the user is accessing.

[0080] Figure 7 is an example diagram for sharing content by dragging and dropping a notification panel icon onto an application icon according to an embodiment of the present disclosure.

[0081] Reference Figure 7 , you can drag and drop the notification panel icon (i.e., Bluetooth icon) onto the "My Files" icon. The current context of the Bluetooth icon includes the last connected device, and the current context of "My Files" is the last selected image set. When you drag and drop the Bluetooth icon onto the "My Files" icon, the image is shared with the last connected device via Bluetooth.

[0082] Fig. 8A , Figure 8B and Figure 8C It is an example illustration of an electronic device performing a context-based action in an augmented reality (AR) mode according to various embodiments of the present disclosure. The proposed method can be applied to the AR field.

[0083] Reference Fig. 8A , when the user points the camera of the electronic device 100 at any environment, the user can obtain the objects recognized in the current frame and the pre-recognized objects on the slider as "available context". The pre-recognized object icon can be dragged and dropped to any recognized object on the current AR frame to perform context-based actions in the electronic device 100. Fig. 8A As shown, AR recognizes "car", "twin towers" (buildings) and other objects in the current frame. Figure 8B As shown in , the user drags the car icon from the slider and hovers over the Twin Towers icon. Figure 8C As shown, when the user drags the car icon and hovers over the "Twin Towers" icon, the electronic device 100 displays all possible actions related to the cars present in the "Twin Towers" building. The "Twin Towers" has stores for car washing, car rental, car repair, car music system installation, etc., which have registered the context "Car (CAR)" for themselves. In addition, the user can drop the car icon on any of these actionable icons. In this example, the user wants to install a car music system. Dragging and dropping the car icon provides the user with available options for this action in the "Twin Towers" building.

[0084] In another example scenario, when the user drags the mail icon and drops the mail icon on the calendar icon, the context of the mail icon (i.e., the last received email) is extracted. If the last received email contains a meeting schedule, the calendar event for the meeting schedule is saved in the calendar icon.

[0085] Fig. 9 is a flowchart of a method for performing a context-based action in an electronic device based on a drag event on an application icon according to an embodiment of the present disclosure.

[0086] Reference Fig. 9 , at operation 902 of flowchart 900, the method includes selecting one or more first graphical objects in the electronic device 100. At operation 904, the method includes detecting a drag event for the selected one or more first graphical objects, the drag event involving manipulation of the one or more first graphical objects. At operation 906, the method includes identifying a context associated with the one or more first graphical objects in response to the detected drag event. At operation 908, the method includes displaying one or more second graphical objects based on the identified context. At operation 910, the method includes: performing one or more actions in the electronic device 100 based on the selection of the one or more second graphical objects.

[0087] Fig.10 is an example diagram of an electronic device displaying one or more actions based on an identified context according to an embodiment of the present disclosure.

[0088] Reference Fig.10 , the user drags an icon (i.e., application 3 icon) and places it on itself. When application 3 is dragged and dropped on itself, the electronic device 100 displays all actions that can be performed on application 3 based on its current context. For example, if application 3 is a professional network application, the current context of the professional network application includes the current / recently viewed profile. The current context of the professional network application may include a profile uniform resource locator (URL), a profile's phone number, and a profile's email identifier (ID). As shown in the inner ring, the various actions recommended by the electronic device 100 include calling the current / latest viewed profile, sending a message to the current / latest viewed profile, sending an email to the current / latest viewed profile, and sharing the current / latest viewed profile. As shown in the outer ring, when a selection of an action (i.e., a call action) is received, the electronic device 100 provides possible methods, such as calling through a phone application or calling through a communication application.

[0089] Fig.11 is an example diagram of an electronic device displaying one or more actions when a drag gesture is performed on an application being accessed at the electronic device according to an embodiment of the present disclosure.

[0090] Reference Fig.11, the user is accessing a professional web application. The user performs a drag gesture on the professional web application. The drag gesture can be performed on any part of the page in the professional web application. When the drag gesture is performed, the electronic device 100 displays all actions that can be performed on the application 3 based on the current context of the application 3. When a selection of an action (i.e., a call action) is received, the electronic device 100 provides the following Fig.10 Thus, with the proposed method, a drag gesture is performed for content sharing without closing the application.

[0091] Fig.12 is a block diagram of a context action engine in an electronic device according to an embodiment of the present disclosure.

[0092] Reference Fig.12 , the context action engine 120 includes a content action framework 122, a content action database (DB) 124, an API manager 126, and a user interface (UI) manager 128. The content action framework 122 includes an action registration module 122a, an application filter 122b, and an action mapper 122c.

[0093] The content action framework 122 provides an API for recording the context of an application, which is used by all applications. Fig.12 As shown, the dragging party is a GUI element that is dragged to perform context-based actions. The holding party is another GUI element, where the dragging party is dragged onto it to complete the context-based drag and drop operation. The action registration module 122a is used by different applications to register support actions for specific content. The application filter 122b is used to filter applications that provide actions for specific input content types. The action mapper 122c is used to obtain actions corresponding to a given data packet name and content type. The content action database 124 is used to map content-action and action-data packet names. The content action database 124 also maintains a record of the current context of each application. When the user performs dragging on the application icon, the content action framework 122 will perform a search in the content action database 124 to determine the context of the application. Similarly, the context of the holding party can be determined. In some embodiments, the content action database 124 can reside in a cloud network. As shown in Table 1, the content action database 124 maintains a record of the current context of each application.

[0094] Table 1

[0095] ID Data Pack Content Type content 1 Com.gallary URI_IMAGE / sdcard / DCIM / 2018070612.jpg 2 Com.phone Contact Number 9560604464 3 Comm.maps Location 28.6208°N, 77.3639°E 4 Com.App1 Profile Er.ankursharma89@gmail.com 5 Com.App2 Profile 8826243497

[0096] The API manager 126 includes an API set for providing interaction between the control layer and the view layer. The API set may include filtering (OnFilter()), hovering (OnHover()), setting actions (SetAction()), and performing actions (PerformAction()). The UI manager 128 includes an application layer and an action layer. The application layer is used to display filtered application icons for a specific content type, while the action layer is used to display a set of supported actions provided by a data package for a specific content type.

[0097] Fig.13 The action of registering an application based on content type according to an embodiment of the present disclosure is shown.

[0098] refer to Fig.12 and Fig.13 At operation 1, the holder registers its content-based action set with the action registration module 122a. By registering, the holder indicates to the contextual action engine 120 the actions it supports for the content. For example, in the case of a content sharing application, the content type of the application may be an image or video, and the action may be sharing or setting a profile picture, etc.

[0099] Fig.14 It is shown that the content of an application is transmitted based on the context of the application according to an embodiment of the present disclosure.

[0100] Reference Fig.12 and Fig.14 , at operation 2, the drag direction application filter 122b populates its content and passes it to the application filter 122a. The content is transferred based on the current context of the dragger. For a given scenario, the content set of the gallery will contain the image URI of the most recently / currently opened image. In addition, the content type can include phone numbers, images, videos, files, locations, email contacts, settings objects, etc.

[0101] Fig.15 Various operations for determining one or more related applications for performing one or more actions according to embodiments of the present disclosure are shown.

[0102] Reference Fig.12 and Fig.15 At operation 3a, the application filter 122b queries the content action DB 124 for potential applications / icons based on the current dragger content. At operation 3b, the content action DB 124DB checks for applications / icons that have registered actions corresponding to the input content set. Only the data packet names of such applications / icons are provided to the application filter 122b. At operation 3c, the application filter 122b passes the filtered data packet names to the application layer of the UI manager using an API call (i.e., OnFilter()).

[0103] Fig.16 It is an example diagram that an electronic device displays multiple applications related to an application selected for performing one or more actions according to an embodiment of the present disclosure.

[0104] Reference Fig.12 and Fig.16 , the UI manager 128 highlights the icon of the received data packet name, and all other icons on the launcher are dimmed.

[0105] Fig.17 Various operations performed when an application icon is dragged near another application icon according to an embodiment of the present disclosure are illustrated.

[0106] Reference Fig.12 and Fig.17 At operation 4a, when the dragged icon reaches the vicinity of any highlighted icon, the application layer calls the onHover() API. The API transmits the data packet name of the closest application / icon to the action mapper 122c. At operation 4b, the action mapper 122c queries the content action DB 124 for a list of actions corresponding to the received data packet. At operation 4c, the content action DB 124 responds with a list of actions mapped to the input data packet of the given content. At operation 4d, the action list is provided to the UI manager action layer through the SetAction() API.

[0107] Fig.18 is an example diagram of an electronic device displaying a recommendation when an application icon is dragged and hovered over one or more application icons according to an embodiment of the present disclosure.

[0108] Reference Fig.18 When the gallery icon reaches the vicinity of the communication application highlight icon on the screen, the OnHover() API is called. The OnHover() API detects the holder data package name as the communication application and sends it to the action mapper 122c. The action mapper 122c queries the content action DB 124 for the corresponding action of the communication application. Fig.18 As shown, the set of actions provided to the UI manager 128 are displayed to the user as action / recommendation options.

[0109] Refer again Fig.12 At operation 5, when the dragging party is dropped from operation 4 onto one of the action options, the PerformAction() API is called. Based on the current context of the holding party, the selected action is performed on the holding party icon.

[0110] Fig.19It is an example illustration of performing various actions on a connected device associated with an electronic device according to an embodiment of the present disclosure.

[0111] Reference Fig.19 , the user can perform smart actions on the wearable device connected to the electronic device 100 without browsing the application associated with the wearable device and without directly interacting with the wearable device. Any other application can be selected as the dragging party, and the application associated with the wearable device can be selected as the holding party. The context of the dragging party determines which wearable device will be used to perform the action.

[0112] Fig. 20A and Fig. 20B is an example diagram of performing various actions on an IoT device in a smart home environment according to various embodiments of the present disclosure.

[0113] Reference Fig. 20A ,The proposed method can be used to perform smart actions on connected IoT devices in a smart home.,When the user drops the icon of a smart home application on any other application,,then the context of the current / last interacted IOT device can be used to perform other,actions.

[0114] Reference Fig. 20B , for example, IoT device 1 is a microwave oven, and IoT device 2 is a television. The context of the microwave oven is the most recently prepared dish, while the context of the television is the last watched program. Therefore, with the proposed method, when a user drops the icon of a smart home application icon over any other application icon, the context of the current / last interacted IoT device can be used to perform context-based actions.

[0115] Fig.21 It is an example diagram of executing actions in an electronic device based on the context of multiple applications according to an embodiment of the present disclosure.

[0116] Reference Fig.21 , using the proposed method, the contexts of multiple applications can be used simultaneously (i.e., through a context window or a context box, where multiple applications are added to the context window) to perform cumulative actions. The user can add icons of more than one application in the context window, and when the user drops the context window on the holding party, single or multiple actions can be performed based on the corresponding contexts of the applications dropped in the holding party.

[0117] The embodiments disclosed herein may be implemented by at least one software program running on at least one hardware device and performing network management functions to control the elements. The elements shown include blocks that may be at least one of a hardware device or a combination of a hardware device and a software module.

[0118] While the present disclosure has been shown and described with reference to various embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. A method for operating an electronic device, the method include: selecting one or more first graphical objects in a display of the electronic device; detecting an event for the selected one or more first graphical objects, the event involving dragging and hovering the one or more first graphical objects over one or more second graphical objects; In response to the detected event, identifying a context associated with each of the one or more first graphical objects and the one or more second graphical objects, wherein the context of the one or more first graphical objects is shared with the one or more second graphical objects; and Based on the identified context, displaying one or more third graphical objects, wherein the one or more third graphical objects correspond to at least one of possible actions or recommended options, and the one or more third graphical objects are displayed when the selected one or more first graphical objects are dragged and hover over the one or more second graphical objects; recognizing that the one or more first graphic objects are dragged and released on a fourth graphic object among the one or more third graphic objects; in response to the one or more first graphic objects being dragged and released on the fourth graphic object, performing one or more actions associated with the fourth graphic object in the electronic device, The context indicates at least one operation performed by at least one application corresponding to the one or more first graphical objects.

2. The method according to claim 1, in, The one or more first graphical objects and the one or more second graphical objects are application icons.

3. The method according to claim 1 or 2, in, The possible actions and / or the recommended options are identified by mapping names associated with the one or more second graphical objects with a stored list of actions associated with the names.

4. The method according to claim 1, in, Identifying a context associated with each of the one or more first graphical objects and the one or more second graphical objects includes extracting a user's recent activity with the one or more first graphical objects and the one or more second graphical objects.

5. The method according to claim 1, in, Performing the one or more actions in the electronic device includes sharing at least one content of the one or more first graphical objects with the one or more second graphical objects.

6. The method according to claim 5, in, Executing the one or more actions in the electronic device includes: When accessing the image via an augmented reality (AR) mode, retrieving one or more pre-identified graphical objects; identifying at least one object in the image; and A drag and drop event is detected for the selected pre-identified graphical object to be placed on the at least one object in the image to perform the one or more actions based on the identified context.