A user interface that integrates heterogeneous software applications through inter-process communication mechanisms.

Abstract

This specification describes a technology and system that enables a user interface for integrating heterogeneous software applications through an inter-process communication mechanism. This technology and system allows a user to interact with widgets that provide information or functionality from multiple heterogeneous software applications through a single interface. Through this single interface, the user can easily view information from these heterogeneous software applications and interact with them. In this way, the user can quickly access, modify, or interact with multiple heterogeneous software applications or their respective information.

Description

Background Art

[0001] Many users access software applications through multiple different user interfaces. These interfaces are often complex or substantially different from each other, especially when the software applications are from different software providers. Thus, these interfaces often have different workflows, layouts, navigation systems, and terminology. This can be confusing and inefficient for users. As a result, users often struggle to navigate and interact with different software applications, which can affect the user's ability to complete tasks using multiple software applications.

[0002] This description of the background art is provided for the purpose of generally indicating the context of the present disclosure. Unless otherwise indicated herein, the materials described in this section are not admitted to be prior art to the present disclosure or the appended claims, either expressly or implicitly.

Summary of the Invention

[0003] This specification describes technologies and systems that enable a user interface to integrate heterogeneous software applications by means of an inter-process communication mechanism. These technologies and systems enable a user to interact with a plurality of widgets that provide information or functions of a plurality of heterogeneous software applications via one interface. Through this one interface, a user can easily view information from these heterogeneous software applications and interact with them. The technologies and systems described thereby enable a user to quickly access, modify, or interact with multiple heterogeneous software applications or their respective information via a single user interface.

[0004] For example, a method for receiving information from multiple heterogeneous entities via one or more inter-process communication (IPC) mechanisms is described. This method presents a portion of the information received from the multiple heterogeneous entities within a user interface, and at least partially simultaneously. This method presents a portion of this information within a widget for each respective heterogeneous entity. The method then allows the user to select a portion of the information via the user interface and selectable controls associated with each widget. In response to receiving a selection, this method provides additional information or functionality related to the information or the heterogeneous software application within the user interface, and independently of the execution of the heterogeneous software application related to the information. Alternatively, in response to receiving a selection, this method automatically triggers the presentation of a complete instance of the interface for the heterogeneous software application via one or more IPC mechanisms, without further user interaction. This complete instance is triggered to provide additional information associated with the selected control.

[0005] This specification also describes computer-readable media having instructions for implementing the methods summarized above and other methods described herein, as well as systems and means for implementing these methods.

[0006] This summary of the invention is provided to introduce a simplified concept of a user interface for integrating heterogeneous software applications through an inter-process communication mechanism, which will be further described below with embodiments and drawings for carrying out the invention. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used in determining the scope of the claimed subject matter.

[0007] Details of one or more embodiments of a user interface for integrating heterogeneous software applications through an inter-process communication mechanism are described herein with reference to the following drawings. Throughout the drawings, the same numbers are used to refer to similar features and components. [Brief explanation of the drawing]

[0008] [Figure 1] This document illustrates an embodiment of a user device that includes a notification manager and a user interface application. [Figure 2] This shows an example of a user interface with selectable controls. [Figure 3] This illustrates the user interface and the relationships between multiple software applications. [Figure 4] This document illustrates an exemplary embodiment of modifying the user interface using selectable (e.g., drag-and-drop) controls. [Figure 5] This shows an example implementation of adding content. [Figure 6] This provides an exemplary implementation of removing a single item within a software application in response to which technology causes changes to other software applications. [Figure 7] This shows exemplary full and feature-limited versions of the software application. [Figure 8] This demonstrates the transfer of information related to interactions with a first software application and its associated content to a second software application within a single user interface. [Figure 9] This demonstrates the presentation of application-related content and the ability to select content on the user interface. [Figure 10] This example demonstrates a user interface application that presents a scaled-down version of an instance of content. [Figure 11]An illustrative diagram of a user interface that responds to selectable controls is shown. [Figure 12] This diagram illustrates different areas of the user interface, each of varying sizes. [Figure 13] This section illustrates exemplary embodiments of additional content and functionality from a software application. [Figure 14] This diagram illustrates the selectable features of a software application. [Figure 15] This shows visual information from several software applications. [Figure 16] This document provides an exemplary environment in which techniques for a multi-application user interface to prioritize notifications can be implemented. [Figure 17] Figure 16 shows an exemplary user interface with five priority notifications. [Figure 18] Figure 17 shows an exemplary embodiment of the user interface. [Figure 19] Figure 17 shows another exemplary embodiment of the user interface. [Figure 20] Another example of a user interface with the five priority notifications shown in Figure 16 is presented along with a sixth priority notification. [Figure 21] This document presents an exemplary environment in which a technology enabling a user interface that integrates heterogeneous software applications through inter-process communication mechanisms can be implemented. [Figure 22] Two exemplary user interfaces are shown that can implement a technology that enables a user interface to integrate heterogeneous software applications through inter-process communication mechanisms. [Figure 23] This document presents other exemplary user interfaces that can implement techniques enabling user interfaces that integrate heterogeneous software applications through inter-process communication mechanisms. [Figure 24] This illustrates an exemplary method for machine learning-based models. [Figure 25]Shows a block diagram of an exemplary computing device and a training computing system. [Figure 26] Shows a block diagram of a training process. [Figure 27] Shows an exemplary method that enables a user interface to integrate heterogeneous software applications by means of an inter-process communication mechanism. [Figure 28] Shows an exemplary method that enables a user interface for simplified and integrated interaction with multiple applications. [Figure 29] Shows an exemplary method that enables a multi-application user interface to prioritize notifications. **Mode for Carrying Out the Invention**

[0009] Overview This specification describes a technology and system that enable a user interface for integrating heterogeneous software applications by means of an inter-process communication mechanism. This technology and system enable a user to interact with widgets (e.g., borderless windows, icons with selectable functions, or thumbnails) associated with multiple heterogeneous software applications via a single interface. This single interface enables the user to easily navigate information or functions that would otherwise be complex and time-consuming. Thus, instead of multiple heterogeneous user interfaces with different structures, formats, processes, and / or windows (each associated with a heterogeneous software application), the user can easily view and use the functions of heterogeneous software applications and their heterogeneous user interfaces. Therefore, this technology and system assist the user in easily interacting with multiple heterogeneous software applications and their information. Specifically, this specification describes an improved user interface that provides an improved mechanism for facilitating user interaction with multiple heterogeneous software applications and enabling user input when interacting with multiple heterogeneous software applications.

[0010] In contrast, consider the disclosed technologies for simplifying and integrating interactions with multiple heterogeneous software applications. These technologies provide a more efficient and user-friendly way for a user to integrate interactions across multiple heterogeneous applications, thereby reducing the complexity of the project and simplifying the way the user interacts with software applications.

[0011] Similarly, consider a user who needs to complete a complex project involving the use of multiple heterogeneous software applications, including a project management application, a team communication application (e.g., a short messaging service (SMS) application or a chat application), and a word processing application. Throughout the day, the user needs to use all three applications, often switching between different interfaces or applications. However, according to the disclosed technology, the user can access all of these heterogeneous applications through a single interface. The interface displays widgets such as simplified and feature-limited versions of each application, allowing the user to quickly navigate between them and use their basic functions. The user can then choose to view information related to the word processing application and then interact with that information in a lightweight manner, such as through a larger but still simplified interface, or entirely through full instances of the interface for the heterogeneous software application to which the information is associated. By using this single-user interface, the user can accomplish project tasks more easily.

[0012] This is just one example of how the described techniques and devices can be used to simplify and integrate interactions between multiple software applications. Other examples and embodiments are described throughout this specification. Hereinafter, this specification returns to an exemplary operating environment, and then exemplary devices, methods, and systems are described.

[0013] Operating environment Considering Figure 1, an exemplary embodiment 100 of a user device (UE) 102 is shown, which includes a notification manager 104 and a user interface application 106, and which can implement techniques for a user interface that integrates heterogeneous software applications via an inter-process communication mechanism. The UE 102 in Figure 1 is shown in various exemplary devices, including, for example, a smartphone 102-1, a tablet 102-2, a laptop 102-3, a desktop computer 102-4, a computing watch 102-5, computing glasses 102-6, a gaming system 102-7, and a home automation and control system 102-8. The UE 102 may also include other devices, such as a television, an entertainment system, an audio system, a car, a drone, a trackpad, a drawing pad, a netbook, an e-reader, a home security system, and other consumer electronics. Note that the UE 102 may be wearable, non-wearable but portable, or relatively fixed (e.g., desktop and consumer electronics).

[0014] UE102 also includes one or more computer processors 108 and one or more computer-readable media 110, including memory media and storage media. Applications and / or operating systems (not shown) implemented as computer-readable instructions on the computer-readable media 110 are executed by the computer processors 108 and can provide some or all of the functions described herein, including some or all of the functions of the notification manager 104 and the user interface application 106 (shown within the computer-readable media 110, but not required, and server or remote technology may be used in whole or in part).

[0015] The notification manager 104 receives notifications from the UE 102. These notifications can be received actively or passively, one at a time, within a group over a period of time or per application, via various methods such as push notifications, application programming interface (API) notifications, passive notifications, local notifications, and in-app notifications. The notification manager 104 is described in more detail below and is partially shown in Figures 16-20 and Figure 29.

[0016] UE102 may also include a network interface 112. UE102 can use the network interface 112 to communicate data over wired, wireless, or optical networks. For example, but not limited to, the network interface 112 may communicate data over a local area network (LAN), wireless local area network (WLAN), personal area network (PAN), wide area network (WAN), intranet, internet, peer-to-peer network, point-to-point network, or mesh network.

[0017] UE102 may also include a display 114. UE102 may use the display 114 to present the user with priority groups of notifications. Display 114 may be any exemplary display, such as the one shown in Figure 1, which illustrates the user interface throughout this specification.

[0018] The domains and functions described herein may be further divided and combined. Thus, different embodiments of UE102 can be used to implement user interfaces that integrate heterogeneous software applications and / or prioritize notifications through inter-process communication mechanisms.

[0019] Simplified and integrated interaction with multiple applications More specifically, as shown in Figure 1, the user interface application 106 presents multiple selectable controls with control regions within the user interface. Each of these controls is associated with a software application, and each can be selected to enable the use of functions associated with those software applications. These functions may be feature-restricted set versions of the functions enabled by a full instance of the software application interface. However, which functions are enabled by the user interface application 106 may be determined based on user selection or likely usage preferences, through machine learning models and various other methods described herein. In addition to enabling the selection and use of feature-restricted set versions of associated software applications, the user interface application 106 can integrate interactions, such as modifying the content of one feature-restricted set version based on its interaction with other content in the feature-restricted set version.

[0020] As an example, consider Figure 2. Figure 2 shows an exemplary environment 200 that can implement a technique enabling a multi-application user interface that integrates heterogeneous software applications through an inter-process communication mechanism. Environment 200 includes a user interface 202, a control area 204, and first, second, and third selectable controls 206, 208, and 210, which are associated with the first, second, and third related software applications 212, 214, and 216, respectively, each of which is provided by the user interface application 106. As shown, the user interface 202 presents the first, second, and third selectable controls 206, 208, and 210 within the control area 204, although the control area 204 can instead be oriented horizontally. These controls are shown with icons associated with the software applications to aid user recall, although this is not mandatory.

[0021] Considering Figure 3, Figure 3 illustrates one exemplary way in which the technology enables simplified and integrated interaction with multiple applications. Here, exemplary Figure 300 includes a user interface 302 with a first instance 302-1 and a second instance 302-2. The first instance 302-1 has a first software application 304 ("Gmail" trademark), and the second instance 302-2 has a second software application 306 ("Calendar"). The first instance 302-1 of user interface 302 presents a function associated with the first software application 304 (e.g., "Inbox"). Here, the user receives an email from a colleague requesting a change of time for an upcoming meeting. The user has the option to automatically respond to the email and selects a positive response. This response is received by user interface application 106 shown in Figures 1 and 2 and is determined to be related to other applications associated with user interface application 106 (e.g., second software application 306). Next, the user interface application 106 instructs the second software application 306 to reflect the response received via the first instance 302-1 (or triggers a feature-limited set version of the second software application 306). Thus, in a single user selection, in this case accepting a time change request received via the "Gmail" (trademark) application, the user interface application 106 causes the other application to accept changes, in this case moving the previous meeting time with "Jon" from 12:00 pm to the accepted 4:00 pm, as requested via email from Jon.

[0022] This is just one example of the many ways in which user interface application 106 enables integrated interaction, in which one interaction via email affects the calendar application. This effect is shown in a second instance 302-2 of user interface 302 ("Change: Meeting with JON") in change 308. This integration of inter-application interaction may be based on prior associations between the first software application 304 and the second software application 306, such as associating the first and second software applications 304 and 306 with substance-based tasks (e.g., tasks with relevant properties), with teams of people (e.g., Jon, the user, and other people associated with the task, friendship / workgroup, or event), through explicit user selection, or through implicit user selection based on the user's history of selections and interactions, or through machine learning models as described elsewhere in this specification. Furthermore, the previous association may be based on an implicit user selection that associates the applications through a previous physical selection for oriented the first software application 304 and the second software application 306, either within the user interface 302 (for example, in the same instance not shown in Figure 3) or in different areas. This will be partially explained below.

[0023] Considering Figure 4, which shows an exemplary environment 400 illustrating additions or modifications to the size or orientation of a feature restriction set version (e.g., an application feature restriction set version), this is one way in which a user interface application 106 can associate a software application, as described in the description of Figure 3. Environment 400 includes a user interface 402 having four instances (the first, second, third, and fourth user interfaces 402-1, 402-2, 402-3, and 402-4), a selectable control 404, a control area 406, an application feature restriction set version area 408 ("app area 408"), and the first, second, and third application feature restriction set versions 410, 412, and 414. The selectable control 404 can be selected, highlighted, or dragged from the control area 406 (an example of the control area 204 in Figure 2) to the application feature restriction set version area 408.

[0024] As shown in Figure 4, the first and second application feature restriction set versions 410 and 412 are located within the app area 408 of the first user interface 402-1. A selectable control 404 is selected and dragged into the app area 408 within the second user interface 402-2. While the selectable control 404 is in the app area 408 (or immediately thereafter), the user interface application 106 causes the third application feature restriction set version 414 to form from the selectable control 404. In the third user interface 402-3, the first application feature restriction set version 410 and the second application feature restriction set version 412 are automatically modified to fit the user interface application 106, which can be done without user interaction within the app area 408, along with the addition of the third application feature restriction set version 414 associated with the selected selectable control 404. When a selectable control 404 is deselected, the user interface application 106 causes the third application function restriction set version 414 to exhibit a size or orientation that fits the application function restriction set version area 408. The user interface application 106 adjusts the first application function restriction set version 410 and the second application function restriction set version 412 without user interaction, as seen in the fourth user interface 402-4.

[0025] Considering Figure 5, which illustrates an exemplary environment 500 in which the technology modifies content between software applications, the environment 500 includes a user interface 502 with four instances (user interfaces 502-1, 502-2, 502-3, and 502-4) and transferable content 504. In user interface 502-1, transferable content 504 is selected by the user. In this example, transferable content 504 is a new note, but transferable content 504 may be a new calendar event, a new document, or other new transferable item. The user can drag transferable content 504 from the first feature limit set version 506 of the first application to the second feature limit set version 508 of the second application, thereby modifying the content of the second application.

[0026] In user interface 502-2, the user begins dragging transferable content 504 from the first feature restriction set version 506 to the second feature restriction set version 508. In user interface 502-3, the user reaches the second feature restriction set version 508 and drops or releases the transferable content 504 into the second feature restriction set version 508. For example, the transferable content 504 is a new note taken from the first feature restriction set version 506 associated with the memo software application and placed in the second feature restriction set version 508 associated with the team meeting application. This action from the user is received by user interface application 106, which integrates the interaction between the two applications and thereby passes the transferable content 504 to the second application. User interface application 106 can identify tasks from one application that may be useful in other applications through previous user history, through algorithms, or through machine learning. More specifically, the machine learning model is trained through user actions and user history to determine when and where a task is needed. Once the user interface application 106 identifies a task action to be completed, it can integrate interactions between the two applications, such as passing transferable content 504 from the first feature limit set version 506 to the second feature limit set version 508.

[0027] Communication of information and / or commands and control between applications (e.g., communication of transferable content 504 between various functional restriction set versions or commands by a user interface application 106 for controlling an SMS software application) can be achieved using inter-process communication (IPC) mechanisms. Various processes of IPC mechanisms are contemplated herein, and each can be used, whole or in part, for communication between software applications and / or entities that control or are associated with these software applications, such as passing data, receiving data, accessing data, and / or controlling the behavior and functionality of software applications. These IPC mechanisms include, but are not limited to, application programming interfaces (APIs), remote procedure calls (RPCs), message queues, shared memory and databases, web services (e.g., Simple Object Access Protocol (SOAP) or Hypertext Transfer Protocol (HTTP)), or file-based communication. For example, any or all of these IPC mechanisms may be utilized by the notification manager 104, the user interface application 106, and the various software applications described herein.

[0028] As another example, transferable content 504 is a newly edited photo dragged from a first feature restriction set version 506 associated with a photo editing software application and dropped into a second feature restriction set version 508 associated with an SMS software application. The user interface application 106 receives this action performed by the user and then integrates the interaction between the two different software applications. The transferable content 504 is passed from the first feature restriction set version 506 of the first application to the second feature restriction set version 508 of the second application, thereby allowing the user to share their creative photo editing work with others.

[0029] As another example, transferable content 504 is a scheduled meeting or event retrieved from a first feature limit set version 506 associated with a calendar software application and placed in a second feature limit set version 508 associated with a task management software application. The user interface application 106 receives this action from the user and integrates the interaction between the two software applications, enabling the transferable content 504 to be passed from the first application to the second. This streamlines the user's workflow and allows them to manage new events or meetings across two different software applications.

[0030] Considering Figure 6, which illustrates an exemplary environment 600 in which a user interface application 106 modifies content within two separate but related applications. Environment 600 includes a user interface 602, as well as first and second content 604 and 606. The first content 604 is within a feature restriction set version 608 associated with a first software application 612, and the second content 606 is within a second feature restriction set version 610 associated with a second software application 614. As seen in Figure 6, the first content 604 is modified by the user ("point B" is erased), and since the first software application 612 and the second software application 614 are related to each other, the second content 606 is automatically modified by the user interface application 106. This can be useful to a user in many different scenarios. Here, the user wants to delete a specific part of a note from a meeting. When selected by the user, the content of the Team Meeting application, as well as the content of the Notes application, is modified by the User Interface application 106 (for example, automatically, with a delay, or immediately). The Team Meeting application receives the changes from the User Interface application 106, and the rest of the user's team sees the edits. Similarly, the user may choose to modify the first content 604 of feature restriction set version 608, in which case the second content 606 of the second feature restriction set version 610 is also added or modified by the User Interface application 106 without the user having to re-realize the selection or spend time and effort in any other way.

[0031] Considering Figure 7, which shows an exemplary environment 700 having a user interface 702, with first, second, and third instances 702-1, 702-2, and 702-3. In the first instance 702-1, the user interface application 106 presents a full-featured version 704 of the software application (here, a “calendar” application 706 with associated selectable controls 708). In the full-featured version 704, the software application is a full-featured instance. In the second and third instances 702-2 and 702-3 of the user interface 702, the user interface application 106 presents a limited-featured version 710. Each limited-featured version 710 includes, respectively, a function that may be represented in the full-featured version, and at least one function that is similar to, or part of, information or content, and at least one piece of information or content. One such example is the full-featured version 704 of the software application; however, in some cases, a widget may be used instead. As used herein, the term "widget" includes at least information / content related to a software application or function. The functionality of a widget may be different from, the same as, or similar to the functionality of a software application. The information / content presented by a widget may be similar to, part of, or a notice relating to information or content. Thus, a widget as used herein may have less, lighter, and less rich content or functionality than a feature-limited set version (e.g., feature-limited set version 710).A widget may be a borderless window, an active control (e.g., an icon with a single selectable function), a thumbnail (e.g., a smaller / reduced version of information or content, or a smaller, lower-resolution image of a larger, higher-resolution image, having part of the information or content that a software application can present), or an applet (e.g., a lightweight, web-page-based user interface module, which may, depending on the applet, be a feature-limited set version, or alternatively, a lightweight, web-page-based user interface module, which may also be a feature-limited set version). In the first instance 702-1, the user interface 106 receives a change from the user and presents a new event ("Meeting with Jon") within the full-featured version 704 of the "Calendar" application 706. The user interface application 106 then also presents the change from the user in a feature-limited set version 710, including the relevant selectable control 708, as seen in the third instance 702-3.

[0032] Considering Figure 8, which shows an exemplary environment 800 having a user interface 802 with first and second instances 802-1 and 802-2. The first instance 802-1 shows that an edit has been made to document 804, which appears within a feature-limited set version 806 of a word processing application. Here, document 804 is a task list ("Required Work") showing tasks and the team members (Jim and Amir) responsible for completing each task. Here, team member "Jim" has completed task number 1. Task completion is indicated by a strikethrough 808 on the task list. This change, striking out task number 1, represents one of many examples of changes to content related to an application, in this case a change to a document in a word processing application. This change is received or determined by the user interface application 106 in Figure 1. The user interface application 106 determines its association with document 804 and other applications, such as calendar events or a team of people with a shared SMS group. In the case of a calendar event, the user interface application 106 determines that document 804 was attached to a meeting invitation. In the case of a team of people, the user interface application 106 determines that document 804 has four different people with editing privileges, including, for example, Jim, Amir, Jon, and Susanne. Either or both of these decisions are sufficient to determine the association of document 804 with other applications. This determination that the content changes to document 804 are associated with a calendar event or a group of people causes the user interface application 106 to then integrate the changes in one application with other applications (for example, by the IPC mechanism described above) by integrating the changes in the content of the word processing application with the other applications.In this example, the user interface application 106 determines that these same four people are having a meeting and that one of the team members ("Jon") made edits to document 804 and is busy with a scheduled event from the start of the meeting through to the end of the meeting. Based on this, the user interface application 106 may automatically edit document 804 without user interaction and / or reattach it to the meeting in the calendar event and / or notify the team members of the changes and the absence of one team member. Here, the user interface application 106 interacts with the SMS application 810 to cause the SMS application 810 to send a team chat 812 indicating the changes to the content and their impact on the team and / or meeting. This SMS instruction 812 is shown in the functional limitation set version 814 of the SMS application 810 in the second instance 802-2.

[0033] Considering Figure 9, which shows an exemplary environment 900 provided by a user interface application 106 (not shown), the environment 900 includes a first selectable control 902, a control area 904, an application function restriction set version area 906, a first software application 908, a user interface 910 with user interface instances 910-1, 910-2, 910-3, and 910-4, and three instances 912 of content represented by content instances 912-1, 912-2, and 912-3. The user may initially select the first selectable control 902 from the control area 904, or select it along the way, in which case the user interface application 106 presents the first software application 908 and one or more instances of the associated content 912 (e.g., as thumbnails) on the application function restriction set version area 906 of the user interface 910. The user interface application 106 may also allow the user to select various different instances of content 912, as shown in instances 912-1, 912-2, and 912-3, such as the last three notes viewed, edited, or created by the user. The selection of a second content instance 912-2 is shown in user interface instance 910-3 by click selection 914. In response to the user selecting an instance 910-2 of content 912 (here "Note 2" rather than "Note 1" or "Note 3"), the user interface application 106 presents the complete content (to the extent that the complete content can be presented, in this case when the note is quite short) in the first software application 908 on user interface 910, as seen in user interface instance 910-4.

[0034] Considering Figure 10, which shows an exemplary environment 1000 in which a user interface application 106 presents a reduced-size, reduced-resolution, limited-function, and / or content-reduction version of one or more instances of content or functionality. Environment 1000 includes content 1002 associated with a software application 1004, in this case three instances of different content (the first, second, and third content instances 1002-1, 1002-2, and 1002-3). As described above, each of these content instances 1002 can be presented in a reduced-size, reduced-resolution, limited-function, and / or content-reduction version. In this example, the three content instances 1002 are different notes from a notes application, and have reduced content because the displayed size is not large enough to include all of the content (e.g., all the text of each note). However, other examples are conceivable, such as reducing content size (e.g., smaller fonts or smaller images), lowering resolution (e.g., images or text), and limiting functionality (e.g., an "edit" selectable control for editing content rather than additional features such as font selection, color changes, or highlighting).

[0035] Environment 1000 also represents a first software application 1004 to which content 1002 is associated, and a user interface 1006 having first, second, third, and fourth user interface instances 1006-1, 1006-2, 1006-3, and 1006-4. As seen in Figure 10, a user interface application 106 (not shown) presents content instances and allows the user to make selections. These may be presented one at a time (see Figure 9) or they may be cycled through one at a time (for example, moving chronologically from the last edited or viewed to an earlier one). The cycle of content instance 1002 is shown in Figure 10, where the first content instance 1002-1 is presented for selection in feature restriction set version 1008 within the first user interface instance 1006-1, and is then selected and dragged off from feature restriction set version 1008 (or to the edge or outside of the second user interface instance 1006-2), as shown by drag selection 1010. Following this selection rejecting the presented content instance 1002-1, the second content instance 1002-2 is similarly rejected and is shown by drag selection 1012 via the third user interface instance 1006-3. Following these, the third content instance 1002-3 is presented, and then, in the fourth user interface instance 1006-4, it is selected and dragged (and dropped) into or onto feature restriction set version 1008, as shown by drag-and-drop selection 1014. Next, the user interface application 106 presents additional content and / or functionality, as shown in the complete content instance 1016.

[0036] Considering Figure 11, which shows an exemplary environment 1100 for drag-and-drop of a selectable control, the environment 1100 includes a first selectable control 1102, a control area 1104, a first associated software application 1106, and a user interface 1108 having instances 1108-1 and 1108-2. The user interface application 106 enables the selection of the first selectable control 1102, here using drag-and-drop. Drag-and-drop of the first selectable control 1102 allows the first selectable control 1102 to be selected by the user, but the user interface application 106 still presents the original selectable control 1102 within the control area 1104. As seen in Figure 11, the first selectable control 1102 can be dragged onto the user interface 1108 while it is still displayed within the control area 1104. When the first selectable control 1102 is dragged onto the user interface 1108, the user interface application 106 presents the first associated software application 1106 on the user interface 1108. When the user drags the first selectable control 1102 onto a portion of the user interface 1108, the user interface application 106 displays a feature-limited set version of the first associated software application 1106. For example, as seen in Figure 11, the user selects a control from the control area 1104 and drags the control onto the user interface 1108. When the user drags the control onto the user interface 1108, the feature-limited set version of the previous software application shifts to match the software application associated with the dragged control.For example, the dragged control may be a memo control associated with the memo application, and therefore, when the control is dragged onto the feature-restricted set version on the user interface 1108, the memo application will be replaced, moved, or resized by other feature-restricted set versions of other applications (e.g., deleting SMS and playlists, as well as resizing the timeline and calendar).

[0037] Considering Figure 12, Figure 12 shows a calendar application 1202 within environment 1200, represented here by selectable controls 1204 (here, icons) for the same software application, both fully functional and feature-limited versions. Environment 1200 includes a first area 1206 and a second area 1208, with the fully functional version 1210 in the first area 1206 and the feature-limited version 1212 in the second area 1208. The user interface 1214 is provided by user interface application 106 (not shown).

[0038] The first area 1206 is of a different size than the second area 1208, and each is located in a different part of the user interface. Within the first area 1206 is the full-featured version 1210 of the software application 1202, and within the second area 1208 is the feature-limited version 1212 of the software application 1202. These two exemplary options are shown to illustrate only two of the various different ranges of features, sizes, and orientations that the user interface application 106 allows and that the user can choose (or that may be determined by the user interface application 106). As illustrated, the full-featured version 1210 includes additional content with additional features such as four additional notifications, larger size and resolution, and the ability to select and subsequently present popup details 1216.

[0039] Considering Figure 13, Figure 13 shows an exemplary environment 1300 having a user interface 1302 that displays a first area 1304, a second area 1306, two feature restriction set versions 1308 and 1310, additional content 1312, and additional functionality 1314, all associated with a software application 1316 ("Notes"). Here, we assume that the size and functionality are reduced from feature restriction set version 1308 to feature restriction set version 1310. In doing so, the user interface application 106 may automatically decrease (or increase) the content or functionality, or both. However, here we assume that the user explicitly chooses to reduce the content by removing the additional content 1312 and additional functionality 1314 (the ability to edit "My To-Do List"). The user chooses to keep two To-Do lists, but the space presents only the first item in the list, removing the editing functionality of one while retaining the editing functionality of the other. This is (partially) illustrated using drag-and-drop selection at arrow 1318. Although not illustrated, the user can select only the "Group To-Do List" or only the "My To-Do List," and remove the editing functions for both, etc. User interface application 106 provides these different customizable feature-limit set versions 1308 and 1310 by enabling this flexibility and allowing the user to select the size, content, and functionality of the application's feature-limit set versions, or to have them automatically selected (for example, based on user history, machine learning).

[0040] Considering Figure 14, which shows an exemplary environment 1400 having selectable functions for a software application, the environment 1400 includes selectable functions 1402, a first content 1404, a first function restriction set version 1406 of the first software application 1408, a second content 1410, and a second function restriction set version 1412 of the second software application 1414. The first content 1404 includes information that enables the selection of selectable functions 1402 in the first function restriction set version 1406 of the first software application 1408. The first content 1404 includes visual information presented to the user within the first function restriction set version 1406 of the first software application 1408. The user interface application 106 can allow a user to modify the content within the first feature restriction set version 1406 of the first software application 1408, and in response, the user interface application 106 modifies the content within the second feature restriction set version 1412 of the second software application 1414. This can be done automatically based on the association between the two software applications 1408 and 1414, where the creation of new calendar events related to the calendar software application is automatically reflected in the reminder software application. In this way, user interactions are integrated, user time is saved, and accuracy and consistency between software applications are improved. The user interface application 106 can pass changes to the content within a feature restriction set for one application to the other application in real time after the content has been fully modified or as the changes are being made. Here, the selection of new events related to the calendar application is done in real time within feature restriction set versions 1406 and 1412. Thus, new events are created and integrated in real time into the interfaces for both applications 1408 and 1414.Although not shown in the diagram, when a user enters text into the first content 1404, the text may be automatically presented in the second content 1410.

[0041] Considering Figure 15, which shows an exemplary environment 1500 that displays various content as visual information within the user interface 1502. The user interface application 106 allows the user to choose to modify the visual information, whether by changing, adding, or deleting it, and to reflect that information in other areas of the user interface 1502 for other related software applications. For example, suppose the user interface 1502 includes feature-limited set versions for the email application 1504, the calendar application 1506, and the SMS application 1508. The visual information includes email 1510, calendar event 1512, and SMS message 1514, respectively. Each of the illustrated visual information formats is a format of content that can be modified via the user interface, and that modification is made for the related software application. That modification may also be reflected in other software applications, and may be reflected in their respective presentations within the feature-limited set versions of those software applications. Thus, a text string ending in "Thank you!" created via the user interface 1502 modifies the content of the SMS application 1508. Furthermore, the changes can also be reflected in related applications, such as reminders created in calendar application 1506 to remind the recipient of a text string indicating "Check with Eric again," based on an analysis of the use of the verb "check" and the first-person pronoun "I." This is just one of many ways in which the technologies and systems described herein enable user interfaces that integrate heterogeneous software applications through inter-process communication mechanisms.

[0042] Notification Manager Figure 16 shows an exemplary environment 1600 that can implement a technique enabling a multi-application user interface for prioritizing notifications. Environment 1600 includes application 1602, shown by example as email application 1602-1, SMS application 1602-2, calendar application 1602-3, and social media application 1602-4, as well as notification manager 104 in Figure 1, user interface application 106 in Figure 1, and user interface 1604. User interface 1604 provides an interface that has information and / or functions related to two or more applications, such as providing notifications from two or more applications and enabling interaction with notifications from two or more applications. Environment 1600 shows notifications related to application 1602, where application 1602 is shown as email application 1602-1, SMS application 1602-2, calendar application 1602-3, and social media application 1602-4. The email application 1602-1 has email notifications 1602-1-1, 1602-1-2, 1602-1-3, and 1602-1-4 (e.g., new email, spam email, unread email, urgent email). The SMS application 1602-2 has SMS notifications 1602-2-1, 1602-2-2, and 1602-2-3 (e.g., text messages, chat, instant messages). The calendar application 1602-3 has calendar notifications 1602-3-1 and 1602-3-2 (e.g., canceled events, meeting times, event reminders, birthday reminders). The social media application 1602-4 has notifications 1602-4-1, 1602-4-2, 1602-4-3, 1602-4-4, 1602-4-5, 1602-4-6, and 1602-4-7 (e.g., likes, tags, follows, comments).

[0043] In environment 1600, notification manager 104 receives notifications from at least two of the applications 1602. These notifications can be received actively or passively, over a period of time within a group, or one at a time per application, and in various ways, including push notifications, API notifications, passive notifications, local notifications, and in-app notifications (or other IPC mechanisms). As shown in Figure 16, notification manager 104 receives 16 notifications from four applications 1602.

[0044] The notification manager 104 prioritizes received notifications into priority groups. A priority group may contain two or more notifications from a set of received notifications, and may exclude one or more notifications from the set. The notification manager 104 determines the priority, such as the illustrated priority group of five notifications from the set of 16 notifications in Figure 16, based on the relationships between notifications, different importance levels of notifications, similarity, chronological order, a machine learning model, or prior user selection.

[0045] When prioritizing, the notification manager 104 presents a group of preferred notifications within the user interface 1604 via the user interface application 106, which is shown here in a single area 1606. More specifically, as shown in the exemplary environment 1600 of Figure 16, the notification manager 104 receives four email notifications 1602-1-1, 1602-1-2, 1602-1-3, and 1602-1-4 from the email application 1602-1, three SMS notifications 1602-2-1, 1602-2-2, and 1602-2-3 from the SMS application 1602-2, two calendar notifications 1602-3-1 and 1602-3-2 from the calendar application 1602-3, and seven notifications 1602-4-1, 1602-4-2, 1602-4-3, 1602-4-4, 1602-4-5, 1602-4-6, and 1602-4-7 from the social media application 1602-4.

[0046] Therefore, notification manager 104 receives 16 different notifications from 4 of the applications 1602. Notification manager 104 can determine priorities based on many factors, such as similar topics or more recent notifications. For example, email notification 1602-1-1 and email notification 1602-1-3 have the same subject, but email notification 1602-1-3 is more recent and describes a meeting that will take place that day. Therefore, notification manager 104 will prioritize email notification 1602-1-3 over email notification 1602-1-1. Notification manager 104 reads email notification 1602-1-2 as spam and therefore does not prioritize it, but notifications manager 104 marks email notification 1602-1-4 as urgent and therefore prioritizes it and adds it to the priority group. Notification manager 104 receives SMS notifications 1602-2-1 and 1602-2-3 and decides not to prioritize them (during working hours) because they are not work-related. SMS notification 1602-2-2 contains the names of people the user is meeting that day, and therefore notification manager 104 adds it to the priority group shown in user interface 1604 by prioritizing it. Notification manager 104 prioritizes calendar notification 1602-3-1 because it is a canceled event that requires action from the user. Calendar notification 1602-3-2 is determined to have the same content as email notification 1602-1-3, but the user prefers email notifications to calendar notifications based on user history. Therefore, notification manager 104 does not prioritize calendar notification 1602-3-2. The user has configured the user interface 1604 to show comment notifications from the social media application 1602-4 (compared to "likes"). Therefore, the notification manager 104 prioritizes social media notifications 1602-4-6 over "like" notifications 1602-4-1, 1602-4-2, 1602-4-4, 1602-4-5, and 1602-4-7.

[0047] The notification manager 104 can cause the user interface application 106 to present priority groups of notifications in many ways, such as by a feature-restricted set version associated with the application that receives each notification, or by excluding low-priority notifications and highlighting high-priority notifications but presenting many or all of the notifications, or by prioritizing groups by listing priority groups at the top of a list that includes non-priority notifications in a set, or by using some other visual priority. In the illustrated example, the notification manager 104 instructs the user interface application 106 to present selected notifications of the priority group within a single area 1606 of the user interface 1604 (explained in more detail in Figure 17). Instead of requiring the user to sift through 16 different notifications across four different applications 1602, the notification manager 104 presents only five notifications that are prioritized according to the user's needs within a single area 1606 of the user interface 1604.

[0048] More specifically, Figure 17 shows an exemplary embodiment 1700 of the user interface 1604 of Figure 16. The exemplary embodiment 1700 of Figure 17 includes a priority notification area 1702, calendar notifications 1602-3-1, email notifications 1602-1-3, email notifications 1602-1-4, SMS notifications 1602-2-2, and social media notifications 1602-4-6.

[0049] More specifically, the user interface 1604 in Figure 17 displays five priority notifications received by the notification manager 104 from the application 1602. In this illustrated example, the priority notifications in the priority group are selected by the notification manager 104 from 16 received notifications based on the required action or response (not just information), chronological importance (e.g., which notifications concern events today, which concern events next week), the relevance of the notifications (e.g., repetitive or useful / informational), and user history.

[0050] The first notification displayed is calendar notification 1602-3-1, which indicates that the user's flight for that day has been canceled. This calendar notification 1602-3-1 takes priority as the first notification because it includes what is happening today and the notification manager 104 determines that the user needs to change their schedule for the day immediately, thus requiring immediate action from the user (e.g., changing the flight, canceling other parts of the trip). The second notification displayed in the user interface 1604 is email notification 1602-1-3, which informs the user that there is a meeting today and is of chronological importance. These factors in determining priority can be determined by user selection, user history, and / or machine learning. For example, a user may set priority preferences to show notifications that require them to do something that day (e.g., attend a meeting).

[0051] The third notification displayed to the user is email notification 1602-1-4. Email notification 1602-1-4 is marked as urgent, which is used by notification manager 104 to determine priority, and notification manager 104 also determines that email notification 1602-1-4 is legitimate (not spam and associated with a known or authorized email address or company) and that it is chronologically important.

[0052] The fourth notification is SMS notification 1602-2-2, which refers to a specific name related to a previous email notification 1602-1-3. In some instances, the user may want the user interface 1604 to display notifications directly related to higher-priority notifications such as email notification 1602-1-3. In such cases, the notifications are related or associated in some way. This relevance can lower or raise the priority. If the relevance is repetitive, it can be lowered in priority, but in this case, the relevance is informative or useful to a higher-priority notification, in this case the previous email notification 1602-1-3.

[0053] The fifth and final notification in the presented priority group is a social media notification 1602-4-6. This notification informs the user that there are new comments on their recent social media posts. As described above, users can explicitly prioritize notification types, or the notification manager 104 can determine the priorities a particular user assigns to different types of notifications through user history and / or a machine learning model. Here, we assume that the user has a habit of choosing to look at more information regarding a notification that is a social media comment (e.g., reading all the comments themselves that are not shown in the priority group presentation, or reading through a list of previous notifications via the user interface of the social media application associated with the comment type of the notification).

[0054] Figure 18 shows an exemplary embodiment 1800 of the user interface 1604 of Figure 16. In the exemplary embodiment 1800 of Figure 18, the user interface application 106 provides a notification popup area 1802 in response to user selection (tap, hover) of a notification displayed within a priority group (e.g., displayed in the priority notification area 1702). The notification popup area 1802 here contains the entire notification, however space constraints for long notifications or non-text content may affect the amount of text or content displayed. For example, as shown in Figure 18, SMS notification 1602-2-2 is selected, and additional content related to this notification is presented within the notification popup area 1802, presenting the user with the full text of the notification. The notification popup area 1802 can popup for any selected notification in the user interface 1604. By enabling this type of expansion within the user interface 1604, the user interface application 106 allows the user to quickly and easily view additional content about a notification. The user does not need to explicitly open the SMS application 1602-2, change the format of the user interface 1604, or otherwise hinder user convenience. The user can simply tap or hover over a priority notification to view additional information related to the selected notification before moving on to another task. The notification popup area 1802 may disappear automatically in a short response, in response to a user selection of some other part of the user interface 1604, or simply when the user ends hovering over or maintaining a tap on the selected notification in the priority notification area 1702.

[0055] As another example, the notification popup area 1802 may display to the user the total number of notifications from the application associated with the selected notification, in this case the social media application 1602-4. As seen in Figure 18, when the SMS notification 1602-2-2 is selected, the notification popup area 1802 pops up with the full text of the notification. In the corner of the notification popup area 1802, the total number of notifications from the SMS application 1602-2 is displayed to the user. For example, if the user selects the social media notification 1602-4-6 below the SMS notification 1602-2, the notification popup area 1802 may not only display the full text of the notification but also indicate that there are 7 unread social media notifications.

[0056] While shown as notification popup area 1802, other forms of extended portions of user interface 1604 may be used, such as making a selected notification larger within a single area 1606 (for example, the text of notification popup area 1802 may be shown in the entire single area 1606, rather than being shown in part of the single area 1606 or not shown at all (not shown in Figure 18)).

[0057] Exemplary user selections include tapping, hovering, and tap-and-hold, but the selection method may include a mouse, touchpad, trackball, pointing stick, touchscreen, graphics tablet, or gesture-based input device.

[0058] Figure 19 shows an exemplary embodiment 1900 of the user interface 1604 of Figure 16. In the exemplary embodiment 1900 of Figure 19, the user interface application 106 provides an application feature restriction set version area 1902 in response to a user selection (tap, hover over) of a notification displayed in a priority group (e.g., displayed in the priority notification area 1702). The user interface application 106 may open ("launch") the application associated with the selected notification, provide separate but similar functionality to the application without launching it, or communicate with the application (assuming the application is already launched) to provide functionality for the feature restriction set version.

[0059] An exemplary embodiment 1900 shows an application functionality restriction set version area 1902 within a user interface 1604. Notifications within the user interface 1604 can be selected, moved, or pushed to open one of the applications 1602 within the application functionality restriction set version area 1902, which is presented to the user on the user interface 1604. For example, the user can select a notification (e.g., SMS notification 1602-2-2), which then opens the application functionality restriction set version area 1902, which is then instructed to present all notifications from that particular application (unread, or unread and read, etc.). For example, the technique here is to instruct the application functionality restriction set version area 1902 to present three unread SMS notifications, such as SMS notifications 1602-2-1, 1602-2-2, and 1602-2-3 in Figure 16, with the content of each notification added or all, as shown in the full content notifications 1904-2-1, 1904-2-2, and 1904-2-3 (respectively). The application functionality restriction set version area 1902 can be used to display the full content of the selected notification (or to the extent feasible), or otherwise. Also note that in this example, the application functionality restriction set version area 1902 can be replaced by the four other functionality restriction set versions shown in Figure 16. The user interface application 106 may or may not do so based on a variety of factors, including the size of the application feature restriction set version area 1902 required to display the relevant notifications or a single complete or nearly complete content notification, the user history or selections for how they want their user interface 1604 to behave, or the existence of application feature restriction set versions associated with selected notifications that already exist within the user interface 1604.Therefore, if one of the four feature restriction set versions in Figure 16 is associated with the application that received the selected notification, the full content or more of the selected notification may simply be presented on top of the existing feature restriction set version. Furthermore, the user interface 1604 may highlight the application feature restriction set version area 1902 in response to a selection or partial selection of a notification within the preferred notification area 1702. In this way, the user can quickly understand where more information about the notification will be presented once the selection is complete (e.g., from hovering to tapping).

[0060] In other embodiments, the user interface application 106 allows the user to interact with notifications presented within the application's functional restriction set version area 1902, which may include actions such as marking them as read or unread, deleting them, or responding to them, even if the application is not launched for the relevant notification. The application's functional restriction set version area 1902 may also display relevant metadata or other information, such as sender information, the date and time of receipt, and any associated attachments or links. The user interface application 106 can also facilitate the user's ability to switch between various applications and their associated notifications, which can be achieved, for example, through the use of tabs or dropdown menus. In these various examples of the user interface 1604, the technology allows the user to easily and simply navigate notifications based on priority, the applications associated with those notifications, and even lower-priority notifications.

[0061] Figure 20 shows another exemplary embodiment 2000 of the user interface 1604 from Figure 16, with different areas and their arrangement. The system 2000 includes a preferred notification area 2002 where preferred notifications from the four software applications 1602 in Figure 16 are placed. Note that this preferred notification area 2002 is larger than the preferred notification area 1702 in Figure 17 and shows six preferred groups of notifications. The user interface 1604 can be modified by user selection or based on the user's history or a machine learning model. For the exemplary embodiment 2000, these techniques add additional emphasis and space for notifications than those shown in Figure 16, where the preferred notification area 2002 is shown larger and more centrally positioned. Furthermore, the exemplary embodiment 2000 includes a notification summary area 2004 that presents the names of the applications from which notifications are received, here “Email” application 1602-1, SMS application 1602-2, calendar application 1602-3, and social media application 1602-4. Additionally, this notification summary area 2004 contains information about the notifications received and / or the set of notifications received but unread, including 15 of the previous 16 notifications, indicating that one of the notifications has been read and / or deleted. An additional priority notification for the priority group is shown in social media notification 1602-4-4, which indicates that a person named "Sophia" has started following the user.

[0062] As shown in Figures 16-20, technology enables many types and arrangements of user interfaces, including larger or smaller notification areas, feature-restricted set versions, and popup areas, which can be adjusted both by the user and through analysis of the user's or other users' behavior. However, through these user interfaces, users can save time and add convenience to notification management.

[0063] Inter-process communication Figure 21 shows an exemplary environment 2100 in which a technique can be implemented that enables a user interface integrating heterogeneous software applications through an inter-process communication mechanism. Environment 2100 includes heterogeneous software applications 2102, heterogeneous entities 2104, information 2106, and an exemplary user interface 2108. These are shown, for example, as the first heterogeneous software application 2102-1, the second heterogeneous software application 2102-2, the third heterogeneous software application 2102-3, and the fourth heterogeneous software application 2102-4; the heterogeneous entities 2104 are shown, for example, as the first heterogeneous entity 2104-1, the second heterogeneous entity 2104-2, the third heterogeneous entity 2104-3, and the fourth heterogeneous entity 2104-4; and the information 2106 is shown, for example, as the first information 2106-1, the second information 2106-2, the third information 2106-3, and the fourth information 2106-4.

[0064] The user interface application 106 receives information 2106 from heterogeneous entities 2104 via interprocess communication mechanisms (directly or through heterogeneous software applications 2102 to which each is associated). This information may be identical or different for each piece of received information. Consider, for example, interprocess communication (IPC) 2110, illustrated with four different IPCs 2110-1, 2110-2, 2110-3, and 2110-4. As partially described above, these various IPCs 2110 may include, to name just a few, application programming interfaces (APIs), remote procedure calls (RPCs), message queues, shared memory and databases, web services (e.g., Simple Object Access Protocol (SOAP) or Hypertext Transfer Protocol (HTTP)), or file-based communication.

[0065] In response to receiving information 2106, the user interface application 106 presents one or more instances of information 2106 in the user interface 2108, along with their associated widgets 2112. For example, the user interface application 106 presents the first part of the first information 2106-1 in the first widget 2112-1, the second part of the second information 2106-2 in the second widget 2112-2, the third part of the third information 2106-3 in the third widget 2112-3, and the fourth part of the fourth information 2106-4 in the fourth widget 2112-4. Each of these widgets 2112 may be a feature-limited set version of the full-featured version associated with the complete instance of the software application. The user interface application 106 may perform this action with part or part of the presented information 2106 simultaneously or sequentially (for example, by presenting the first piece of information 2106-1 and the second piece of information 2106-2 simultaneously or one at a time). The user interface application 106 may present the information 2106 in various forms and orientations, such as those shown in Figures 4 and 7-20.

[0066] Considering Figure 22 in two examples, Figure 22 shows an environment 2200 having exemplary first and second user interface instances 2202 and 2204, respectively. The first user interface instance 2202 shows a list-oriented presentation of widgets 2112 and their associated information 2106. The second user interface instance 2204 shows a tile-oriented presentation of widgets 2112 and their associated information 2106.

[0067] In these exemplary orientations, and in the orientations of other illustrated drawings described herein, the user interface application 106 allows selection of each piece of information 2106. This selection may be made in various ways, such as by hotkeys, mouse clicks, audio, and touch via a touch-enabled display. Here, the example shows selection by touch selection.

[0068] The environment 2200 includes a selectable control 2206, a touch selection 2208, additional information 2210, an associated instance 2106 of the information, and an associated widget 2112. These are represented, for example, by a first selectable control 2206-1, a second selectable control 2206-2, a third selectable control 2206-3, and a fourth selectable control 2206-4, and the additional information 2210 is represented, for example, by a first additional information 2210-1, a second additional information 2210-2, a third additional information 2210-3, and a fourth additional information 2210-4. The user interface application 106 enables the selection of at least a first portion of the first information 2106-1, or a selection associated with that portion, via a first selectable control 2206-1 associated with a first widget 2112-1 in the user interface instance 2202. As seen in Figure 22, each piece of information 2106 has its own selectable control 2206 at the same location within its associated widget 2112 (although this is not required). The widget 2112 may be of various types described or illustrated herein, for example, the first widget 2112-1 is a borderless window, active control, or thumbnail (as described and shown, for example, in Figure 10).

[0069] In response to receiving a selection of the first selectable control 2206-1, the user interface application 106 provides first additional information 2210-1 (or additional functionality) associated with the first information 2106-1 in the user interface instance 2204.

[0070] For example, the third piece of information 2106-3 may be a notification about a new message, and the associated widget 2112-3 may be an SMS application window. Thus, in 2208, in response to the user selecting the third selectable control 2206-3, the user interface application 106 provides third additional information 2210-3, which may be the entirety of the new message. The third piece of information 2106-3 includes "New message: From Amit, Amit: Were you able to read the preparation documents before the meeting?". Part 2212 of the third piece of information 2106-3, "New message: From Amit," is shown. The third piece of information 2210-3 includes "Amit: Were you able to read the preparation documents before the meeting?". This additional information 2210-3 is presented by the user interface 106 in response to the user's selection.

[0071] Figure 23 shows an exemplary environment 2300 in which a technique can be implemented to trigger the presentation of a complete instance of an interface for a software application. Triggering the presentation can be achieved automatically without any further effective user interaction to present additional information or the implementation of selected functions related to that information. This environment 2300 presents, via user interface 2302, a first complete instance 2304 of a first software application 2306, first information 2106-1, first selectable controls 2206-1, first additional information 2210-1, first IPC mechanism 2110-1, and user interface application 106.

[0072] Here, in response to receiving a selection of the first selectable control 2206-1, the user interface application 106 causes the presentation of a first complete instance 2304 of the first software application 2306 on the user interface 2302. The user interface application 106 also causes the first additional information 2210-1 to be automatically presented on the user interface 2302 via the first IPC mechanism 2110-1 after the first complete instance 2304 has been presented, without any further user interaction.

[0073] For example, the user interface application 106 may present a first complete instance 2304 of an email software application (e.g., a first software application 2306) on the user interface 2302. Within the first complete instance 2304, the user interface application 106 may present, via a first IPC mechanism 2110-1, a notification center with new and read emails (e.g., first information 2106-1), and additional functionality (e.g., first additional information 2210-1) that allows the user to choose to automatically respond to new emails.

[0074] In various embodiments, the user interface application 106 and / or the notification manager 104 perform actions based on a machine learning model. Many exemplary uses are described herein, such as using the machine learning model to consolidate content changes across multiple related software applications and / or to prioritize sets of notifications into preferred groups. This machine learning model can be determined based on a history of previous user behavior, such as by the user of the user interface 1604 and / or other users. The history of previous user behavior can be determined through user selection history (local or remote, one user or more). For example, the history of previous user behavior can be used to train the machine learning model. The history of previous user behavior may include multiple users not associated with the user interface. The history of previous user behavior may also include selection history showing selections by other preferred groups of other user interfaces associated with the user interface application.

[0075] Machine learning models, training, and techniques In various embodiments, the user interface application 106 and / or the notification manager 104 perform actions based on a machine learning model. Many exemplary uses are described herein, such as using the machine learning model to consolidate content changes across multiple related software applications and / or to prioritize sets of notifications into preferred groups. This machine learning model can be determined based on a history of previous user behavior, such as by the user of the user interface 1604 and / or other users. The history of previous user behavior can be determined through user selection history (local or remote, one user or more). For example, the history of previous user behavior can be used to train the machine learning model. The history of previous user behavior may include multiple users not associated with the user interface. The history of previous user behavior may also include selection history showing selections by other preferred groups of other user interfaces associated with the user interface application.

[0076] For the purposes of this disclosure, a machine learning model is any model that accepts inputs, analyzes and / or processes the inputs based on algorithms derived through machine learning training, and provides outputs. Model inputs (a set of notifications) can be analyzed or processed by the model to produce model outputs (e.g., preferred groups of notifications, content passed from the interface of one software application to the interface of another software application).

[0077] Figure 24 shows an exemplary machine learning block diagram 2400. Input 2402 is fed to training module 2404, which processes input 2402. Training module 2404 provides a predictive output 2406. The predictive output 2406 may represent a known correlation with input 2402, such as a preferred notification from a set of notifications. Machine learning may be used to integrate content changes. For example, a user may receive an email notification containing an event invitation, and a machine-trained model can identify relevant details of the event and automatically create a corresponding calendar entry without the user having to manually add the calendar entry. The training module 2404 may include, but is not limited to, several specified methods for processing input 2402 and producing output 2406, including, but not limited to, multilayer perceptron neural networks (MLP) 2408, convolutional neural networks (CNN) 2410, k-means clustering 2412, Boltzmann machines 2414, and recurrent neural networks (RNNs). Using input 2402, training module 2404, and output 2406, a machine learning model M2416 can be generated.

[0078] In some examples, input 2402 may be a training input labeled with known output correlation values, and these known values ​​can be used to optimize output 2406 during training using an optimization / loss function.

[0079] The disclosed machine learning method involves collecting data about the user's history from various sources, including but not limited to user interactions with websites, mobile apps, and social media activity. Next, relevant features are extracted from the collected data (e.g., a CNN), which involves selecting the most important information to use for making predictions. Relevant features may include, for example, the user's interaction history with notifications, the user's location, and the times when the user is most active. As an example, a user might receive an SMS message from a colleague asking for help with a task. Based on relevant features such as the user's task priority level, preferred contacts, and task categories the user tends to engage with, the model may create a task within a task management application corresponding to the details from the SMS message. After feature extraction, an appropriate machine learning algorithm is selected based on the characteristics of the data and the type of problem being solved. A supervised learning algorithm may be used, in which a model (e.g., pre-trained model 2608) is trained on labeled data (e.g., label 2606 in Figure 26). During training, the algorithm adjusts its internal parameters to minimize the difference between its predictions and the actual importance levels of notifications in the training data. Once trained, the model can be deployed to predict the importance of new notifications based on the user's history. These predictions can then be used to deliver important notifications to the user in a timely and relevant manner. For example, a user might have social media applications that share notifications about likes, comments, and new posts. Based on training with labeled data, the model knows that this user frequently engages with posts related to a particular topic, or posts from specific friends or content creators. Using this information, the model can send the most important notifications to the user to quickly update and inform them about posts from friends or favorite sports players, for example. This ensures that the user stays connected to the social media network but is not flooded with irrelevant and meaningless notifications.

[0080] In some embodiments, the machine learning model described herein may be trained in a training computing system as described above and then provided to one or more computing devices for storage and / or execution. For example, a model trainer may be located in the training computing system. The training computing system may be comprised of or separate from one or more computing devices that implement the machine learning model. As an example, Figure 25 shows a block diagram 2500 of an exemplary computing device 2502 having one or more processing devices 2504, a memory device 2506 having data 2508 and instructions 2510, and a machine learning model 2512, wherein the exemplary computing device 2502 communicates with an exemplary training computing system 2522 having one or more processing devices 2524, a memory device 2526 having data 2528 and instructions 2530, and a model trainer 2532.

[0081] In some embodiments, the machine learning model 2512 can be trained offline or online. In offline training (also called batch training), the model is trained across a static training dataset. In online training, the model is continuously trained (or retrained) as new training data becomes available (for example, while performing inference using the model).

[0082] Model trainer 2532 can train, update, or personalize the machine learning model 2512 by enabling centralized training (e.g., based on a centrally stored dataset) or decentralized training (e.g., distributed training or federated learning) of the machine learning model 2512.

[0083] The machine learning models described herein can be trained according to one or more of various different training types or techniques. For example, a machine learning model can be trained using supervised learning, in which the machine learning model is trained on a training dataset that includes instances or examples with labels. Labels can be applied manually by experts, generated through crowdsourcing, or provided by other techniques (e.g., physics-based or complex mathematical models). In some embodiments, training examples can be provided by a user computing device 2502 if the user gives consent (e.g., for the user's notification selection history). For example, the user can select a note in a note application and associate it with other selected documents in a word processing application. By using supervised learning to train the model based on examples such as those described above, the machine learning model can learn to recognize patterns and correlations between specific notes and documents, such as a common project or a particular meeting. When the user selects a note, the model can suggest integrating the note with the corresponding document because the model has determined that there is a common task related between the two. In some embodiments, this process can be called model personalization. In some instances, the model can be trained using unsupervised learning techniques, where the training data is unlabeled and correlations are found by the machine learning training module 2404, independent of explicit labels applied before machine learning training. In some examples, a combination of supervised and unsupervised methods may be seen in the same machine learning training.

[0084] As an example, Figure 26 shows a block diagram of an exemplary training process 2600 in which a machine learning model 2608 is trained using training data 2602, which includes exemplary input data 2604 with label 2606. The machine learning model 2608 produces output data 2610 (e.g., priority groups for notifications). As an example, suppose a user has a work task they are focused on. During the training of the model 2608, which will be used by the user interface application 106 or the notification manager 104, the input data 2604 consists of messages, notes, or other tasks, where the other tasks may be common tasks or related projects, etc., to train the model 2608 to recognize patterns or correlations between the software application data. Once training is complete, the machine learning model 2608 can be used to consolidate the content and prioritize notifications to the user. One example is that the machine learning model 2608 can automatically associate new messages from colleagues with the user's current work task they are focused on. The machine learning model 2608 can identify associations between notifications and applications via training data 2602 (e.g., previous user history). This integration allows users to easily access all relevant information from their applications. Other training processes besides the exemplary process shown in Figure 26 can also be used, including processes that do not use labels 2606.

[0085] The machine learning model 2608 can be trained by optimizing the objective function 2612. For example, in some embodiments, the objective function 2612 may be a loss function that compares (e.g., determines the difference between) the output data 2610 generated by the model 2608 from the training data 2602 with the labels 2606 associated with the training data 2602 (e.g., ground truth labels). For example, the loss function could evaluate the sum or mean of the squared differences between the output data 2610 and the labels 2606. In other examples, the objective function 2612 may be a cost function that represents the cost of a particular outcome or output data, or may include a cost function. Other objective functions 2612 may include margin-based techniques such as triplet loss or maximum margin training.

[0086] An exemplary method for integrating heterogeneous software applications using inter-process communication mechanisms. Figure 27 shows an exemplary method 2700 of a technique that enables a user interface for integrating heterogeneous software applications through one or more inter-process communication mechanisms. Method 2700 is presented as a set of blocks specifying the actions to be implemented, but is not necessarily limited to the order or combination shown for each block to implement the actions. Furthermore, any one or more actions may be repeated, combined, rearranged, or linked to provide a variety of additional and / or alternative methods. Some parts of the following description may refer to one of the aforementioned drawings or to entities or processes detailed in other drawings, but these references are illustrative examples only. These techniques are not limited to implementation by one or more entities operating on a single device.

[0087] In step 2702, the first information is received by the user interface application from a first heterogeneous entity associated with the first information via a first inter-process communication (IPC) mechanism. As shown in Figure 21, the user interface application 106 receives information 2106 from the heterogeneous entity 2104 via the IPC mechanism 2110. Information 2106 is associated with the heterogeneous entity 2104. More specifically, the user interface application 106 receives the first information 2106-1 from the first heterogeneous entity 2104-1 via the first IPC mechanism 2110-1.

[0088] In step 2704, the second information is received by the user interface application from a second heterogeneous entity associated with the second information via a second IPC mechanism. As shown in Figure 21, the user interface application 106 receives the second information 2106-2 from the second heterogeneous entity 2104-2 via the second IPC mechanism 2110-2.

[0089] In 2706, the user interface application presents, within the user interface, at least a first portion of the first information in a first widget and at least a second portion of the second information in a second widget. As seen in Figure 21, the user interface application 106 presents the first portion of the first information 2106-1 in the first widget 2112-1 within the user interface 2108. The user interface application 106 may present the information 2106-1, 2106-2, 2106-3, and 2106-4 in parallel (e.g., simultaneously) or one at a time. The user interface application 106 may also present the information 2106 in various orientations, such as those shown in Figures 4 and 7-20. Thus, the user interface application 106 may present multiple widgets 2112, each separately within its own area of ​​the user interface 2108, or combined within the same area of ​​the user interface. Furthermore, the user interface application 106 may present information in a format different from the format that may be presented in the complete user interface instance of its associated software application. In doing so, the user experience can be improved by, in some cases, consistently presenting information that might otherwise be in different formats.

[0090] In 2708, the user interface application enables the user to select at least a first portion of the first information and at least a second portion of the second information, or controls associated therewith, through the user interface via a first selectable control associated with the first widget and a second selectable control associated with the second widget, respectively. As shown in Figure 22, the user interface application 106 enables the user to select the first information 2106-1 via a first selectable control 2206-1 associated with the first widget 2112-1. Furthermore, the user interface application 106 enables the selection of information 2106-2, 2106-3, and 2106-4 via selectable controls 2206-2, 2206-3, and 2206-4, respectively.

[0091] In some cases, heterogeneous software applications include similar functions such as delete, zoom in / out, or respond, which perform actions such as deleting text or image portions, zooming in or out of text or images, or responding to notifications, respectively. If complete instances of each heterogeneous software application were presented instead, the user may need to find these similar functions and navigate to them in different locations, icons, selection methods, etc., in the heterogeneous complete instances of the heterogeneous software applications. However, the user interface application 106 resolves this inferior user experience by enabling controls located in the same position within different widgets (e.g., the upper right corner of each of the first and second widgets), and the controls enable similar functions. For example, selecting to cut, copy, or paste information presented within a widget can be achieved within each widget using the same user experience (e.g., the same icon and position).

[0092] Alternatively, a single control can be used for multiple widgets for similar functionality, such as a single selectable image, icon, or button that enables the implementation of functionality for one or more of the widgets. One example of such a method is to choose to open the complete instance in a single control within a user interface that presents multiple widgets, where the complete instance associated with the last interacted widget, or the most recent update or change to that widget, is opened.

[0093] Both of these methods can save time and reduce user confusion, thereby improving the user experience. However, if the user wants to fully engage with the complete instance of the interface for the software application associated with the presented information, the technology allows for the option of presenting the complete instance, as shown in Figure 23.

[0094] In 2710, the user interface application may, in response to receiving a selection of a first selectable control or a second selectable control, provide first or second additional information or additional functionality associated with the first or second information, respectively. The user interface application provides additional information within the user interface independently of the execution of the first software application associated with the first information and independently of the execution of the second software application associated with the second information. As shown in Figure 22, when a selectable control 2206 is selected, the user interface application 106 provides additional information 2210 associated with information 2106 within the user interface 2204.

[0095] Alternatively, or in addition to 2710, in 2712, a user interface application may, in response to a selection of a first selectable control, trigger the presentation of a first complete instance of the interface of a first software application, or a second complete instance of the interface of a second software application. The user interface application may automatically, without further user interaction, trigger the presentation of additional first or additional second information within the first or second complete instance, respectively. Furthermore, by triggering the presentation of the first complete instance of the interface of the first software application, the first complete instance may be presented within the user interface (e.g., replacing most or all of the user interface area currently presented by user interface application 106), or separately in a new window, etc. As partially described above, the presentation of a first complete instance of the interface of a first software application within a user interface (e.g., user interfaces 602, 910, 1502, or 2302) may involve communication with related heterogeneous entities or software applications.

[0096] This can be partially achieved by using an IPC mechanism, such as by the user interface 106 passing selections and / or selected information or functions to a heterogeneous entity associated with a software application that can present a first complete instance. As can be seen in Figure 23, in response to the user selecting a first selectable control 2206-1 within the user interface 2302, the user interface application 106 triggers the presentation of a first complete instance 2304 of the first software application 2306 by a first IPC mechanism 2110-1. The user interface application 106 automatically presents the first information 2106-1 and the first additional information 2210-1 within the first complete instance 2304. Thus, with a single selection, the technique allows the user to select information or functions for that information to be presented in a complete instance. The user does not need to individually open / launch the appropriate complete instance of the interface and the information or functions to be presented or realized. Instead, this is realized by the user interface application 106 in response to the above selection (e.g., user selection 2208 in Figure 22).

[0097] Exemplary methods that enable simplified and integrated interaction with multiple applications Figure 28 shows an exemplary method 2800 of a technique that enables a user interface for simplified and integrated interaction with multiple applications. Method 2800 is presented as a set of blocks specifying the actions to be implemented, but is not necessarily limited to the order or combination shown for each block to implement the actions. Furthermore, any one or more actions may be repeated, combined, rearranged, or linked to provide a variety of additional and / or alternative methods. Some parts of the following description may refer to one of the aforementioned drawings or to entities or processes detailed in other drawings, but these references are illustrative only. These techniques are not limited to implementation by one or more entities operating on a single device.

[0098] In 2802, two or more selectable controls within the user interface control area are presented within the user interface by the user interface application, and each of these two or more selectable controls is associated with two or more software applications. As shown in Figure 2, user interface application 106 presents three selectable controls 206, 208, and 210 within user interface 202. The three selectable controls 206, 208, and 210 are located within the control area 204 within user interface 202 and are associated with the first, second, and third associated software applications 212, 214, and 216. The three associated software applications 212, 214, and 216 are provided by user interface application 106.

[0099] In 2804, a user interface application may allow the selection of two or more selectable controls. As seen in Figure 2, user interface application 106 allows the selection of three selectable controls 206, 208, and 210 within the control area 204. A cursor is shown in Figure 2 to illustrate the user interface application 106 that enables the selection.

[0100] In 2806, in response to a first selection of a first selectable control, the user interface application presents a first feature-limited set version of the first full-featured set version associated with a first full instance of the interface of the first software application. As shown in Figure 12, the first full-featured set version 1210 and the first feature-limited set version 1212 are presented to the user interface 1214 by the user interface application 106 and associated with the first software application 1202 (e.g., the calendar application 1202). The feature-limited set version 1212 is smaller than the full-featured set version 1210. As shown, the full-featured set version 1210 includes additional content, a larger size and resolution, and additional features.

[0101] In 2808, in response to a second selection of a second selectable control, the user interface application presents a second feature-limited set version of the second full feature-limited version associated with the second full instance of the interface of the second software application. As seen in Figure 13, the user interface application 106 presents a second feature-limited set version 1308 associated with the software application 1316 (e.g., the memo application 1316). A smaller feature-limited set version 1310 is also included, illustrating part of the possible size range of feature-limited set versions.

[0102] In 2810, the user interface application enables interaction with the first content presented through a first feature-limited set version of the first software application, as seen in Figures 6-10 and 12-15.

[0103] In 2812, the user interface application modifies the second content of the second feature restriction set version of the second software application, and the modification is based on interaction with the first content enabled through the first feature restriction set version of the first software application. As shown in Figure 6 (and other figures), the user interface application 106 modifies content within two separate but related software applications 612 and 614. The first content 604 is within the first feature restriction set version 608 associated with the first software application 612, and the second content 606 is within the second feature restriction set version 610 associated with the second software application 614. The first content 604 is modified by the user, and since the first and second software applications 612 and 614 are related to each other, the second content 606 is automatically modified by the user interface application 106, and the modification is presented to the second feature restriction set version 610 associated with the second software application 614. For example, a user may want to remove a specific portion of notes from a meeting. When selected by the user, the content within the notes application (e.g., the first software application 612) (e.g., the first content 604) is not only immediately modified by the user interface application, but the content within the team meeting application (e.g., the second software application 614) (e.g., the second content 606) is also automatically adjusted to match the changes in the content from the notes application.

[0104] An exemplary method for enabling a multi-application user interface to prioritize notifications. Figure 29 illustrates Method 2900, a technique for enabling a multi-application user interface to prioritize notifications. Method 2900 is presented as a set of blocks specifying the actions to be implemented, but is not necessarily limited to the order or combination shown for each block to implement the actions. Furthermore, any one or more actions may be repeated, combined, rearranged, or linked to provide a variety of additional and / or alternative methods. Some parts of the following description may refer to the exemplary operating environment 1600 in Figure 16, or entities or processes detailed in other drawings, but these references are illustrative only. These techniques are not limited to implementation by one or more entities operating on a single device.

[0105] In 2902, a set of notifications associated with at least the first and second software applications is received by the user interface application. As seen in Figure 16, the notification manager 104 receives notifications from at least two applications 1602. As shown in the illustration, the four applications 1602 have 16 different notifications. Email application 1602-1 has 4 notifications, SMS application 1602-2 has 3 notifications, calendar application 1602-3 has 2 notifications, and social media application 1602-4 has 7 notifications. While these notifications may come in separately and / or periodically, the notification manager 104 is configured to interact with multiple applications 1602, such as by receiving notifications from each application 1602 as a group over a period of time. The notification manager 104 may receive notifications in a variety of ways, including but not limited to push notifications, API notifications, passive notifications, local notifications, and in-app notifications.

[0106] In 2904, sets of notifications are prioritized into priority groups. A priority group includes two or more notifications from a set of notifications and excludes one or more notifications from that set. Notification manager 104 determines the prioritization based on relationships between notifications, different levels of importance of notifications, machine learning models, or prior user selection, such as illustrated priority groups of five or six notifications from 16 sets of notifications (in Figures 16 and 20, respectively).

[0107] For example, the notification manager 104 can prioritize sets of notifications based on the relationships between them. Sets of notifications have relationships based on the fact that the notifications relate to a single project, such as a substance-based task associated with both the first and second applications. For example, the notification manager 104 can determine a single project based on the user's application usage history. For example, application usage may indicate that one of the first and second applications is used more than the other, or that it is used more across other applications that are also associated with the described user interface and set of notifications. Furthermore, the notification manager 104 can prioritize sets of notifications based on whether they relate to the same work project, such as through analysis of metadata and / or text within the notifications that indicate relevance to a topic or project (e.g., a work project where the user needs to go and meet a person named "Bill"), because the notifications share common terminology, metadata, or people.

[0108] In another embodiment, the notification manager 104 may prioritize a set of notifications based on the different importance of the notifications within that set. The notification manager 104 determines the different importance based on the response deadline, which is determined from within the notification text or from the metadata associated with the notification.

[0109] In other instances, the notification manager 104 prioritizes sets of notifications into preferred groups based on prior user selections, such as the user associating a first and second application with a project, or based on the user's configuration of the user interface 1604 (for example, placing feature-limited set versions of notifications on the user interface 1604 for a work calendar application, a work email application, and work-related social media accounts). Users can configure the user interface 1604 by selecting feature-limited set versions to display on the user interface 1604's homepage and adjusting their sizes, etc. Feature-limited set versions may be associated with a first and second application (for example, as shown in Figures 16 and 20). The notification manager 104 may also prioritize sets of notifications into preferred groups based on shared project names, common groups of employees and / or vendors, or various metadata about parts of the notification.

[0110] The process of prioritizing a set of notifications involves removing some or more notifications from the set, leaving a priority group. Notification manager 104 may remove notifications from a set of notifications based on the age of the removed notifications. The removed notifications may be older than one of the remaining notifications, and the removed notifications and the remaining notifications may be associated through a common group, task, or project, or through a single application of a first and second application. For example, notifications in a set of notifications may include email, SMS messages or texts, information received via an API, push notifications, social media likes, social media mentions, tags, calendar events, missed video chats, or missed calls.

[0111] In 2906, the notification preference group is presented by the user interface application within a single area of ​​the user interface. The preference group may be presented as shown in Figures 17, 18, 19, or 20. The notification preference group may be within a single area of ​​the user interface or within a feature-restricted set version area (e.g., the preference notification area 2002 in Figure 20). The user interface application occupies substantially the majority of the display of a mobile electronic device, such as a touch-sensitive display on a smartphone or tablet, and is large enough on a larger display to cover multiple areas, each having at least one notification area.

[0112] In operation 2908, the user interface application may allow the selection of one notification from a group of notifications in a priority group via the notification area. In response to the selection of a notification via the notification area, the notification manager 104 may open or access the software application associated with the selected notification (for example, in operation 2902, the notification was received from the software application or from a server entity associated with the software application). (As shown in Figure 16) The user interface application 106 may cause the software application to open a notification user interface showing additional information about the notification and provide the user interface application with additional information to present in other areas (for example, the notification popup area 1802 in Figure 18). Furthermore, the user interface application 106 may stop presenting the notification in areas corresponding to the additional information presented in the feature restriction set version area (for example, the application feature restriction set version area 1902 in Figure 19).

[0113] In one embodiment, the notification in the priority group of notifications is an email notification. The priority group of notifications presented may be a portion of the email associated with the email notification. Furthermore, in response to the selection of notifications, the user interface may present additional portions of the email. These additional portions may be substantially all of the email text presented within the extension of the user interface, a feature-limited set version associated with an email service application and present within the user interface, or the user interface of the email service application. In another embodiment, the notification in the priority group of multiple notifications may be an SMS notification, a social media notification, a task management notification, a productivity notification, a calendar notification, a document editing notification, a video conferencing notification, a phone call notification, or a collaboration update notification.

[0114] As described above, the user interface can allow the selection of one notification from among multiple notifications in a priority group via a single area, such as the priority notification area 1702 in Figure 17.

[0115] In 2910, in response to the selection of a notification, the user interface application instructs the software application associated with the selected notification to open, launch, install, access, or otherwise utilize or simulate it. In doing so, the user interface 1604, which functions as a software application or simulation, is instructed to open the notification user interface, such as the application functionality limit set version area 1902 in Figure 19.

[0116] Throughout this disclosure, computing systems (e.g., UE102 in Figure 1, client devices, server devices, computers, or other types of computing systems) are described as being able to analyze user-related information (e.g., heterogeneous software application data), for example, the heterogeneous software applications described with respect to Figure 21. In addition to the above, users may be given controls that allow them to choose whether and when the systems, programs, and / or features described herein can enable the collection of information (e.g., content related to the user's applications, such as notifications, user behavior, and other information). Computing systems may be configured to use information only after receiving explicit permission from the user to use the data. Furthermore, individual users may always have control over what programs can and cannot do with the information. Furthermore, collected information may be preprocessed in one or more ways so that personally identifiable information is removed before it is transferred, stored, or otherwise used. For example, a user's identity may be processed in such a way that personally identifiable information about the user cannot be determined, or the user's geographical location may be generalized so that location information (e.g., city, zip code, or state level) cannot be obtained in a way that the user's specific location cannot be determined. Thus, users can control whether information about themselves and their devices is collected, and if such information is collected, how it is used by computing systems and / or remote computing systems.

[0117] Examples Various embodiments are described herein, including a first exemplary method (Example 1) for receiving a set of notifications in a user interface application associated with at least a first and a second software application. The set of notifications includes one or more first notifications associated with the first software application and one or more second notifications associated with the second software application. The method prioritizes the set of notifications into priority groups, each priority group including two or more of the notifications in the set of notifications and excluding one or more of the notifications in the set of notifications. The method prioritizes based on the relationships between the notifications in the set of notifications and the different importance of the notifications. The method can then present, or trigger, the priority groups of notifications within a single area of ​​the user interface via the user interface application.

[0118] Example 2: The method according to Example 1, wherein the notification of the priority group of notifications is an email notification, and presenting the priority group of notifications further includes presenting a portion of the email associated with the email notification, and presenting additional portions of the email in response to the selection of the notification.

[0119] Example 3: The method according to Example 2, wherein the presentation of the additional portion of the email is to present substantially all of the text of the email, the presentation is within an extended portion of the user interface, and the feature-limited set version associated with the email service application is within the user interface, or within the user interface of the electronic service application.

[0120] Example 4: The method according to Example 1, wherein the notification of the priority group of notifications is an SMS notification, and presenting the priority group of notifications further includes presenting a portion of the SMS associated with the SMS notification, and presenting additional portions of the SMS in response to the selection of the notification.

[0121] Example 5: The method according to Example 4, wherein the presentation of the additional portion of the SMS is to present substantially all of the text of the SMS, the presentation is within the extended portion of the user interface, and the feature-limited set version associated with the SMS service application is within the user interface, or within the user interface of the SMS service application.

[0122] Example 6: The method according to Example 1, wherein the notification of the preferred group of notifications is a social media notification, and presenting the preferred group of notifications further includes presenting a portion of the social media associated with the social media notification, and presenting additional portions of the social media in response to the selection of the notification.

[0123] Example 7: The method according to Example 6, wherein the presentation of the additional portion of the social media is to present substantially all of the text of the social media, the presentation is within the extended portion of the user interface, and the feature-limited set version associated with the social media service application is within the user interface, or within the user interface of the social media service application.

[0124] Example 8: The method according to any one of the prior embodiments, wherein the set of notifications is received over a period of time and via push notifications from the first and second software applications.

[0125] Example 9: The method according to Example 1, wherein prioritizing the set of notifications into the priority group is the same as prioritizing the set of notifications, removing notifications from the set of notifications, and the remaining notifications from the set of notifications are the priority group.

[0126] Example 10: The method according to Example 9, wherein the removal of a notification from the set of notifications is based on the age of the removed notification, which is older than one of the remaining notifications, and the removed notification and the remaining notifications are associated through a common group, task, or project, or through a single software application of the first and second software applications.

[0127] Example 11: The method according to Example 1, wherein the first software application is a calendar application, the first related notification is a calendar event associated with a project, the second software application is an information generation application, and the second related notification is a notification that a change has been made, a change has been requested, or consent has been requested for an information document of the information collection application associated with the first related notification.

[0128] Example 12: The method according to Example 1, wherein the first software application is a social media application, and the first related notification is a new message, like, comment, follow, post, or tag in the social media application associated with a project, and the second software application is a task management application, and the second related notification is a task reminder, task assignment, task update, or task comment associated with a task in the task management application associated with the first related notification.

[0129] Example 13: The method according to Example 1, wherein the first software application is an email application, and the first associated notifications are new emails, email delivery failures, calendar invitations, or meeting reminders associated with a project, and the second software application is a contact management application, and the second associated notifications are new contacts, contact deletions, contact updates, or lead status associated with the first associated notification.

[0130] Example 14: The method according to Example 1, wherein the first software application is an email application, and the first related notifications are new emails, email delivery failures, calendar invitations, or reminders for meetings related to a project, and the second software application is a writing enhancement application, and the second related notifications are grammatical suggestions, tone and formality suggestions, or vocabulary enhancement suggestions associated with the first related notification.

[0131] Example 15: The method according to Example 1, wherein the first software application is a document editing application, the first related notification is an update, new comment, or change made by another user associated with the project, the second software application is a writing enhancement application, and the second related notification is a plagiarism alert, writing style suggestion, clarity and conciseness suggestion, or grammar and spelling error alert associated with the first related notification.

[0132] Example 16: The method according to Example 1, wherein the first software application is a customer relationship management application, the first related notification is a customer activity or team activity alert associated with a project, the second software application is a user authentication application, and the second related notification is a login, security alert, or privacy alert associated with the first related notification.

[0133] Example 17: The method according to Example 1, wherein the first software application is a memo application, the first related notification is a reminder, collaboration update, or sync update associated with a project, the second software application is a mind mapping application, and the second related notification is a new or updated diagram integrating the memo associated with the first related notification.

[0134] Example 18: The method according to Example 1, wherein the first software application is an SMS application, and the first related notifications are new SMS messages associated with a project, SMS read receipts, group updates, SMS deliveries, SMS delivery failures, or spam alerts, and the second software application is a project management application, and the second related notifications are new tasks or calendar reminders, new activities, or collaboration alerts associated with the first related notifications.

[0135] Example 19: The method according to Example 1, wherein the first software application is a video conferencing application, the first related notification is a reminder or invitation for a meeting associated with a project, the second software application is an automated note-taking application, and the second related notification is a new note alert, note update, note sharing, or collaboration alert associated with the first related notification.

[0136] Example 20: The method according to Example 1, wherein the prioritization based on the relationships between notifications within the set of notifications and the different importance of the notifications uses a machine learning model.

[0137] Example 21: The method according to Example 20, wherein the machine learning model is constructed based on a history of user behavior.

[0138] Example 22: The method according to any one of Examples 1 to 21, wherein prioritizing the set of notifications to the priority group is based on a prior user selection that associates the first and second applications with the project.

[0139] Example 23: A computer-readable storage medium comprising an instruction that causes the processor to implement one of the methods described in Examples 1 to 22 in response to execution by the processor.

[0140] Example 24: An electronic computing device comprising at least one processor and a computer-readable storage medium containing instructions that, in response to execution by the at least one processor, instruct the electronic computing device to implement any one of the methods described in Examples 1 to 22.

[0141] Various embodiments, including Method 25 of the Embodiment, are described herein. The Method of Embodiment 25 implements an operation in which a user interface application presents two or more selectable controls within the user interface, within the control area of ​​the user interface, each of the two or more selectable controls being associated with two or more software applications. The Method of Embodiment then implements an operation in which the selection of the two or more selectable controls is enabled, and in response to a first selection of a first selected control among the two or more selectable controls associated with a first software application of the two or more software applications, a first feature-limited set version of a first full feature-set version associated with a first full instance of the interface of the first software application is presented within the first area of ​​the user interface. The Method of Embodiment then implements an operation in which, in response to a second selection of a second selected control among the two or more selectable controls associated with a second software application of the two or more software applications, a second feature-limited set version of a second full feature-set version associated with a second full instance of the interface of the second software application is presented within the second area of ​​the user interface. The method of the embodiment further includes enabling interaction with first content presented through the first feature-limited set version of the first software application through the first area of ​​the user interface, and modifying second content of the second feature-limited set version of the second software application, wherein the modification is based on the interaction with the first content enabled through the first feature-limited set version of the first software application.

[0142] Example 26: The method according to Example 25, further comprising changing the size or orientation of both the first and second regions in response to the selection of the first region or the first selectable control.

[0143] Example 27: The method according to Example 26, wherein changing the size or orientation of both the first and second regions is performed automatically on the second region without user interaction or selection of the second region.

[0144] Example 28: Modifying the second content of the second feature-limit set version of the second software application is the method of Example 25, based on a prior association between the first software application and the second software application.

[0145] Example 29: The method according to Example 28, wherein the prior association is based on an explicit user selection that associates the first and second software applications with substance-based tasks.

[0146] Example 30: The method of Example 28, wherein the prior association is based on an implicit user selection that associates the first and second software applications through a selection that orients the first and second applications within the user interface.

[0147] Example 31: The method according to Example 25, wherein modifying the second content of the second feature-limit set version of the second software application is based on an explicit user selection enabled through a selectable user control associated with the first or second area.

[0148] Example 32: The method according to Example 25, wherein modifying the second content is in response to interaction with the first content presented via the first feature-limited set version of the first software application.

[0149] Example 33: The method according to Example 32, wherein the interaction with the first content adds, deletes, or modifies the first content, and the modification to the second content adds, deletes, or modifies the second content.

[0150] Example 34: The method of Example 32, further comprising passing information associated with the interaction with the first content to the second software application, and modifying the second content in the second feature-limit set version of the second software application in response to passing the information to the second software application and in response to information received from the second software application.

[0151] Example 35: The method of Example 25, further comprising passing information associated with the interaction with the first content to the second software application, and receiving changes or notifications received from the second software application, wherein the changes or notifications are based on the information associated with the interaction, and the method further comprises presenting the received changes or notifications within the user interface.

[0152] Example 36: The method of Example 25, further comprising presenting one or more instances of content or functionality associated with the first software application in response to an initial or intermediate selection of the first control among the two or more selectable controls, enabling the selection of the one or more instances of content or functionality associated with the first software application, and receiving the selection of a selected instance among the one or more instances, wherein presenting the first feature-limited set version within the first area of ​​the user interface is the method of presenting the selected instance among the one or more instances.

[0153] Example 37: The method of Example 36, wherein presenting one or more instances of the content or functionality associated with the first software application is to present a version of the content or functionality in which the size of one or more instances is reduced, the resolution is lowered, the functionality is limited, or the content is reduced.

[0154] Example 38: The method according to Example 37, wherein the presentation of one or more instances of the content or functionality associated with the first software application is one at a time.

[0155] Example 39: The method according to Example 25, wherein enabling the selection of two or more selectable controls allows the drag-and-drop selection of the first selected control into the user interface, and the presentation responds to the drag-and-drop of the first selected control.

[0156] Example 40: The method of Example 25, wherein the first and second regions are of different sizes and positions within the user interface, enabling the selection of two or more selectable controls, enabling the selection of regions of different sizes, and presenting the first functionally limited set version within the first region in response to the selection of the first region.

[0157] Example 41: The method of Example 40, wherein the first area is larger than the second area, and presenting the first feature-limited set version within the first area presents additional content in relation to the presentation in the second area.

[0158] Example 42: The method according to Example 41, wherein the first region is larger than the second region, and presenting the first functionally limited set version within the first region allows for additional functionality in relation to the presentation in the second region.

[0159] Example 43: The method according to Example 25, wherein the first feature-limited set version is a feature-limited set version, a borderless window, an active control, a thumbnail, or an applet, and the first complete instance of the interface for the first software application is the full feature instance associated with the execution instance of the first software application.

[0160] Example 44: The method according to Example 25, wherein the first content includes one or more selectable functions of the first feature-limit set version of the first software application, enabling interaction with the first content enables selection of the one or more selectable functions, and modifying the second content of the second feature-limit set version of the second software application is based on the first selection of the first selected function from the one or more selectable functions of the first feature-limit set version of the first software application.

[0161] Example 45: The method according to Example 25, wherein the first content includes visual information presented within the first feature-limited set version of the first software application, enabling interaction with the first content enables modification of the visual information, and modifying the second content of the second feature-limited set version of the second software application is based on selected modifications to the visual information presented within the first feature-limited set version of the first software application.

[0162] Example 46: The method according to Example 45, wherein the visual information includes some or all of an email, SMS message, social media comment, calendar event, word processing document, or image.

[0163] Example 47: Modifying the second content of the second feature-limited set version of the second software application based on interaction with the first content enabled through the first feature-limited set version of the first software application, is the method of Example 25, wherein the second content is modified based on a machine learning model.

[0164] Example 48: The method according to Example 47, wherein the machine-learned model is based on a history of previous user behavior, and the history of previous user behavior is used as truth data for training the machine-learned model.

[0165] Example 49: The method of Example 48, wherein the prior user behavior includes a prior interaction with the second content by the user receiving the interaction with the first content.

[0166] Example 50: A computer-readable storage medium comprising instructions that cause the processor to implement, either alone or in combination, any one of the methods described in Examples 25 to 49 in response to execution by the processor.

[0167] Example 51: An electronic computing device comprising at least one processor and a computer-readable storage medium containing instructions that, in response to execution by the at least one processor, cause the electronic computing device to implement, alone or in combination, any one of the methods described in Examples 25 to 49.

[0168] Various embodiments, including Method 52 of the Embodiment, are described. The Method of Embodiment 52 implements an operation which includes a user interface application receiving first information by a first inter-process communication (IPC) mechanism, the first information being from a first heterogeneous entity associated with the first information, and the operation further includes receiving second information by a second IPC mechanism, the second information being from a second heterogeneous entity associated with the second information. The Method of the Embodiment then implements an operation which includes a user interface application presenting, at least partially simultaneously, within the user interface, at least a first portion of the first information in a first widget and at least a second portion of the second information in a second widget. The Method of the Embodiment then implements an operation which includes enabling selection of at least a first portion of the first information and at least a second portion of the second information, respectively, through a first selectable control associated with the first widget and a second selectable control associated with the second widget, respectively, through the user interface. The method of the embodiment, in response to receiving the selection, then implements an operation within the user interface that provides, independently of the execution of a first heterogeneous software application associated with the first information and independently of the execution of a second heterogeneous software application associated with the second information, first or second additional information or additional functionality associated with the first information or the second information, respectively.Alternatively, the method of the embodiment, in response to receiving the selection, then triggers the presentation of a first complete instance of the interface for the first heterogeneous software application, or a second complete instance of the interface for the second heterogeneous software application, respectively, and triggering such presentation includes, automatically, without further user interaction, the first or second additional information or additional functionality associated with the first or second information, respectively, within the first or second complete instance or the second complete instance, respectively.

[0169] Example 53: The method according to Example 52, wherein the first widget is a first feature-limited set version of the first complete instance, and the first widget is a borderless window, active control, or thumbnail.

[0170] Example 54: The method of Example 52 or 53, wherein the first complete instance of the interface for the first heterogeneous software application is presented within the user interface.

[0171] Example 55: The method according to any one of Examples 52 to 54, comprising providing, in response to receiving the selection of the first selectable control or the second selectable control, the first or second additional information or the additional function associated with the first information or the second information, respectively.

[0172] Example 56: The method according to any one of Examples 52 to 55, wherein the presentation is to present at least the first portion of the first information in the first widget and at least the second portion of the second information in the second widget, and the presentation is within the same area of ​​the user interface.

[0173] Example 57: The method according to any one of Examples 52 to 56, wherein the first information is presented in a modified format, the modified format being different from the format in which the interface of the first heterogeneous software application is configured to present the first information.

[0174] Example 58: The method according to any one of Examples 52 to 57, wherein the first and second IPC mechanisms are different IPC mechanisms, and the first IPC mechanism is an application programming interface, remote procedure call, message queue, simple object access protocol, hypertext transfer protocol, or file-based communication.

[0175] Example 59: The method according to any one of Examples 52 to 58, comprising, in response to receiving a selection of the first selectable control or the second selectable control, the first or second IPC mechanism respectively causes the presentation of the first complete instance of the interface of the first heterogeneous software application or the second complete instance of the interface of the second heterogeneous software application, and the causing of the presentation automatically includes, without further user interaction, the addition of the first information or the addition of the second information within the first complete instance or the second complete instance, respectively.

[0176] Example 60: The method according to any one of Examples 52 to 59, wherein the first or second information is a notice associated with the first or second heterogeneous software application, respectively.

[0177] Example 61: The method according to Example 60, wherein at least the first portion of the first information is presented in the first widget, or at least the second portion of the second information is presented in the second widget, and is presented in the notification area of ​​the user interface.

[0178] Example 62: The method according to any one of Examples 52 to 61, wherein the presentation of the first complete instance of the interface of the first heterogeneous software application within the user interface includes passing the instruction for the selection to the first heterogeneous entity by the first IPC mechanism.

[0179] Example 63: The method according to any one of Examples 52 to 62, wherein the first heterogeneous software application and the second heterogeneous software application include similar functionality, and the similar functionality displayed has heterogeneous visual controls in the first complete instance of the interface of the first heterogeneous software application and in the second complete instance of the interface of the second heterogeneous software application, and the method enables controls having the same visual controls in the first widget and the second widget, and the same visual controls enable the similar functionality in either the first heterogeneous software application or the second heterogeneous software application or any of their associated widgets.

[0180] Example 64: The method according to Example 63, wherein the similar functions are a text or image deletion function, a zoom in or zoom out function, or a response function, the response function being associated with the first or second information, respectively.

[0181] Example 65: A computer-readable storage medium comprising instructions that cause the processor to implement one of the methods described in Examples 52 to 64 in response to execution by the processor.

[0182] Example 66: An electronic computing device comprising at least one processor and a computer-readable storage medium containing instructions that, in response to execution by the at least one processor, instruct the electronic computing device to implement any one of the methods described in Examples 52 to 64.

[0183] conclusion While the techniques and / or methods for user interfaces that integrate heterogeneous software applications through inter-process communication mechanisms and embodiments of apparatus that enable such integration are described in a language specific to the features and / or methods, it should be understood that the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary embodiments that enable user interfaces that integrate heterogeneous software applications through inter-process communication mechanisms.

Claims

1. It is a method, The method includes receiving first information by a first inter-process communication (IPC) mechanism, wherein the first information is from a first heterogeneous entity associated with the first information, and the method further includes: The method further includes receiving second information by a second IPC mechanism, wherein the second information is from a second heterogeneous entity associated with the second information, and the method further includes A user interface application presents, at least partially and simultaneously, within the user interface, at least a first portion of the first information in a first widget and at least a second portion of the second information in a second widget. Through the user interface and through a first selectable control associated with the first widget and a second selectable control associated with the second widget, it is possible to select at least the first portion of the first information and at least the second portion of the second information, or to make selections associated with those portions, respectively. In response to receiving the aforementioned selection, Within the user interface, independently of the execution of a first heterogeneous software application associated with the first information, and independently of the execution of a second heterogeneous software application associated with the second information, to provide first additional information, second additional information, or additional functions associated with the first information or the second information, respectively, or A method comprising: causing the presentation of a first complete instance of the interface for the first heterogeneous software application, or a second complete instance of the interface for the second heterogeneous software application, wherein causing the presentation automatically includes, without further user interaction, the first additional information, the second additional information, or additional functionality associated with the first or second information, respectively, within the first or second complete instance.

2. The method according to claim 1, wherein the first widget is a first feature-limited set version of the first complete instance, and the first widget is a borderless window, an active control, or a thumbnail.

3. The method according to claim 1 or 2, wherein presenting the first complete instance of the interface for the first heterogeneous software application is to present the first complete instance within the user interface.

4. The method of any of the prior claims, comprising providing, in response to receiving the selection of the first selectable control or the second selectable control, the first additional information, the second additional information, or the additional function associated with the first information or the second information, respectively.

5. The method according to any of the prior claims, wherein the presentation is to present at least the first portion of the first information in the first widget and at least the second portion of the second information in the second widget, and the presentation is within the same area of ​​the user interface.

6. The method according to any of the prior claims, wherein the presentation of at least the first portion of the first information within the first widget is in a modified format, the modified format being different from the format in which the interface of the first heterogeneous software application is configured to present the first information.

7. The method according to any one of the prior claims, wherein the first and second IPC mechanisms are different IPC mechanisms, and the first IPC mechanism is an application programming interface, a remote procedure call, a message queue, a simple object access protocol, a hypertext transfer protocol, or a file-based communication.

8. The method of any of the prior claims, comprising, in response to receiving a selection of the first selectable control or the second selectable control, the first or second IPC mechanism respectively causes the presentation of the first complete instance of the interface of the first heterogeneous software application or the second complete instance of the interface of the second heterogeneous software application, wherein causing the presentation automatically includes, without further user interaction, the addition of the first information or the addition of the second information within the first complete instance or the second complete instance, respectively.

9. The method according to any one of the prior claims, wherein the first or second piece of information is a notice associated with the first or second heterogeneous software application, respectively.

10. The method according to claim 9, wherein the presentation of at least the first portion of the first information within the first widget, or the presentation of at least the second portion of the second information within the second widget, is performed within the notification area of ​​the user interface.

11. The method according to any of the prior claims, wherein causing the presentation of the first complete instance of the interface of the first heterogeneous software application within the user interface includes passing the instruction for the selection to the first heterogeneous entity by the first IPC mechanism.

12. The method according to any of the prior claims, wherein the first heterogeneous software application and the second heterogeneous software application include similar functionality, and the similar functionality displayed has heterogeneous visual controls in the first complete instance of the interface of the first heterogeneous software application and in the second complete instance of the interface of the second heterogeneous software application, the method enables controls having the same visual controls in the first widget and the second widget, and the same visual controls enable the similar functionality in either the first heterogeneous software application or the second heterogeneous software application or any of their associated widgets.

13. The method according to claim 12, wherein the aforementioned similar functions are a function to delete text or image portions, a function to zoom in or zoom out, or a response function, the response function being associated with the first or second information, respectively.

14. A computer-readable storage medium that includes an instruction causing the processor to execute any one of the methods described in claims 1 to 13 in response to execution by the processor.

15. An electronic computing device comprising at least one processor and a computer-readable storage medium containing instructions that, in response to execution by the at least one processor, instruct the electronic computing device to perform any one of the methods according to claims 1 to 13.

Images