Computer system user interface with fast note management

Abstract

The present technology provides an integrated method of taking notes on a client computing device. The embedded app is integrated with an operating system level user interface of the client device. The method utilizes system UI level access to user data, thereby supporting functionality including providing context for notes. This may include marking a current context in the note, such as other information relating to what the user is doing while recording the note and / or how the note is triggered. When a user interacts with a particular app or other program, the context may provide suggestions associated with a task. The technique enables a user to quickly record notes within its current task flow via a system-level tool. The present technology provides a user with an opportunity to quickly write down thoughts and process it later, while providing easy access at the operating system level and also capturing contexts from interactions that motivate notes.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of the filing date of U.S. patent application No. 18 / 105,294, filed on February 3, 2023, the entire disclosure of which is expressly incorporated herein by reference. Background Art

[0002] It can be helpful for users to take electronic notes on their computers, mobile phones, or other client devices. This can be done using dedicated apps or as a comment-type feature within word processing programs, presentation programs, and the like. However, it can be challenging for users to manage the notes they take. Furthermore, notes taken within dedicated apps or as comments within programs may not provide useful context because they are isolated within their respective apps or programs. Summary of the Invention

[0003] The present technology relates to a holistic approach for electronic note-taking on a client device. Note-taking can be performed via an embedded app that integrates with the client device's operating system-level user interface (system UI). This user interface can support existing apps or other programs that may or may not have note-taking features. Through this integration, the operating system is able to bridge access to user data at the system UI level and support functionality, including displaying and supporting context for notes. This can be done across multiple apps or other programs, depending on user interaction with the client device and contextual information.

[0004] According to one aspect, a computer-implemented method includes: receiving, by one or more processors of a client device, an indication for creating a note from a user of the client device; receiving, by the one or more processors, user-generated note information via a system-level user interface module; identifying, by the one or more processors, via the system-level user interface module, one or more actions or events that occur on the client device when the indication for creating the note is received; generating, by the one or more processors, via the system-level user interface module, one or more context suggestions for the note, the one context suggestion being based on the one or more actions or events; creating, by the one or more processors, via the system-level user interface module, the note comprising the user-generated note information and at least a given context suggestion of the one or more context suggestions; and storing the note in a memory associated with the client device.

[0005] In one example, the method further includes the system-level user interface module presenting a visualization with the generated one or more contextual suggestions to the user. Alternatively or additionally, the instruction to create the note is received based on a selection of an icon presented in a user interface displayed on a display screen of the client device. Alternatively or additionally, generating the one or more contextual suggestions for the note includes generating a link to information about a meeting or event that was occurring when the instruction to create the note was received.

[0006] Alternatively or in addition to the above example, generating the one or more contextual suggestions for the note may include generating contextual information about an action taken by the user within a predetermined amount of time before receiving the instruction to create the note. The action taken may include triggering the note when playing media content.

[0007] Alternatively or in addition to the above examples, the one or more actions or events may include determining that the user's focus has shifted from the application or webpage to the note.Alternatively or in addition to the above examples, the method may further include presenting a set of note-taking options to the user.

[0008] Alternatively or in addition to the above example, the method may further include: upon receiving the indication, the one or more processors collecting information about active applications or windows on the client device via the system-level user interface module; wherein generating the one or more contextual suggestions is further based on the information about active applications or windows on the client device. Here, the method may further include creating a bookmark about the state of the open application or window. Alternatively or in addition to the above example, storing the note in a memory may include storing the note in a memory corresponding to the system-level user interface module.

[0009] According to another aspect, a computer-implemented method includes: receiving, by one or more processors of a client device, an indication to access a note, the note comprising user-generated note information and at least one contextual suggestion, wherein the at least one contextual suggestion is based on one or more actions or events that occurred on the client device before creation of the note; retrieving, by the one or more processors, the note from a memory via a system-level user interface module; identifying, by the one or more processors, whether to present the note via a note-taking application integrated as part of a system-level user interface of the client device or via another application executable by the client device; and generating, by the one or more processors, via the system-level user interface module, a graphical representation of the note comprising the user-generated note information and the at least one contextual suggestion, the graphical representation being configured to be presented to a user of the client device.

[0010] In one example, upon identifying to present the note via the other application, the method includes sending note information including information about the at least one contextual suggestion to the other application. In another example, upon identifying to present the note via the other application, the method includes transmitting the note information including information about the at least one contextual suggestion to another client device associated with the user of the client device.

[0011] As an alternative to or in addition to the above examples, the method may further include: receiving input for selecting the at least one context suggestion from the user of the client device when generating the graphical representation; accessing, by the one or more processors, information corresponding to the selection of the at least one context suggestion; and presenting the information corresponding to the selection of the at least one context suggestion to the user.

[0012] According to another aspect, a computing device is provided, the computing device comprising: a memory configured to store system user interface information; and one or more processors operatively coupled to the memory. The one or more processors are configured to: receive an instruction to create a note from a user of the computing device; obtain, via a system-level user interface module of the computing device, user-generated note information; identify, via the system-level user interface module, one or more actions or events that occurred on the computing device when the instruction to create the note was received; generate, via the system-level user interface module, one or more contextual suggestions for the note, the one contextual suggestion being based on the one or more actions or events; create, via the system-level user interface module, the note comprising the user-generated note information and at least a given contextual suggestion of the one or more contextual suggestions; and store the note in the memory.

[0013] In one example, the one or more processors are further configured to present a visualization with the generated one or more contextual suggestions to the user via the system-level user interface module, or present a set of note-taking options to the user. Alternatively or additionally, generating the one or more contextual suggestions for the note includes at least one of: generating a link to information about a meeting or event that was occurring when the instruction to create the note was received; or generating contextual information about actions taken by the user within a predetermined amount of time before receiving the instruction to create the note.

[0014] According to a further aspect, a computing device is provided, comprising: a memory configured to store system user interface information; and one or more processors operatively coupled to the memory. The one or more processors are configured to: receive an indication to access a note, the note comprising user-generated note information and at least one contextual suggestion, wherein the at least one contextual suggestion is based on one or more actions or events that occurred on the computing device prior to creation of the note; retrieve the note from the memory via a system-level user interface module of the computing device; identify whether to present the note via a note-taking application integrated as part of the system-level user interface of the computing device or via another application executable by the computing device; and generate, via the system-level user interface module, a graphical representation of the note comprising the user-generated note information and the at least one contextual suggestion, the graphical representation configured to be presented to a user of the client device. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1A-Figure 1C Shown are examples of user interfaces in accordance with aspects of the present technology.

[0016] Figure 2 An example client device is shown for use in accordance with aspects of the present technology.

[0017] Figure 3A-3C User interface features in accordance with aspects of the present technology are shown.

[0018] Figures 4A-4D Example windows for different note-taking options in accordance with aspects of the present technology are shown.

[0019] Figure 5A-5B An example of note storage according to aspects of the present technology is shown.

[0020] Figures 6A-6D An example of contextual suggestions in accordance with aspects of the present technology is shown.

[0021] Figure 7 Sharing examples according to aspects of the present technology are shown.

[0022] Figure 8A-8B A system for use with various aspects of the present technology is shown.

[0023] Figure 9A-9B Example methods according to aspects of the present technology are shown. DETAILED DESCRIPTION

[0024] The electronic note-taking technology described herein utilizes integration between a note-taking app and system-level surfaces (e.g., the login screen, shelf, or system tray, etc.) and data accompanying the operating system-level user interface (system UI) to provide context to the user and use this context to enrich the user experience. This includes tagging the current context in the note, such as to indicate what the user was doing when the note was taken and / or other information related to how the note was triggered. When the user is interacting with a particular app or other program, context can be used to provide suggestions associated with the primary task. This can include referencing recent apps or documents, providing links to content via a browser, and so on. System-level integration also avoids the problem of isolated notes tied to individual applications and allows context or other information to be shared across apps or even across client devices.

[0025] Thus, the solution described herein enables users to quickly take short notes in their current task flow via a system-level tool. This approach is designed for note-taking as a secondary task and need not replace primary note-taking. Instead, the present technology provides users with the opportunity to quickly jot down an idea and process it later, while providing easy access at the operating system level and also capturing the context from the interaction that inspired the note. In addition to the beneficial enhancements to users, the technical benefits to computing systems relate to efficient management of note-related content that can be shared across various types of apps on client devices, or synchronized across different devices associated with a user (e.g., laptop, tablet, and smartwatch), such as via cloud-based synchronization methods. Example System

[0026] Figure 1A An example 100 of a user interface for a client device such as a laptop or tablet computer is shown, although the present technology can be used with other types of client devices. As shown, the displayable area of ​​the user interface includes a first section 102, which can be associated with an app or other program, and a second section 104. In this example, as shown in the first section 102, the program can be a word processor that provides options for formatting text, adding or editing objects, performing page formatting or document editing, etc.

[0027] The second section 104 of the user interface may be a shelf or system tray, which in this example is not part of or associated with the program. Here, as shown, there are a number of icons 106 in the shelf or tray, as well as subsections 108 indicating the status of various parts of the client device (e.g., WiFi signal strength and battery level) and the local time. In this example, one of the icons 106 is NA note taking app for integration with or otherwise association with the user interface of the system. The user can access the note taking app via the icon 106 N According to one aspect, icon 106 N Rather than app icons, pod icons are associated with system-level functionality. While app icons are entry points to specific apps running on a client device, pod icons can be thought of as shortcuts to secondary tasks or tools associated with the system itself.

[0028] Although shown in the shelf or tray, the note taking icon or other indicia may be presented elsewhere on the display of the client device. For example, the icon may be incorporated into the app in the first section 102. An example 120 of this is shown in FIG. Figure 1B , where the embedded icon 122 is located in the first section 102, where it is associated with the word processor app of this example. Figure 1C In another example 140 shown in FIG, note taking can be initiated from the lock screen. Here, in addition to a login section 142 and other icons such as for a status indicator (144), shutting down or restarting the client device (146), and / or logging out (148), a note taking icon or other indicia 150 can be presented.

[0029] Because the note-taking feature can be integrated with or otherwise associated with the system UI, user interface data can be maintained or otherwise managed by the system UI module. This enables latency optimization for efficient load times for the note-taking feature. It also enables embedding the feature in one or more specific locations in the system UI to provide the fastest possible access. Regarding the technical benefit of load time optimization, the system is able to load the note-taking app in the background so that by the time the user signals their intent to take a note, the note-taking app is already in a ready state. This allows the system to display the app embedded in the system UI with very low latency once the user has signaled their intent.

[0030] Figure 2, an example client device arrangement 200 is shown in FIG. As shown, the client device includes a processing module 202 having one or more computer processors such as a central processing unit 204 and / or a graphics processor 206 and a memory module 208 configured to store instructions 210 and data 212. The processors may or may not operate in parallel and may include ASICs, controllers, and other types of hardware circuitry. The processor is configured to receive information from a user via a system user interface module 214 and present the information to the user on a display device of a display module 216 via the user interface module. For example, during different modes at different times or under different circumstances, the user interface module may cause the display device to present a screen saver, a login screen, and apps and other features. The display module 216 has a display interface and may be configured as a touch screen that implements user input via a stylus or other tool or by the user physically touching the screen.

[0031] System user interface module 214 is configured to receive user input and store user interface data 215.User interface module 214 can receive commands from the user via user input piece, and converts these commands to submit to given processor.User interface module can be linked to web browser (not shown).As the supplement or replacement to keyboard, keypad, mouse pad and / or touchpad, microphone, input based on gesture or other type of input device, user input can comprise touch screen as stated above.Keyboard, keypad, mouse pad and / or touchpad can be the part of computing device, or can be connected to base via cable or other wired connection, or can be physically separated from the integrated client device, and be configured to be connected via one or more wireless connections such as Bluetooth ™ , WiFi, ultra-wideband (UWB), infrared etc.User interface module 214 is operatively connected to display module 216.

[0032] User interface data may include information associated with the note-taking app. For example, such information may indicate not only where and how to present a sign to launch / access the note-taking app, but also which programs (e.g., a word processor, presentation program, spreadsheet program, web browser, messaging app, etc.) the app is associated with. User interface data may also include contextual information for notes taken by a user via the note-taking app.

[0033] The display module 216 may include a circuit system for driving a display device to present graphical information and other information to the user. In other words, the display device is configured to present visual content. An actuator for providing tactile feedback or other information to the user may be incorporated into a touch screen of the display module (not shown). Graphics information may be generated by the graphics processor 206, while the CPU 204 manages the overall operation of the integrated client device 200. As an example only, graphical information may be displayed on the display module 216 in response to a user query. For example, the processing module may use the instructions and data stored in the memory module 208 to run a browser application or other service, and present information associated with the browser application or other service to the user via the display module 216. The memory module may include a database or other storage device for browser information, location information, etc. In one scenario, the user interface data 215 may be part of the data 212 stored in the memory module 208.

[0034] The memory module 208 may be implemented as one or more of one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. The memory module 208 may include, for example, flash memory and / or NVRAM, and may be embodied as a hard drive or a memory card. Alternatively, the memory module 208 may also include removable media (e.g., a DVD, CD-ROM, or USB thumb drive). One or more areas of the memory module 208 may be writable, while other areas may include read-only (or otherwise write-protected) memory. In one implementation, the computer program product is tangibly embodied in an information carrier. Although Figure 2 The processor, memory module, and other elements of the integrated client device are shown functionally as being within the same overall block, but such components may or may not be stored within the same physical housing. For example, some or all of the instructions and data may be stored on an information carrier that is a removable storage medium (e.g., an optical drive, a high-density tape drive, or a USB drive) that is connectable to the base or display housing, and other instructions and data may be stored within a read-only computer chip integrated into the base or display housing.

[0035] Data 212 and / or 215 may be retrieved, stored, or modified by the processor according to instructions 210. For example, the data may be stored in a computing device register, in a relational database as a table with multiple different fields and records, in an XML document, or in a flat file. The data may also be formatted in any computing device readable format.

[0036] Instructions 210 may be any set of instructions to be executed directly by a processor (such as machine code) or any set of instructions to be executed indirectly (such as a script). For example, the instructions may be stored as computing device code on a computing device readable medium. In this regard, the terms "instructions" and "program" may be used interchangeably herein. The instructions may be stored in an object code format for direct processing by a processor, or in any other computing device language, including scripts or independent source code modules that are interpreted on demand or compiled in advance.

[0037] like Figure 2 As shown, the client device 200 includes a communication module 218 for communicating with other devices and systems (including other client devices, servers, and databases). The communication module 218 includes a wireless transceiver; optionally, the module may alternatively or additionally include a wired transceiver. The client device 200 can communicate with other remote devices via the communication module 218 using a variety of configurations and protocols, including short-range communication protocols such as near field communication (NFC), Bluetooth™, Bluetooth™ Low Energy (BLE), UWB or other ad hoc networks, the Internet, an intranet, a virtual private network, a wide area network, a local network, a private network using one or more company-proprietary communication protocols, Ethernet, WiFi, and HTTP, and combinations thereof.

[0038] In addition, the example device shown includes one or more positioning and orientation sensors 220. Positioning and orientation sensors 220 are configured to determine the position and orientation of one or more parts of the computing device 200 (such as a display housing) relative to the base. For example, these components may include a GPS receiver for estimating the latitude, longitude, and / or altitude of the integrated client device, and an accelerometer, gyroscope, or another direction / speed detection device such as an inertial measurement unit (IMU) that can determine the orientation of the display housing relative to the base of the device (and the rate of change of the positioning of the client device). The client device 200 may also include one or more cameras 222 (such as an integrated webcam) for capturing still images and recording video streams. The device may also include one or more microphones 224, a speaker 226, and a power module 228. Example Operation

[0039] From a user's perspective, note-taking via the apps described herein can be a seamless experience. According to one aspect of the present technology, such notes can be referred to as "Quick Notes" for ease of creation and use. Interacting with such notes can function identically regardless of whether the Quick Note is generated from the lock screen, the system tray, or within a program (which can have its own dedicated note-taking feature). Thus, a consistent user interface can be presented even when using different types of apps from different companies (e.g., a word processing app, a presentation app, a web browser, and a video conferencing app).

[0040] In addition to different ways to launch Quick Notes, there are also different ways to capture such notes, and a variety of use cases. For example, if a user wants to capture information immediately, he or she can invoke the Quick Notes app from the lock screen, system tray, or other shortcut. A user may want to take notes while interacting with a specific program, such as while browsing a website, watching a video, or marking action items during a meeting (e.g., during a video conference). Or a user may want to take notes as part of their regular workflow using familiar tools. This can include using a stylus when in tablet mode, using a keyboard when in desktop mode, or even using gestures during hands-free operation.

[0041] Figure 3A The diagram shows a diagram where a user can, for example, use an arrow icon 302 (eg, using a mouse, stylus, or gesture) to hover over icon 106. N An example 300 on the left. Here, hovering can be emphasized by a shadow 304. The result of hovering can be a pop-up 306 including information about the quick note feature. N The selection of will start the quick note feature. When selecting, different options may be presented to the user. For example, Figure 3B As shown in example 310 of FIG, an options bar or other feature 312 may be presented on the display to show the user various note taking options, such as typing into the field (3141), using the checkbox (3142), freehand writing (3143), inserting a link (3144), etc. Figure 3B As shown, icon 106 N is shaded to indicate selection. Figure 3CA different interface 320 is shown including different options, including a selection to always float the quick note options bar on top of apps (322), a list of favorite apps that can be opened when a note associated with the selected app is also created (324), and the type of note to be created, such as typed (3261), using check boxes (3262), freehand (3263), with a link (3264), or as an audio note (3265).

[0042] Figure 4A An example 400 is shown showing a typing window 402 for a user to enter notes. Figure 4B Another example 410 is shown showing a checkbox UI 412. As shown, the user can enter different options, and the system can also provide default options. Once one or more notes have been selected (e.g., with a check mark), the user can save the quick note and / or send it to a favorite using a specific app. Figure 4C Yet another example 420 for freehand note taking is shown, where a freehand UI 422 includes options for clearing, saving, and sending notes, as well as different pen sizes and colors. The windows in these examples can be presented as part of the system UI and are not associated with a specific app that is open. These windows can instead be presented as an overlay to or within the UI of the app that the user is currently working in (e.g., a word processor, video call, or web browser).

[0043] Figure 4DAn example 430 is shown in which a user can select content 432 within a UI (e.g., information that can be presented in a page of a web browser), such as text, a hyperlink, an icon, an image, or a video, and add the selection to a note. Here, a button 434 (or a pop-up or other selector) may be presented to the user. If there are multiple notes, the interface may list all or a subset of the notes to which the selection can be copied. Once content is selected, the presentation of this selection in the note may be formatted by the system UI according to the type of note. For example, for a text entry note or a freehand note, the content may be presented to appear in the same manner as the original (e.g., as a block of text that reads "See here for more info about how to implement this exciting operating system with your device"). However, for a checkbox-type note, the size of the content may be fixed or otherwise truncated to fit next to the checkbox (e.g., "See here for more info ... (See here for more information ...)").

[0044] According to one aspect, the information for the quick note can be saved in different ways. This can include local storage, for example, as a component of a system UI (e.g., Figure 2 User interface data associated with the system UI module 214). Figure 5A An example of this is shown in FIG, which shows a set of notes (e.g., notes 1-6) stored as user interface data, such as part of a system UI. Alternatively or additionally, the notes can be saved to one or more selected apps on the client device. This may be appropriate when these apps have their own corresponding note-taking features, which may be isolated. Figure 5B Such an example is shown in , where program data for a particular app includes a set of app notes (e.g., app notes 1-3) and a separate set of quick notes (e.g., notes AC).

[0045] Quick notes can be stored using the same folder structure or other format used to store app-specific notes. There can be a flag or other identifier associated with each note to identify whether it is associated with a specific app, or whether it is part of the system UI note set. Notes can also be saved remotely, for example, in a cloud storage device or a backend service that supports one or more process web apps. This may involve automatically syncing notes (for example, every 10 minutes or some other time frame). When saved in the cloud or as part of the system UI, quick notes can be easily accessed across different client devices.

[0046] In one scenario, for a given app, all app notes and quick notes may be accessible via that given app. However, app notes may or may not be accessible from the system UI. And if a quick note is opened from the system UI (e.g., from the system tray, shelf, or other access point that is not within a given app), the ability to edit that quick note from within the given app may be automatically disabled. This prevents inadvertent creation of multiple iterations of the same quick note. Scene

[0047] The system provides the ability to tag the user's current context within the operating system. This provides the user with the option to add a contextual tag (or other marker) that references what they were doing when they took a note (or immediately before taking a note). Suggestions can come from the user's primary tasks at that moment, such as the most recent apps used, documents opened, or links visited. Because this can be done at the operating system (system UI) level with the note-taking app, rather than within a specific app, this can be a powerful tool that enables the system to show the user context regardless of what program they are currently using.

[0048] There are several interesting aspects of context for notes, including the information entered by the user when they create a note, and the suggestions shown to the user when the user retrieves or is otherwise presented with the stored note. This can include viewing a document (e.g., a line item in a spreadsheet) or on a website (e.g., Figure 4D The contextual information may depend on how the note was triggered (created or retrieved), the app the person was using at the time (if any), and other factors. Context may be used to provide suggestions for information to consider (e.g., a web page or video to view), as well as to remind the user of actions or events related to the note.

[0049] The following are different examples of how the system can prompt context when a user creates a note. For example, if a note is triggered during a calendared meeting (e.g., the user joins a video call or other meeting based on a pre-existing calendar event), the system can automatically provide the user with a contextual suggestion to attach a link (or a link as a tag) to the note. This link can be to a recording of the meeting, notes about the meeting, or other information about the meeting. In another example, if a note is triggered during a meeting and the user's screen or a specific item from the user is being presented in the meeting, the system can provide a contextual suggestion to attach a link to the presentation, a screenshot of the presentation, or the content of the item being presented.

[0050] In another example, temporal interactions can provide contextual signals. Thus, if a note is triggered within a certain amount of time (e.g., within 5-30 seconds) after a screenshot (or other action) is taken, the system can automatically suggest adding the screenshot (or information about the other action) to the note. Alternatively, if a screenshot (or other action) is taken while a note is open, the system can automatically suggest adding the screenshot and / or a link to the screenshot's location to the note. Other examples of actions include browsing to a specific webpage, opening a document, looking up contact information (e.g., an email address or phone number), etc. The system UI can store copies of screenshots in the user interface data so that if the original screenshot is deleted or not saved by the user, the information remains. Thus, if a screenshot or other action is associated with a given note, when the note is later opened, information about the associated action is presented to the user. This can include presenting the screenshot (or document or contact information, etc.) or providing a link to access it. Alternatively, as the user browses through one or more screenshots, if any of the screenshots are associated with a note, the system UI can indicate the existence of a corresponding note.

[0051] In a further example, if a note is triggered and the system determines that the user's focus shifts from a website they were viewing (e.g., by going to a different browser tab or switching to a different app), the system may automatically offer to add a link to the website (or a link as a smart tag) to the note. In yet another example, if a note is open and the user copies something (e.g., selects text, an image, a video link, or a URL), the system may automatically offer to add a copy of the item and / or a link to the copied location to the note. As discussed above, in Figure 4D Such an example is shown in .

[0052] In another example, if a note is triggered while a video (or audio) is playing, the system can automatically suggest adding a link to the video (or audio) at a specific timestamp. Here, if a note has already been triggered or opened, and the user then pauses playback of the content, the system can suggest adding a link at the pause timestamp. And in a further example supplementing any of these scenarios, the system can suggest adding a date to the note. Alternatively or additionally, the user can be prompted to add an emoji, gif, sticker, or other logo to the note.

[0053] In one scenario, contextual reminders can use @mentions of individuals, dates, calendar events, places, directions, links to another app, or links to specific files (within an app) via tags. Thus, if a user is in a conference call, the system can display a reminder to add the names or email addresses of one or more participants on the call to the notes. Or, a date tag, such as "@tomorrow" or "@Thursday," can be displayed.

[0054] Figure 6A An example 600 is shown in which a note 602 is already open. The user has entered information 604 into the note, here "Soccer alternative?" Assume that at the time note 602 was opened, the user was listening to a podcast or watching a video. In this example, the system can automatically populate contextual suggestions 606, or the user can copy a link to this content into note 602. This content (here "Futsal: the game and how it is played") can be added with a timestamp associated with when playback was paused, when the note was started, or when a link to the content was added to the note.

[0055] Figure 6B Another example 610 is shown in which a note 612 has been opened. Here, the user has entered several pieces of information into the note. This includes a reminder 612 to "Refine the presentation slide deck" and a comment about a meeting, "Set up meeting with development team" 614. In this example, the system can automatically create a context suggestion 616 for a meeting invite, or the user can create an invite and paste a copy of the link to the meeting into the note 612. In this example, an @mention option 618 can be included in the note UI. As examples, upon selecting @mention, the user can be prompted by the UI to include members from the development team, to create a calendar invite, and / or to set up a video link.

[0056] Figure 6C An example 620 is shown in which the system UI suggests a context to the user via a note taking app. Here, the user has entered a note, such as Figure 6B . In this case, the note-taking UI may present a pop-up or other visualization 622 for the suggested context. This visualization may be populated with different suggestions based on the content of the note (e.g., using keyword evaluation or other contextual analysis of the content) or by what the user was doing when taking the note. As shown, this may include presenting options for quick sync and / or meeting setup.

[0057] Figure 6D Another example 630 is shown where the system UI suggests context options to the user via the note taking app. Here, the user has entered a note, such as Figure 6B In this case, the note taking UI may present a pop-up or other visualization 632 for the suggested context. Similar to Figure 6C , this visualization can present different suggestions based on the content of the note (e.g., using keyword evaluation or other contextual analysis of the content). As shown, this can include presenting contextual options to remind the user to set up a video conference, a link to the current slide deck, or to check their calendar to set up a meeting time to go through the slide deck.

[0058] These examples are not limiting, and users are always free to ignore, reject, or otherwise dismiss contextual suggestions. A given suggestion may also be time-constrained. For example, if a user does nothing with a suggestion and ignores it for at least a threshold amount of time (e.g., 5-10 seconds or more or less), the suggestion may be dismissed.

[0059] According to another aspect, the context for a note can involve an action the user might want to take. For example, if a user types into a note that begins with a verb or keyword, which can be part of a default word list generated by the system or identified by the user, the system can surface an action item to add to a task list or calendar event. Or if the note indicates the need to schedule a meeting, the system can surface an action item to create a calendar event. And if the note indicates the need to send an email, the system can surface an action item to start the email (e.g., a suggestion to launch an email app). These types of actions are non-limiting.

[0060] Even when not prompted by the system, the user may decide to create a context and add it to a note.

[0061] Once a note is created, the user can retrieve the note by launching the note-taking app, whether from the system UI or within a specific app. For example, the system UI may generate a graphical representation of the note that includes any user-generated note information and any contextual suggestions associated with the note. The graphical representation may be presented on the client device on which the graphical representation was created, or on another client device that has received the note. When input is received from the user for selecting a contextual suggestion, information corresponding to the selection is accessed by the system, and information corresponding to the contextual suggestion is presented to the user. Thus, when the contextual suggestion corresponds to a video or music, the graphical representation may provide the user with the option of resuming playback of the video or music or otherwise launching this content (e.g., see Figure 6A ). When the contextual suggestion corresponds to a meeting (see, for example, Figure 6B-6D ), then a link to a calendar invite, a meeting link, etc. can be provided. When the context suggestion corresponds to a presentation or other material (e.g., see Figure 6D ), you can provide a link to a slide deck or other materials for the presentation.

[0062] When a note is open, the note-taking app allows the user to select, copy, paste, and / or delete some or all of the content in the note that includes the context information. If the context information provides a link to the content (e.g., a text file, music, or video), selecting the context tag or other indicia will bring the UI to the content (or otherwise open the content in the UI).

[0063] The user can create a new note from scratch or from within another note. If this is done from an existing note, the system can automatically save the existing note and start a new note. The interface can enable the user to name or rename the note, choose where to save the note, or move the note from the system UI to a specific app, or vice versa. If the client device is currently offline, the note is saved locally and can be uploaded to the backend location of the user's choice when it comes back online.

[0064] Based on a triggering item or event associated with the context information in the note, the note may be resurfaced, eg, opened by the user or by the system.

[0065] Sharing notes across apps on a client device can be accomplished as follows. When a note associated with the system UI is opened, the system is configured to suggest context and present the context to the user. For example, when a user clicks to open a quick note, the system collects information about which specific apps or windows are open on the client device. Bookmarks can be created for those items, which have a reference by which the context can be restored at a later time. The note information, including any relevant context information associated with the note, can then be projected (sent) to a specific app or destination, such as a spreadsheet, calendar, or video conferencing app. If the user accesses a web-based app (e.g., a progressive web app), the note information can be shared via a "service worker", which is a virtual proxy between the client device side (e.g., a browser app) and the network.

[0066] Figure 7 A sharing example 700 is shown. As shown, the note-taking UI may include an icon or other indicator 702 for sharing a note. When selected, a pop-up or other visualization 704 is presented, identifying different options for apps to which the note can be shared. This can include local apps (e.g., a word processing app or a presentation app) as well as remote apps such as a progressive web calendar app, where different users may have access to different calendars. Selection of one or more of these options causes some or all of the content of the note (including contextual information) to be shared with those programs. Thus, because the operating system has access to the browser, locally running programs, progressive web apps, and everything on the screen, note information managed via the system UI can be selectively shared with different apps as appropriate, including relevant contextual information that may be appropriate for a particular app.

[0067] A user may have multiple applications / windows on screen at the same time. In this case, they may want to associate all of them with a given note, none of them with a given note, or a subset of them with a given note. The system can enable users to disambiguate which context(s) they are interested in, in other words, curate which (if any) contexts should be associated with a note.

[0068] In addition to the above, controls may be provided to the user, allowing the user to choose whether and when the systems, programs, or features described herein may implement the collection of user information, including contextual information that may be associated with a quick note. In addition, certain data may be processed in one or more ways before being stored or used, such that personally identifiable information is removed. For example, the user's identity may be processed so that personally identifiable information about the user cannot be determined, or the user's geographic location may be generalized (such as to a city, zip code, or state level) when location information is available so that the user's specific location cannot be determined. Thus, the user can control what information is collected about the user, how that information is used, and what information is provided to the user. Example Network

[0069] Although the note-taking app focused on the system UI resides on one user device, as discussed above, notes including contextual information can be shared among apps that can reside locally on that user device as well as apps maintained remotely. This can include progressive web apps that can be in the cloud or other remote services. It can also include apps stored on other client devices of a particular user (e.g., laptop, tablet, mobile phone, smartwatch, etc.).

[0070] Figure 8A and Figure 8B 806 . The following are intuitive and functional diagrams of an example system 800 that includes multiple computing devices and databases connected via a network. For example, computing device 802 can be a cloud-based server system that provides or otherwise supports one or more cloud-based apps, games, or other programs. Database 804 can store app information, game data, user profile information, and / or other information. In one example, database 804 can store note-based information (including context information) from a system UI-based note-taking app associated with a user's client device. The server system can access the database via network 806. The client device can include one or more of a desktop computer 808 or a laptop or tablet PC 810. Other client devices can include handheld devices, including personal communication devices, such as mobile phones or PDAs 812 or tablet computers 814. Another example is a shared device 816, which can be a smart display or large monitor that can be in a conference room, classroom, or other collaborative environment.

[0071] In one example, computing device 802 may include one or more server computing devices having multiple computing devices, such as a load balancing server farm or a cloud computing system, that exchange information with different nodes of a network to receive data from, process, and transmit data to other computing devices. For example, computing device 802 may include one or more server computing devices capable of communicating with any of computing devices 808-816 via network 806. This may be done as part of hosting one or more collaborative apps (e.g., a video conferencing program, an interactive spreadsheet app, or a multiplayer game) or services (e.g., a movie or podcast streaming service).

[0072] like Figure 8B As shown, each of computing devices 802 and 808-816 may include one or more processors, memory, data, and instructions. The memory stores information accessible by one or more processors, including instructions and data that can be executed or otherwise used by the processors. The memory can be of any type capable of storing information accessible by the processor, including computing device-readable media. The memory is a non-transitory medium such as a hard drive, a memory card, an optical disc, a solid-state drive, etc. The system may include different combinations of the aforementioned, thereby storing different portions of the instructions and data on different types of media. The instructions can be any set of instructions to be executed directly by the processor (such as machine code) or any set of instructions to be executed indirectly (such as scripts). For example, the instructions can be stored as computing device code on a computing device-readable medium. In this regard, the terms "instructions," "module," and "program" can be used interchangeably herein. The instructions can be stored in object code format for direct processing by the processor, or in any other computing device language, including scripts or independent source code modules that are interpreted on demand or compiled in advance.

[0073] The processor may be any conventional processor, such as a commercially available CPU. Alternatively, each processor may be a dedicated device, such as an ASIC, a graphics processing unit (GPU), a tensor processing unit (TPU), or other hardware-based processor. Figure 8B While the processor, memory, and other elements of a given computing device are functionally shown as being located within the same box, such a device may actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical housing. Similarly, the memory may be a hard drive or other storage medium located in a separate housing from the processor (e.g., located in a cloud computing system such as server 802). Thus, references to a processor or memory will be understood to include references to a collection of processors or memories that may or may not operate in parallel.

[0074] The computing device may include all components typically used in connection with a computing device, such as the processor and memory described above, and a user interface subsystem for receiving input from a user and presenting information to the user (e.g., text, images, and / or other graphical elements, audibly and / or via tactile feedback). The user interface subsystem may include one or more user inputs (e.g., at least one front-facing (user) camera, a mouse, a keyboard, a touch screen, and / or a microphone) and one or more display devices operable to display information (e.g., text, images, and / or other graphical elements). Other output devices, such as speakers, may also provide information to the user.

[0075] The user-related computing devices (e.g., 808-816) can communicate with the back-end computing system (e.g., server 802) and / or each other via one or more networks such as network 806. The network 806 and the intermediate nodes may include various configurations and protocols, including short-range communication protocols such as Bluetooth™, Bluetooth LE™, the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local network, a private network using one or more company-specific communication protocols, Ethernet, WiFi, and HTTP, and various combinations of the foregoing. Such communications can be facilitated by any device (such as a modem and a wireless interface) capable of transmitting data to and from other computing devices. In some implementations, the user-related computing devices may also communicate with each other without also communicating with the back-end computing system. This can be accomplished via the network 806 or using one or more ad hoc and / or peer-to-peer communication links directly between the devices.

[0076] Figure 9A An example method 900 is shown in accordance with various aspects of the present technology. At box 902, the method includes receiving, by one or more processors of a client device, an indication to create a note from a user of the client device. At box 904, the method includes receiving, by the one or more processors, via a system-level user interface module, user-generated note information. Then, at box 906, the method includes identifying, by the one or more processors, via the system-level user interface module, one or more actions or events that occurred on the client device when the indication to create the note was received. At box 908, the method includes generating, by the one or more processors, via the system-level user interface module, one or more contextual suggestions for the note, a contextual suggestion based on the one or more actions or events. Then, at box 910, the method includes creating, by the one or more processors, via the system-level user interface module, a note that includes the user-generated note information and at least a given contextual suggestion of the one or more contextual suggestions. And at box 912, the note is stored in a memory associated with the client device.

[0077] Figure 9B Another example method 920 according to aspects of the present technology is shown. At box 922, the method includes receiving, by one or more processors of a client device, an indication to access a note. The note includes user-generated note information and at least one contextual suggestion. The at least one contextual suggestion is based on one or more actions or events that occurred on the client device before the note was created. At box 924, the method includes retrieving, by the one or more processors, the note from a memory via a system-level user interface module. At box 926, the method includes identifying, by the one or more processors, whether to present the note via a note-taking application that is integrated as part of a system-level user interface of the client device or via another application executable by the client device. And at box 928, the method includes generating, by the one or more processors, via the system-level user interface module, a graphical representation of the note that includes the user-generated note information and the at least one contextual suggestion. The graphical representation is configured for presentation to a user of the client device.

[0078] Although the technology herein has been described with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the technology. Therefore, it should be understood that various modifications may be made to the illustrative embodiments, and that other arrangements may be devised without departing from the spirit and scope of the technology as defined by the appended claims.

Claims

1. A computer-implemented method comprising: receiving, by one or more processors of a client device from a user of the client device, an instruction to create a note; receiving, by the one or more processors via a system-level user interface module, user-generated note information; identifying, by the one or more processors via the system-level user interface module, one or more actions or events that occur on the client device when the indication to create the note is received; generating, by the one or more processors via the system-level user interface module, one or more contextual suggestions for the note, the one or more contextual suggestions being based on the one or more actions or events; creating, by the one or more processors via the system-level user interface module, the note comprising the user-generated note information and at least a given contextual suggestion of the one or more contextual suggestions; as well as The note is stored in memory associated with the client device. 2 . The method of claim 1 , further comprising the system-level user interface module presenting a visualization with the generated one or more contextual suggestions to the user. 3 . The method of claim 1 , wherein the stored notes are associated with a note-taking application integrated as part of a system-level user interface of the client device.

4. The method of claim 1, wherein the indication to create the note is received based on a selection of an icon presented in a user interface displayed on a display screen of the client device. 5 . The method of claim 1 , wherein generating the one or more contextual suggestions for the note comprises generating a link to information about a meeting or event that was occurring when the indication to create the note was received.

6. The method of claim 1, wherein generating the one or more contextual suggestions for the note comprises generating contextual information about actions taken by the user within a predetermined amount of time before receiving the indication to create the note. The method of claim 6 , wherein the action taken comprises triggering the note while media content is playing.

8. The method of claim 1, wherein the one or more actions or events include determining that the user's focus has shifted from an application or web page to the note.

9. The method of claim 1, further comprising presenting a set of note-taking options to the user.

10. The method of claim 1, further comprising: upon receiving the indication, collecting, by the one or more processors via the system-level user interface module, information about active applications or windows on the client device; Wherein generating the one or more contextual suggestions is further based on the information regarding active applications or windows on the client device. The method of claim 10 , further comprising creating a bookmark regarding the status of the open application or window.

12. The method of claim 1, wherein storing the note in a memory comprises storing in a memory corresponding to the system-level user interface module.

13. A computer-implemented method comprising: receiving, by one or more processors of a client device, an indication to access a note, the note comprising user-generated note information and at least one contextual suggestion, wherein the at least one contextual suggestion is based on one or more actions or events occurring on the client device prior to creation of the note; retrieving, by the one or more processors via a system-level user interface module, the note from a memory; identifying, by the one or more processors, whether to present the note via a note-taking application integrated as part of a system-level user interface of the client device or via another application executable by the client device; as well as A graphical representation of the note including the user-generated note information and the at least one contextual suggestion is generated by the one or more processors via the system-level user interface module, the graphical representation being configured to be presented to a user of the client device.

14. The method of claim 13, wherein upon identifying to present the note via the other application, the method comprises sending note information including information about the at least one contextual suggestion to the other application.

15. The method of claim 13, wherein upon identifying to present the note via the another application, the method comprises transmitting note information including information regarding the at least one contextual suggestion to another client device associated with the user of the client device.

16. The method of claim 13, further comprising: receiving, while generating the graphical representation, input from the user of the client device for selection of the at least one contextual suggestion; accessing, by the one or more processors, information corresponding to the selection of the at least one contextual suggestion; as well as The information corresponding to the selection of the at least one contextual suggestion is presented to the user.

17. A computing device comprising: a memory configured to store system user interface information; as well as one or more processors operatively coupled to the memory, the one or more processors configured to: receiving an instruction to create a note from a user of the computing device; obtaining user-generated note information via a system-level user interface module of the computing device; identifying, via the system-level user interface module, one or more actions or events that occur on the computing device when the indication to create the note is received; generating, via the system-level user interface module, one or more contextual suggestions for the note, the one or more contextual suggestions being based on the one or more actions or events; creating, via the system-level user interface module, the note comprising the user-generated note information and at least a given contextual suggestion of the one or more contextual suggestions; as well as The note is stored in the memory.

18. The computing device of claim 17, wherein the one or more processors are further configured to present a visualization with the generated one or more contextual suggestions to the user via the system-level user interface module, or present a set of note-taking options to the user.

19. The computing device of claim 17, wherein generating the one or more context suggestions for the note comprises at least one of: generating a link to information about a meeting or event that was occurring when the indication to create the note was received; or Contextual information is generated regarding actions taken by the user within a predetermined amount of time before receiving the indication to create the note.

20. A computing device comprising: a memory configured to store system user interface information; as well as one or more processors operatively coupled to the memory, the one or more processors configured to: receiving an indication to access a note, the note comprising user-generated note information and at least one contextual suggestion, wherein the at least one contextual suggestion is based on one or more actions or events occurring on the computing device prior to creation of the note; retrieving the note from the memory via a system-level user interface module of the computing device; identifying whether the note is to be presented via a note-taking application integrated as part of a system-level user interface of the computing device or via another application executable by the computing device; as well as A graphical representation of the note including the user-generated note information and the at least one contextual suggestion is generated via the system-level user interface module, the graphical representation being configured to be presented to a user of the client device.

Images